Generated banner for llm-wiki-kit
Generated logo for llm-wiki-kit

llm-wiki-kit

LLM wiki kit

First committed by Ivy-Chen1999

Published in Ivy-Chen1999/llm-wiki-kit. Publisher page

20 GitHub stars

What it is

Clone-and-go agent skills + a starter Obsidian vault for an AI-maintained personal wiki. Based on Karpathy's LLM Wiki pattern.
— the description by Ivy-Chen1999

AutoGineer overview, from the brain’s own files

llm-wiki-kit implements the LLM Wiki pattern, which shifts from traditional RAG (searching raw documents) to an agent-driven process that summarizes and structures information into linked Obsidian notes. By processing sources once, the system creates a persistent, queryable knowledge base. The kit includes 20 agent skills for tasks like ingesting URLs, mining conversation history, synthesizing notes, and maintaining wiki health, alongside a pre-configured Obsidian vault template.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

The system organizes data into three layers: raw sources, the wiki (markdown notes with YAML frontmatter), and agent skills. Users interact with an agent that loads these skills to perform actions. The vault uses a flat structure with specific folders for notes, raw staging, journals, and system files. Skills are linked to the agent's environment, allowing the agent to automatically process, link, and maintain the wiki based on natural language commands.

Folders
FolderRoleFiles
skillsagent-instructions24
personal-skillsfolder4
vault-templatefolder12

The agent writes

Requirements

Apps: Obsidian

Set it up

  1. This brain runs code automatically: 1 script. Review install.sh before opening the folder in Claude Code. No step below turns a hook on for you.

    • install.sh: sh install.sh
  2. Download the brain as a ZIP and unzip it where you keep your notes or project: https://mp-staging.autogineer.com/agent-brains/ivy-chen1999/llm-wiki-kit/download
  3. cd llm-wiki-kit && claude

  4. AutoGineer scanned these files at commit f511a35.

  5. Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/Ivy-Chen1999/llm-wiki-kit/blob/f511a3520e1902a53c2776c209e6fc62d3cfa8da/README.md#setup
  6. First committed by Ivy-Chen1999. Published in Ivy-Chen1999/llm-wiki-kit. Licence: MIT License.

Usage

AutoGineer overview, from the brain’s own files

  • Ingest articles, PDFs, or web pages into the wiki.
  • Query the wiki to retrieve information with source links.
  • Synthesize connections between existing notes.
  • Audit the wiki for orphans, broken links, or stale content.
  • Automate the capture of chat history from Claude or CLI sessions.
  • Maintain a controlled tag taxonomy and visual graph structure.
“Save this link to my wiki”
The agent fetches the URL, processes the content, and creates a structured note in the wiki.
“What do I know about X?”
The agent searches the wiki notes and provides an answer with links to the source material.
“Audit my wiki”
The agent checks for orphans, broken links, and contradictions, reporting on the health of the knowledge base.
“Color my graph by category”
The agent updates the Obsidian graph visualization based on the categories defined in the note frontmatter.

Documentation by Ivy-Chen1999

llm-wiki-kit

A set of agent skills and a starter Obsidian vault for keeping a personal wiki that an LLM helps maintain. Based on Andrej Karpathy's LLM Wiki pattern.

The idea: instead of searching raw documents every time you have a question (RAG), the agent processes a source once and writes it into structured, linked Obsidian notes. Questions are then answered from those notes. You pick the sources and ask the questions; the agent does the summarizing, linking, and filing.

How it's organized

There are three layers:

  1. Raw sources — your original material (articles, PDFs, notes, screenshots). Never modified.

  2. The wiki — markdown notes with YAML frontmatter and [[wikilinks]], kept in the vault.

  3. The skills — the instruction files in skills/ that tell the agent how to maintain the wiki.

What's in this repo

llm-wiki-kit/
├── skills/          20 agent skills
├── vault-template/  an empty vault you can copy and start using
├── .env.example     config (vault path, sources, search)
├── install.sh       links the skills into your agent's skills directory
├── setup-qmd.sh     optional: installs and indexes local search
└── LICENSE
Skills
SkillWhat it does
llm-wikiExplains the pattern and retrieval approach. Start here.
wiki-setupInitializes a new vault (structure, system files, config).
wiki-ingestTurns a raw source into wiki notes.
ingest-urlFetches a URL and ingests it.
data-ingestIngests structured/tabular data.
claude-history-ingestMines Claude conversation history into the wiki.
codex-history-ingestMines Codex CLI session history into the wiki.
wiki-queryAnswers questions from the wiki, with links to source notes.
wiki-researchRuns web research and files the results into the wiki.
wiki-updateRevises existing notes as things change.
wiki-synthesizeBuilds notes that connect several existing notes.
cross-linkerFinds and adds missing [[wikilinks]].
tag-taxonomyKeeps tags to a controlled list.
wiki-lintChecks for orphans, broken links, contradictions, stale notes.
wiki-statusReports what's new and what's ready to ingest.
wiki-dashboardGenerates a vault overview.
wiki-rebuildRebuilds the index and cache from the notes.
wiki-exportExports the wiki to other formats.
wiki-captureDrops rough notes into the staging area.
graph-colorizeColors the Obsidian graph by category.

Requirements

  • Obsidian, to read and edit the vault.

  • An agent that loads skills from a directory — Claudian (the Obsidian + Claude plugin), Claude Code, or similar.

  • Optional: qmd, a local search engine (BM25 + vector). Without it, the skills use Grep instead, so it's not required. See setup-qmd.sh or run npm install -g @tobilu/qmd.

Setup

Two commands. The installer is interactive and does everything else (links the skills, asks where the vault should live, creates it, writes the config):

git clone https://github.com/Ivy-Chen1999/llm-wiki-kit.git
cd llm-wiki-kit
./install.sh

Then follow the final instructions it prints: open the vault folder in Obsidian, and ask your agent to ingest a source or answer a question.

Notes:

  • The installer links skills into ~/.claude/skills, which Claude (Claude Code and the Claudian Obsidian plugin) reads automatically. For another agent: ./install.sh --skills-dir <that agent's skills dir>.

  • It offers to install local search (qmd) at the end. Saying no is fine — the skills fall back to Grep. You can run ./setup-qmd.sh later instead.

  • Re-running is safe: existing skills, vault, and config are left untouched.

Working with more than one vault

The skills resolve the target vault in this order (highest first):

  1. an OBSIDIAN_VAULT_PATH environment variable (per-session override),

  2. a .env at the current working directory / vault root (vault-scoped),

  3. ~/.obsidian-wiki/config (the global default).

So to keep a vault self-contained, drop a .env at its root pointing OBSIDIAN_VAULT_PATH at itself (and give it its own QMD_WIKI_COLLECTION). Work from that vault and the skills target only it — the global default and your other vaults are untouched.

How to use it — just say what you want

You don't memorize skill names. Skills trigger automatically from what you say — Claude matches your message to the right one. Open your vault in Claude and talk normally:

You want to…Just say something like…
Add an article / file / folder"add this to my wiki", "ingest these docs"
Add a web page"save this link to my wiki", paste a URL + "add this"
Ask what you know"what do I know about X?", "find everything on Y"
Quick answer (no deep read)"quick answer: …", "just scan for …"
Save the current chat"save this conversation to my wiki"
Connect related notes"find missing links", "my wiki feels disconnected"
Find cross-cutting themes"what concepts keep coming up together?"
Check health / fix issues"audit my wiki", "what needs fixing?"
See what's been ingested"what's the status of my wiki?"
Research a topic from the web"research X and file it", "deep dive on Y"
Tidy up tags"clean up my tags", "normalize tags"
Color the graph"color my graph by category"
Start fresh"archive and rebuild the wiki"

If a phrasing doesn't trigger the right skill, just name it ("use wiki-ingest") — but normal language usually works.

Vault layout

The vault-template/ uses a flat layout: all notes live in notes/, and the category is set in frontmatter (category: concept|entity|reference|insight|synthesis).

my-wiki/
├── notes/        the notes
├── raw/          staging area for unprocessed sources
├── journal/      daily notes
├── Templates/    note and daily-note templates
└── _system/
    ├── index.md  table of contents (maintained by the skills)
    ├── hot.md    short summary of recent activity
    ├── log.md    append-only change log
    └── tags.md   tag list

If you prefer one folder per category, wiki-setup can generate that layout instead.

Make it your own

The 20 skills above are the shared core. The skills that pay off most are usually personal — shaped around how you work. You're encouraged to add your own:

  • a retro skill that writes a project post-mortem into the vault,

  • a digest skill that pulls your todos + field news into a dated entry,

  • your own ingest house-style, or a dispatcher that routes "/wiki do X".

personal-skills/ has a blank template and two worked examples (retro, digest) to copy and adapt. Personal skills live in ~/.claude/skills/, not in this repo — so updating the shared core (git pull) never touches them.

Credits

  • The pattern comes from Andrej Karpathy's LLM Wiki (the three-layer structure and the idea of processing a source once instead of re-searching it each time). His original write-up is included in skills/llm-wiki/references/karpathy-pattern.md.

  • Built for Obsidian.

  • The skills were taken from a personal vault and generalized: hardcoded paths, personal collection names, and non-English text removed.

License

MIT — see LICENSE.

skills/cross-linker/SKILL.md


name: cross-linker description: > Scan the Obsidian wiki and automatically discover missing cross-references between pages. Use this skill when the user says "link my pages", "find missing links", "cross-reference", "connect my wiki", "add wikilinks", "what pages should be linked", or after any large ingestion to ensure new pages are woven into the existing knowledge graph. Also trigger when the user mentions "orphan pages" in the context of wanting to connect them, or says things like "my wiki feels disconnected" or "pages aren't linked well". This is a write-heavy skill — it actually modifies pages to add links, unlike wiki-lint which just reports issues.

Cross-Linker — Automated Wiki Cross-Referencing

You are weaving the wiki's knowledge graph tighter by finding and inserting missing [[wikilinks]] between pages that should reference each other but currently don't.

Follow the Retrieval Primitives table in llm-wiki/SKILL.md. Build the registry in Step 1 by grepping frontmatter only (not full pages). Reserve full Read for the unlinked-mention detection pass, and even there, only read pages whose summaries/titles make them plausible link targets. Blind full-vault reads are what this framework exists to avoid.

Before You Start

  1. Resolve the vault path (precedence, highest first): the OBSIDIAN_VAULT_PATH environment variable if set, else a .env in the current working directory (vault-scoped), else ~/.obsidian-wiki/config (global default).

  2. Read index.md to get the full inventory of pages and their one-line descriptions

  3. Skim log.md to see what was recently ingested (focus linking effort on new pages)

Step 1: Build the Page Registry

Glob all .md files in the vault (excluding _archives/, .obsidian/). For each page, extract:

  • Filename (without .md) — this is the wikilink target

  • Title from frontmatter

  • Aliases from frontmatter (if any)

  • Tags from frontmatter

  • Category from frontmatter or directory inference

  • One-line summary — first sentence or title field

Build a lookup table:

page_name → { path, title, aliases, tags, summary }

This is your "vocabulary" — every entry in this table is a valid wikilink target.

For each page in the vault:

  1. Read the full content

  2. Extract existing wikilinks — find all [[...]] references already present

  3. Search for unlinked mentions — check if the page's text contains any of these, without being wrapped in [[...]]:

    • Page filenames (e.g., the word "MyProject" appears but [[projects/my-project/my-project]] is missing)

    • Page titles from frontmatter

    • Aliases from frontmatter

    • Entity names, project names, concept names from the registry

  4. Check for semantic connections — pages that share multiple tags or are in the same project directory but don't link to each other

Matching Rules
  • Case-insensitive matching for names (e.g., "my-project" matches page MyProject)

  • Diacritic-insensitive matching — normalize both the page name and the body text with Unicode NFKD (decompose accented characters to base + combining marks, strip combining marks) before comparing. This ensures body text "Muller" matches page [[entities/müller]] and vice versa.

  • Skip self-references — a page shouldn't link to itself

  • Skip common words — don't link "the", "and", generic terms. Only match on distinctive names

  • Prefer the shortest unambiguous wikilink path — use [[page-name]] not [[full/path/to/page-name]] when the name is unique across the vault

  • Don't link inside code blocks or frontmatter

  • Don't double-link — if [[foo]] already appears on the page, don't add another

Step 3: Score and Rank Suggestions

Not every possible link is worth adding. Score each candidate using a composite signal, then tag it with a confidence label.

Scoring
SignalPointsExample
Exact name match in text+4"MyProject" appears in body text → link to my-project.md
Shared tags (2+)+2Both tagged #ai #agent but no link between them
Same project, no link+2Both under projects/my-project/ but don't reference each other
Mentioned entity/concept+2Page mentions "knowledge graphs" → link to [[concepts/knowledge-graphs]]
Cross-category connection+2Source is in concepts/, target is in entities/ (or skills/ ↔ synthesis/) — different knowledge layers make this link more architecturally valuable
Peripheral→hub reach+2Source page has ≤ 2 total links (peripheral) but target has ≥ 8 (hub) — connecting a loose page to a load-bearing concept
Partial name match+1"graph" appears but page is knowledge-graphs — plausible but ambiguous
Confidence labels

Tag each candidate with a confidence label based on its score:

ScoreLabelAction
≥ 6EXTRACTEDLink is effectively certain — exact mention or very strong match. Apply inline.
3–5INFERREDLink is a reasonable inference — shared context, cross-category, peripheral→hub. Apply inline or as Related section.
1–2AMBIGUOUSWeak or partial match. Skip unless user specifically asks to connect loose pages.

Only act on EXTRACTED and INFERRED candidates. Include the confidence label in the Cross-Link Report so the user can review INFERRED links before trusting them.

For each page with missing links:

4a: Inline linking (preferred)

Find the first natural mention of the term in the body text and wrap it in wikilinks:

Before:

This project uses knowledge graphs to connect entities.

After:

This project uses [[concepts/knowledge-graphs|knowledge graphs]] to connect entities.

Use the [[path|display text]] format when the wikilink path differs from the display text.

If the term isn't mentioned naturally in the body but the pages are semantically related (shared tags, same project), add a ## Related section at the bottom of the page:

## Related

- [[projects/my-project/my-project]] — Also uses AI agents for research automation
- [[concepts/knowledge-graphs]] — Core technique used in this project

If a ## Related section already exists, append to it. Don't duplicate existing entries.

Step 5: Score Misc Page Affinity

After the main linking pass, update affinity scores for all pages in misc/ (pages with promotion_status: misc in their frontmatter, or located under the misc/ directory).

For each misc page:

  1. Collect outgoing links — all [[wikilinks]] in the page body

  2. Collect incoming links — grep the vault for [[misc/<slug>]] and [[<slug>]] references

  3. For each linked page (both directions), check if it belongs to a project:

    • Lives under projects/<project-name>/

    • Has a project: frontmatter field matching a project name

  4. Group by project name and sum: outgoing_links + incoming_links

  5. Update the affinity frontmatter block on the misc page:

affinity:
  obsidian-wiki: 3
  another-project: 1
  1. If any project's score ≥ 3: flag this page as a promotion candidate and record it for the report

Efficiency note: only read the full body of misc pages — other pages only need a frontmatter grep to determine their project membership.

Step 6: Report

Present a summary:

## Cross-Link Report

### Links Added: 23 across 12 pages

| Page | Links Added | Confidence | Type |
|---|---|---|---|
| `projects/my-project/my-project.md` | 3 | EXTRACTED | 2 inline, 1 related |
| `entities/jane-doe.md` | 5 | INFERRED | 3 inline, 2 related |
| ... | | | |

### Orphan Pages Remaining: 2
- `references/foo.md` — no incoming or outgoing links found
- `concepts/bar.md` — could not find related pages

### Misc Promotion Candidates: N
Pages in misc/ that have ≥ 3 connections to a single project — ready to be promoted:

| Page | Top Project | Score |
|---|---|---|
| `misc/web-martinfowler-articles-microservices.md` | `obsidian-wiki` | 4 |

To promote: move the page to `projects/<project-name>/references/` and update all backlinks.

### Pages Skipped: 3
- `index.md`, `log.md` — special files
- `_archives/*` — archived content

Step 7: Update Log and Hot Cache

Append to log.md:

- [TIMESTAMP] CROSS_LINK pages_scanned=N links_added=M pages_modified=P orphans_remaining=Q misc_affinity_updated=R promotion_candidates=S

hot.md — Read $OBSIDIAN_VAULT_PATH/hot.md (create from the template in wiki-ingest if missing). Update Recent Activity with a one-line summary of what was linked — e.g. "Cross-linked 23 mentions across 12 pages; 2 orphans remain." Keep the last 3 operations. Update updated timestamp.

Tips

  • Run after every ingest. New pages are almost always poorly connected. This is the fix.

  • Be conservative with inline links. Only link the first natural mention, not every occurrence.

  • Don't touch pages in _archives/. Those are frozen snapshots.

  • Respect existing structure. If a page carefully curates its links in a ## Key Concepts section, add to that section rather than creating a separate ## Related.

  • Entity pages are link magnets. An entity like jane-doe should be linked from almost every project page. Prioritize these.

48 files

Advantages

  • Processes sources once into structured notes rather than performing repeated RAG searches. evidence: README.md
  • Supports custom personal skills that persist outside the shared core repository. evidence: README.md

Limitations

  • Requires an agent capable of loading skills from a directory, such as Claude Code or the Claudian plugin. evidence: README.md
  • Local search functionality requires an optional external dependency (qmd) to avoid falling back to Grep. evidence: README.md

Security

Risk level: Medium risk

This brain runs

Install scripts (1)

  • install.sh: sh install.sh

Agent instructions that run or fetch something (48)

  • personal-skills/digest.example/SKILL.md: Run web searches for the user's chosen topics (edit this list to your field).
  • personal-skills/retro.example/SKILL.md: `git log --oneline -20` (for the development arc)
  • skills/claude-history-ingest/SKILL.md: ├── audit.jsonl # Audit log — tool calls, file reads, commands run
  • skills/cross-linker/SKILL.md: **Run after every ingest.** New pages are almost always poorly connected.
  • skills/graph-colorize/SKILL.md: Tell the user to open it once, then re-run.
  • skills/graph-colorize/SKILL.md: **New category value appears later**: a category not in the mapping gets no entry and its nodes stay default gray — re-run to pick it up.
  • skills/graph-colorize/SKILL.md: Re-running is safe: each run creates a new backup, only `colorGroups` is rewritten.
  • skills/ingest-url/SKILL.md: Read `.manifest.json` to check if this URL was already ingested · uses the network
  • skills/ingest-url/SKILL.md: **Git remote name** — run `git remote get-url origin 2>/dev/null` from the current working directory.
  • skills/ingest-url/SKILL.md: **If available:** Use `defuddle <url>` (via Bash) to retrieve a clean, stripped-down markdown version of the page. · uses the network
  • skills/llm-wiki/SKILL.md: Put a `.env` at a vault's root (or run the skill from inside it) and that vault wins, regardless of the global default.
  • skills/llm-wiki/SKILL.md: **`~/.obsidian-wiki/config`** — the global default, written by `install.sh`.
  • skills/wiki-export/SKILL.md: Neo4j Cypher `MERGE` statements — paste into Neo4j Browser or run with `cypher-shell`:
  • skills/wiki-export/SKILL.md: **Re-running is safe** — all output files are overwritten on each run
  • skills/wiki-ingest/SKILL.md: If your model doesn't support vision, skip image sources and tell the user which files were skipped so they can re-run with a vision-capable model.
  • skills/wiki-lint/SKILL.md: Also verify after every lint run: are `_system/index.md` and `_system/hot.md` up to date?
  • skills/wiki-lint/SKILL.md: Run these checks in order.
  • skills/wiki-lint/SKILL.md: Run the `cross-linker` skill targeted at the fragmented tag — it will surface and insert the missing links
  • skills/wiki-lint/SKILL.md: Run the `cross-linker` skill first if affinity scores look stale (e.g., `affinity: {}` on a page with many wikilinks)
  • skills/wiki-lint/SKILL.md: For each pair, run a quick grep to count pages that link to both:
  • skills/wiki-lint/SKILL.md: Run `/wiki-synthesize` to automatically discover and fill the top gaps
  • skills/wiki-lint/SKILL.md: - **#systems** — 7 pages, cohesion=0.06 ⚠️ — run cross-linker on this tag
  • skills/wiki-lint/SKILL.md: | [[Caching]] × [[Consistency]] | 5 pages | Run `/wiki-synthesize` |
  • skills/wiki-lint/SKILL.md: | [[Testing]] × [[Observability]] | 3 pages | Run `/wiki-synthesize` |
  • skills/wiki-rebuild/SKILL.md: They can now run:
  • skills/wiki-research/SKILL.md: **Source Verification (mandatory before filing any note):** For every claim you intend to write into a wiki page, you must be able to point to a URL you successfully fetched with `WebFetch`. · uses the network
  • skills/wiki-research/SKILL.md: For each angle, run **2-3 `WebSearch` queries** using varied phrasing
  • skills/wiki-research/SKILL.md: For the top 2-3 results per angle, use `WebFetch` (or `defuddle <url>` if available — cleaner extraction) to get content · uses the network
  • skills/wiki-research/SKILL.md: Run **up to 5 targeted searches** specifically addressing these gaps.
  • skills/wiki-research/SKILL.md: Run one final targeted pass (2-3 searches) to find authoritative resolution
  • skills/wiki-setup/SKILL.md: Install instructions: see `.env.example` (QMD section).
  • skills/wiki-setup/SKILL.md: Tell the user about these recommended community plugins (they install manually):
  • skills/wiki-setup/SKILL.md: Run a quick sanity check:
  • skills/wiki-setup/SKILL.md: Run `wiki-status` to see what's available to ingest
  • skills/wiki-setup/SKILL.md: Run `wiki-ingest` to add their first sources
  • skills/wiki-setup/SKILL.md: Run `claude-history-ingest` to mine their Claude conversations
  • skills/wiki-setup/SKILL.md: Run `codex-history-ingest` to mine their Codex sessions (if they use Codex)
  • skills/wiki-setup/SKILL.md: Run `wiki-status` again anytime to check the delta
  • skills/wiki-status/SKILL.md: **Graph delta since last run.** Compare the current link graph to the snapshot stored in the previous `_insights.md`:
  • skills/wiki-status/SKILL.md: Flag: pages that were isolated last run but now have incoming links ("newly connected: X, Y")
  • skills/wiki-status/SKILL.md: Flag: pages that lost incoming links since last run ("link target may have been renamed: A, B")
  • skills/wiki-status/SKILL.md: At the very end, embed a compact graph snapshot as an HTML comment so the next run can diff against it.
  • skills/wiki-status/SKILL.md: - **#systems** — 7 pages, cohesion 0.06 ⚠️ run cross-linker on this tag
  • skills/wiki-status/SKILL.md: ## Graph Delta Since Last Run
  • skills/wiki-synthesize/SKILL.md: Run this for your top candidate concepts and intersect the result sets.
  • skills/wiki-synthesize/SKILL.md: This gives the user visibility into what the wiki thinks is worth exploring without forcing every synthesis in one run.
  • skills/wiki-update/SKILL.md: If `~/.obsidian-wiki/config` doesn't exist, tell the user to run `bash setup.sh` from their obsidian-wiki repo first.
  • skills/wiki-update/SKILL.md: Use `git log <last_commit>..HEAD --oneline` to see what's new.
  • Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:28)
  • Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:60)
  • Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:22)
  • Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:43)
  • Low risk: scripts-present (install.sh)

Attribution and licence

First committed by
GitHub avatar of Ivy-Chen1999Ivy-Chen1999
Publisher
Yuxin (Ivy) Chen (Community)
Source
Ivy-Chen1999/llm-wiki-kit · commit f511a35
Licence
MIT License (files served here)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
Ivy-Chen1999

AutoGineer is not affiliated with or endorsed by Ivy-Chen1999.

Download

The ZIP holds the files AutoGineer serves, with ATTRIBUTION.md and the licence. Download llm-wiki-kit.zip