llm-wiki-kit

LLM wiki kit

First committed by Ivy-Chen1999

Published in Ivy-Chen1999/llm-wiki-kit. Publisher page

20 GitHub starsUpdated 2026-06-23

What it is

Clone-and-go agent skills + a starter Obsidian vault for an AI-maintained personal wiki. Based on Karpathy's LLM Wiki pattern.

— the description by Ivy-Chen1999

AutoGineer overview, from the brain’s own files

llm-wiki-kit implements the LLM Wiki pattern, which shifts from traditional RAG (searching raw documents) to an agent-driven process that summarizes and structures information into linked Obsidian notes. By processing sources once, the system creates a persistent, queryable knowledge base. The kit includes 20 agent skills for tasks like ingesting URLs, mining conversation history, synthesizing notes, and maintaining wiki health, alongside a pre-configured Obsidian vault template.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

The system organizes data into three layers: raw sources, the wiki (markdown notes with YAML frontmatter), and agent skills. Users interact with an agent that loads these skills to perform actions. The vault uses a flat structure with specific folders for notes, raw staging, journals, and system files. Skills are linked to the agent's environment, allowing the agent to automatically process, link, and maintain the wiki based on natural language commands.

Folders
Folder	Role	Files
skills	agent-instructions	24
personal-skills	folder	4
vault-template	folder	12

The agent writes

skills/llm-wiki/references/karpathy-pattern.md evidence: README.md

Requirements

Apps: Obsidian

Set it up

This brain runs code automatically: 1 script. Review install.sh before opening the folder in Claude Code. No step below turns a hook on for you.
- install.sh: sh install.sh
Download the brain as a ZIP and unzip it where you keep your notes or project: https://mp-staging.autogineer.com/agent-brains/ivy-chen1999/llm-wiki-kit/download
cd llm-wiki-kit && claude
AutoGineer scanned these files at commit f511a35.
Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/Ivy-Chen1999/llm-wiki-kit/blob/f511a3520e1902a53c2776c209e6fc62d3cfa8da/README.md#setup
First committed by Ivy-Chen1999. Published in Ivy-Chen1999/llm-wiki-kit. Licence: MIT License.

Usage

AutoGineer overview, from the brain’s own files

Ingest articles, PDFs, or web pages into the wiki.
Query the wiki to retrieve information with source links.
Synthesize connections between existing notes.
Audit the wiki for orphans, broken links, or stale content.
Automate the capture of chat history from Claude or CLI sessions.
Maintain a controlled tag taxonomy and visual graph structure.

“Save this link to my wiki”: The agent fetches the URL, processes the content, and creates a structured note in the wiki.
“What do I know about X?”: The agent searches the wiki notes and provides an answer with links to the source material.
“Audit my wiki”: The agent checks for orphans, broken links, and contradictions, reporting on the health of the knowledge base.
“Color my graph by category”: The agent updates the Obsidian graph visualization based on the categories defined in the note frontmatter.

Documentation by Ivy-Chen1999

llm-wiki-kit

A set of agent skills and a starter Obsidian vault for keeping a personal wiki that an LLM helps maintain. Based on Andrej Karpathy's LLM Wiki pattern.

The idea: instead of searching raw documents every time you have a question (RAG), the agent processes a source once and writes it into structured, linked Obsidian notes. Questions are then answered from those notes. You pick the sources and ask the questions; the agent does the summarizing, linking, and filing.

How it's organized

There are three layers:

Raw sources — your original material (articles, PDFs, notes, screenshots). Never modified.
The wiki — markdown notes with YAML frontmatter and [[wikilinks]], kept in the vault.
The skills — the instruction files in skills/ that tell the agent how to maintain the wiki.

What's in this repo

llm-wiki-kit/
├── skills/          20 agent skills
├── vault-template/  an empty vault you can copy and start using
├── .env.example     config (vault path, sources, search)
├── install.sh       links the skills into your agent's skills directory
├── setup-qmd.sh     optional: installs and indexes local search
└── LICENSE

Skills

Skill	What it does
`llm-wiki`	Explains the pattern and retrieval approach. Start here.
`wiki-setup`	Initializes a new vault (structure, system files, config).
`wiki-ingest`	Turns a raw source into wiki notes.
`ingest-url`	Fetches a URL and ingests it.
`data-ingest`	Ingests structured/tabular data.
`claude-history-ingest`	Mines Claude conversation history into the wiki.
`codex-history-ingest`	Mines Codex CLI session history into the wiki.
`wiki-query`	Answers questions from the wiki, with links to source notes.
`wiki-research`	Runs web research and files the results into the wiki.
`wiki-update`	Revises existing notes as things change.
`wiki-synthesize`	Builds notes that connect several existing notes.
`cross-linker`	Finds and adds missing `[[wikilinks]]`.
`tag-taxonomy`	Keeps tags to a controlled list.
`wiki-lint`	Checks for orphans, broken links, contradictions, stale notes.
`wiki-status`	Reports what's new and what's ready to ingest.
`wiki-dashboard`	Generates a vault overview.
`wiki-rebuild`	Rebuilds the index and cache from the notes.
`wiki-export`	Exports the wiki to other formats.
`wiki-capture`	Drops rough notes into the staging area.
`graph-colorize`	Colors the Obsidian graph by category.

Requirements

Obsidian, to read and edit the vault.
An agent that loads skills from a directory — Claudian (the Obsidian + Claude plugin), Claude Code, or similar.
Optional: qmd, a local search engine (BM25 + vector). Without it, the skills use Grep instead, so it's not required. See setup-qmd.sh or run npm install -g @tobilu/qmd.

Setup

Two commands. The installer is interactive and does everything else (links the skills, asks where the vault should live, creates it, writes the config):

git clone https://github.com/Ivy-Chen1999/llm-wiki-kit.git
cd llm-wiki-kit
./install.sh

Then follow the final instructions it prints: open the vault folder in Obsidian, and ask your agent to ingest a source or answer a question.

Notes:

The installer links skills into ~/.claude/skills, which Claude (Claude Code and the Claudian Obsidian plugin) reads automatically. For another agent: ./install.sh --skills-dir <that agent's skills dir>.
It offers to install local search (qmd) at the end. Saying no is fine — the skills fall back to Grep. You can run ./setup-qmd.sh later instead.
Re-running is safe: existing skills, vault, and config are left untouched.

Working with more than one vault

The skills resolve the target vault in this order (highest first):

an OBSIDIAN_VAULT_PATH environment variable (per-session override),
a .env at the current working directory / vault root (vault-scoped),
~/.obsidian-wiki/config (the global default).

So to keep a vault self-contained, drop a .env at its root pointing OBSIDIAN_VAULT_PATH at itself (and give it its own QMD_WIKI_COLLECTION). Work from that vault and the skills target only it — the global default and your other vaults are untouched.

How to use it — just say what you want

You don't memorize skill names. Skills trigger automatically from what you say — Claude matches your message to the right one. Open your vault in Claude and talk normally:

You want to…	Just say something like…
Add an article / file / folder	"add this to my wiki", "ingest these docs"
Add a web page	"save this link to my wiki", paste a URL + "add this"
Ask what you know	"what do I know about X?", "find everything on Y"
Quick answer (no deep read)	"quick answer: …", "just scan for …"
Save the current chat	"save this conversation to my wiki"
Connect related notes	"find missing links", "my wiki feels disconnected"
Find cross-cutting themes	"what concepts keep coming up together?"
Check health / fix issues	"audit my wiki", "what needs fixing?"
See what's been ingested	"what's the status of my wiki?"
Research a topic from the web	"research X and file it", "deep dive on Y"
Tidy up tags	"clean up my tags", "normalize tags"
Color the graph	"color my graph by category"
Start fresh	"archive and rebuild the wiki"

If a phrasing doesn't trigger the right skill, just name it ("use wiki-ingest") — but normal language usually works.

Vault layout

The vault-template/ uses a flat layout: all notes live in notes/, and the category is set in frontmatter (category: concept|entity|reference|insight|synthesis).

my-wiki/
├── notes/        the notes
├── raw/          staging area for unprocessed sources
├── journal/      daily notes
├── Templates/    note and daily-note templates
└── _system/
    ├── index.md  table of contents (maintained by the skills)
    ├── hot.md    short summary of recent activity
    ├── log.md    append-only change log
    └── tags.md   tag list

If you prefer one folder per category, wiki-setup can generate that layout instead.

Make it your own

The 20 skills above are the shared core. The skills that pay off most are usually personal — shaped around how you work. You're encouraged to add your own:

a retro skill that writes a project post-mortem into the vault,
a digest skill that pulls your todos + field news into a dated entry,
your own ingest house-style, or a dispatcher that routes "/wiki do X".

personal-skills/ has a blank template and two worked examples (retro, digest) to copy and adapt. Personal skills live in ~/.claude/skills/, not in this repo — so updating the shared core (git pull) never touches them.

Credits

The pattern comes from Andrej Karpathy's LLM Wiki (the three-layer structure and the idea of processing a source once instead of re-searching it each time). His original write-up is included in skills/llm-wiki/references/karpathy-pattern.md.
Built for Obsidian.
The skills were taken from a personal vault and generalized: hardcoded paths, personal collection names, and non-English text removed.

License

MIT — see LICENSE.

skills/wiki-synthesize/SKILL.md

name: wiki-synthesize description: > Systematically discover synthesis opportunities across the Obsidian wiki — pairs or clusters of concepts that co-occur frequently across pages but have no synthesis page connecting them. Creates new synthesis/ pages that draw explicit cross-cutting conclusions. Use when the user says "synthesize my wiki", "find connections", "what concepts keep coming up together", "/wiki-synthesize", or after a large ingest when the vault has grown significantly.

Wiki Synthesize — First-Class Synthesis Discovery

You are scanning the wiki for concepts that co-occur across many pages but have no dedicated synthesis page connecting them. Your job is to surface these gaps and fill the most valuable ones with cross-cutting synthesis pages.

Before You Start

Resolve the vault path (precedence, highest first): the OBSIDIAN_VAULT_PATH environment variable if set, else a .env in the current working directory (vault-scoped), else ~/.obsidian-wiki/config (global default)..
Read index.md to get the full page inventory.
Read hot.md if it exists — it surfaces recent activity and active threads that may already point to synthesis opportunities.
Read _meta/taxonomy.md to understand the tag vocabulary.

Step 1: Build the Co-occurrence Map

Scan every non-special page in the vault (skip index.md, log.md, hot.md, _insights.md, _meta/*, _archives/*, _raw/*).

For each page, collect:

All [[wikilinks]] it contains (outgoing links)
Its tags frontmatter
Its category frontmatter

Build a co-occurrence matrix: for every pair of concept/entity pages (A, B), count how many other pages link to both A and B. This is their co-occurrence score.

You don't need to be exhaustive — aim for the top 20-30 pairs by co-occurrence score. Use Grep to find backlinks efficiently:

grep -rl "\[\[ConceptA\]\]" "$OBSIDIAN_VAULT_PATH" --include="*.md"

Run this for your top candidate concepts and intersect the result sets.

Step 2: Filter Out Already-Synthesized Pairs

Check the synthesis/ directory for existing pages. For each existing synthesis page:

Read its sources frontmatter or its body for [[wikilinks]]
Mark those concept pairs as already covered

Remove covered pairs from your candidate list.

Step 3: Score and Rank Candidates

For each remaining candidate pair (or cluster of 3+), assign a synthesis value score:

Signal	Points
Co-occurrence count ≥ 5	+3
Co-occurrence count 3-4	+2
Co-occurrence count 1-2	+1
Concepts are in different categories (cross-domain)	+2
Concepts share tags but live in different folders	+1
One or both concepts are tagged as hubs in `_insights.md`	+1
A synthesis would resolve a flagged contradiction	+2

Pick the top 5 candidates. If the user asked for a specific topic ("synthesize everything about observability"), filter candidates to that domain first.

Step 4: Draft Synthesis Pages

For each top candidate, create a page in synthesis/ using this template:

---
title: <Concept A> × <Concept B>
category: synthesis
tags: [<shared tags>, <domain tags>]
sources: [<all pages that link to both>]
created: TIMESTAMP
updated: TIMESTAMP
summary: "Cross-cutting synthesis of how <A> and <B> interact, with implications for <domain>."
provenance:
  extracted: 0.2
  inferred: 0.7
  ambiguous: 0.1
---

# <Concept A> × <Concept B>

## The Connection

*What makes these two concepts worth synthesizing together — the non-obvious relationship that pages about each individually don't capture.*

## Where They Co-occur

*The pages and contexts where both appear. What situations bring them together.*

## Cross-cutting Insight

*The conclusion that only becomes visible when you look at both together. This is the point of the page — the thing you couldn't see from either concept page alone.*

## Tensions and Trade-offs

*Where the two concepts pull in opposite directions. Unresolved contradictions. Cases where applying one undermines the other.*

## Open Questions

*What this synthesis surfaces that the wiki doesn't yet have an answer for. Good candidates for future research.*

## Related

- [[<Concept A>]]
- [[<Concept B>]]
- [[<other related pages>]]

Synthesis pages are mostly ^[inferred]. You are drawing connections across sources — that's synthesis by definition. Apply ^[inferred] to cross-cutting conclusions and ^[ambiguous] where sources disagree.

The title format is A × B — this signals to readers that it's a synthesis page, not a page about either concept alone.

Step 5: Back-link from Source Pages

For each synthesis page you created, add a link to it from the two (or more) concept pages it synthesizes. In the concept page, add to its ## Related section:

- [[Concept A × Concept B]] — synthesis

If the concept page has no ## Related section, add one at the bottom.

Step 6: Report Synthesis Opportunities Not Taken

After creating pages for the top 5, list the next 10 candidates in your output — pairs that scored well but you didn't write pages for. This gives the user visibility into what the wiki thinks is worth exploring without forcing every synthesis in one run.

Format:

Skipped (consider next time):
- [[Caching]] × [[Consistency]] — co-occurs in 4 pages, cross-domain
- [[Testing]] × [[Observability]] — co-occurs in 3 pages, shares tags
...

Step 7: Update Special Files

index.md — Add entries for all new synthesis pages.

log.md — Append:

- [TIMESTAMP] WIKI_SYNTHESIZE pages_scanned=N synthesis_created=M candidates_skipped=K

hot.md — Read $OBSIDIAN_VAULT_PATH/hot.md (create from the template in wiki-ingest if missing). Update Recent Activity with what was synthesized — e.g. "Synthesized 5 cross-cutting pages: Caching × Consistency, Testing × Observability, …". Update Active Threads with any open questions the synthesis surfaced. Update updated timestamp.

Quality Checklist

[ ] Every synthesis page has a summary: field (≤200 chars)
[ ] Every synthesis page links back to its source concepts
[ ] Source concept pages link forward to the synthesis page
[ ] No synthesis page just restates what's already on the source pages — it must add a cross-cutting insight
[ ] index.md and log.md updated
[ ] hot.md updated

Tips

A synthesis page that only summarizes its sources is useless. The value is the connection — the thing neither source page says explicitly.
Don't synthesize for synthesis's sake. If two concepts just happen to appear together a lot without a real conceptual link, skip them.
Three-way syntheses are powerful but rare. Only create them when three concepts form a genuine triangle of mutual influence — not just because all three appear in the same project page.
Check _insights.md first. The wiki-status skill may have already flagged synthesis candidates there — start with those before running the co-occurrence scan from scratch.

48 files

Advantages

Processes sources once into structured notes rather than performing repeated RAG searches. evidence: README.md
Supports custom personal skills that persist outside the shared core repository. evidence: README.md

Limitations

Requires an agent capable of loading skills from a directory, such as Claude Code or the Claudian plugin. evidence: README.md
Local search functionality requires an optional external dependency (qmd) to avoid falling back to Grep. evidence: README.md

Security

Risk level: Medium risk

This brain runs

Install scripts (1)

install.sh: sh install.sh

Agent instructions that run or fetch something (48)

personal-skills/digest.example/SKILL.md: Run web searches for the user's chosen topics (edit this list to your field).
personal-skills/retro.example/SKILL.md: `git log --oneline -20` (for the development arc)
skills/claude-history-ingest/SKILL.md: ├── audit.jsonl # Audit log — tool calls, file reads, commands run
skills/cross-linker/SKILL.md: **Run after every ingest.** New pages are almost always poorly connected.
skills/graph-colorize/SKILL.md: Tell the user to open it once, then re-run.
skills/graph-colorize/SKILL.md: **New category value appears later**: a category not in the mapping gets no entry and its nodes stay default gray — re-run to pick it up.
skills/graph-colorize/SKILL.md: Re-running is safe: each run creates a new backup, only `colorGroups` is rewritten.
skills/ingest-url/SKILL.md: Read `.manifest.json` to check if this URL was already ingested · uses the network
skills/ingest-url/SKILL.md: **Git remote name** — run `git remote get-url origin 2>/dev/null` from the current working directory.
skills/ingest-url/SKILL.md: **If available:** Use `defuddle <url>` (via Bash) to retrieve a clean, stripped-down markdown version of the page. · uses the network
skills/llm-wiki/SKILL.md: Put a `.env` at a vault's root (or run the skill from inside it) and that vault wins, regardless of the global default.
skills/llm-wiki/SKILL.md: **`~/.obsidian-wiki/config`** — the global default, written by `install.sh`.
skills/wiki-export/SKILL.md: Neo4j Cypher `MERGE` statements — paste into Neo4j Browser or run with `cypher-shell`:
skills/wiki-export/SKILL.md: **Re-running is safe** — all output files are overwritten on each run
skills/wiki-ingest/SKILL.md: If your model doesn't support vision, skip image sources and tell the user which files were skipped so they can re-run with a vision-capable model.
skills/wiki-lint/SKILL.md: Also verify after every lint run: are `_system/index.md` and `_system/hot.md` up to date?
skills/wiki-lint/SKILL.md: Run these checks in order.
skills/wiki-lint/SKILL.md: Run the `cross-linker` skill targeted at the fragmented tag — it will surface and insert the missing links
skills/wiki-lint/SKILL.md: Run the `cross-linker` skill first if affinity scores look stale (e.g., `affinity: {}` on a page with many wikilinks)
skills/wiki-lint/SKILL.md: For each pair, run a quick grep to count pages that link to both:
skills/wiki-lint/SKILL.md: Run `/wiki-synthesize` to automatically discover and fill the top gaps
skills/wiki-lint/SKILL.md: - **#systems** — 7 pages, cohesion=0.06 ⚠️ — run cross-linker on this tag
skills/wiki-lint/SKILL.md: | [[Caching]] × [[Consistency]] | 5 pages | Run `/wiki-synthesize` |
skills/wiki-lint/SKILL.md: | [[Testing]] × [[Observability]] | 3 pages | Run `/wiki-synthesize` |
skills/wiki-rebuild/SKILL.md: They can now run:
skills/wiki-research/SKILL.md: **Source Verification (mandatory before filing any note):** For every claim you intend to write into a wiki page, you must be able to point to a URL you successfully fetched with `WebFetch`. · uses the network
skills/wiki-research/SKILL.md: For each angle, run **2-3 `WebSearch` queries** using varied phrasing
skills/wiki-research/SKILL.md: For the top 2-3 results per angle, use `WebFetch` (or `defuddle <url>` if available — cleaner extraction) to get content · uses the network
skills/wiki-research/SKILL.md: Run **up to 5 targeted searches** specifically addressing these gaps.
skills/wiki-research/SKILL.md: Run one final targeted pass (2-3 searches) to find authoritative resolution
skills/wiki-setup/SKILL.md: Install instructions: see `.env.example` (QMD section).
skills/wiki-setup/SKILL.md: Tell the user about these recommended community plugins (they install manually):
skills/wiki-setup/SKILL.md: Run a quick sanity check:
skills/wiki-setup/SKILL.md: Run `wiki-status` to see what's available to ingest
skills/wiki-setup/SKILL.md: Run `wiki-ingest` to add their first sources
skills/wiki-setup/SKILL.md: Run `claude-history-ingest` to mine their Claude conversations
skills/wiki-setup/SKILL.md: Run `codex-history-ingest` to mine their Codex sessions (if they use Codex)
skills/wiki-setup/SKILL.md: Run `wiki-status` again anytime to check the delta
skills/wiki-status/SKILL.md: **Graph delta since last run.** Compare the current link graph to the snapshot stored in the previous `_insights.md`:
skills/wiki-status/SKILL.md: Flag: pages that were isolated last run but now have incoming links ("newly connected: X, Y")
skills/wiki-status/SKILL.md: Flag: pages that lost incoming links since last run ("link target may have been renamed: A, B")
skills/wiki-status/SKILL.md: At the very end, embed a compact graph snapshot as an HTML comment so the next run can diff against it.
skills/wiki-status/SKILL.md: - **#systems** — 7 pages, cohesion 0.06 ⚠️ run cross-linker on this tag
skills/wiki-status/SKILL.md: ## Graph Delta Since Last Run
skills/wiki-synthesize/SKILL.md: Run this for your top candidate concepts and intersect the result sets.
skills/wiki-synthesize/SKILL.md: This gives the user visibility into what the wiki thinks is worth exploring without forcing every synthesis in one run.
skills/wiki-update/SKILL.md: If `~/.obsidian-wiki/config` doesn't exist, tell the user to run `bash setup.sh` from their obsidian-wiki repo first.
skills/wiki-update/SKILL.md: Use `git log <last_commit>..HEAD --oneline` to see what's new.

Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:28)
Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:60)
Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:22)
Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:43)
Low risk: scripts-present (install.sh)