llm-wiki-kit

LLM wiki kit

First committed by Ivy-Chen1999

Published in Ivy-Chen1999/llm-wiki-kit. Publisher page

20 GitHub starsUpdated 2026-06-23

What it is

Clone-and-go agent skills + a starter Obsidian vault for an AI-maintained personal wiki. Based on Karpathy's LLM Wiki pattern.

— the description by Ivy-Chen1999

AutoGineer overview, from the brain’s own files

llm-wiki-kit implements the LLM Wiki pattern, which shifts from traditional RAG (searching raw documents) to an agent-driven process that summarizes and structures information into linked Obsidian notes. By processing sources once, the system creates a persistent, queryable knowledge base. The kit includes 20 agent skills for tasks like ingesting URLs, mining conversation history, synthesizing notes, and maintaining wiki health, alongside a pre-configured Obsidian vault template.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

The system organizes data into three layers: raw sources, the wiki (markdown notes with YAML frontmatter), and agent skills. Users interact with an agent that loads these skills to perform actions. The vault uses a flat structure with specific folders for notes, raw staging, journals, and system files. Skills are linked to the agent's environment, allowing the agent to automatically process, link, and maintain the wiki based on natural language commands.

Folders
Folder	Role	Files
skills	agent-instructions	24
personal-skills	folder	4
vault-template	folder	12

The agent writes

skills/llm-wiki/references/karpathy-pattern.md evidence: README.md

Requirements

Apps: Obsidian

Set it up

This brain runs code automatically: 1 script. Review install.sh before opening the folder in Claude Code. No step below turns a hook on for you.
- install.sh: sh install.sh
Download the brain as a ZIP and unzip it where you keep your notes or project: https://mp-staging.autogineer.com/agent-brains/ivy-chen1999/llm-wiki-kit/download
cd llm-wiki-kit && claude
AutoGineer scanned these files at commit f511a35.
Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/Ivy-Chen1999/llm-wiki-kit/blob/f511a3520e1902a53c2776c209e6fc62d3cfa8da/README.md#setup
First committed by Ivy-Chen1999. Published in Ivy-Chen1999/llm-wiki-kit. Licence: MIT License.

Usage

AutoGineer overview, from the brain’s own files

Ingest articles, PDFs, or web pages into the wiki.
Query the wiki to retrieve information with source links.
Synthesize connections between existing notes.
Audit the wiki for orphans, broken links, or stale content.
Automate the capture of chat history from Claude or CLI sessions.
Maintain a controlled tag taxonomy and visual graph structure.

“Save this link to my wiki”: The agent fetches the URL, processes the content, and creates a structured note in the wiki.
“What do I know about X?”: The agent searches the wiki notes and provides an answer with links to the source material.
“Audit my wiki”: The agent checks for orphans, broken links, and contradictions, reporting on the health of the knowledge base.
“Color my graph by category”: The agent updates the Obsidian graph visualization based on the categories defined in the note frontmatter.

Documentation by Ivy-Chen1999

llm-wiki-kit

A set of agent skills and a starter Obsidian vault for keeping a personal wiki that an LLM helps maintain. Based on Andrej Karpathy's LLM Wiki pattern.

The idea: instead of searching raw documents every time you have a question (RAG), the agent processes a source once and writes it into structured, linked Obsidian notes. Questions are then answered from those notes. You pick the sources and ask the questions; the agent does the summarizing, linking, and filing.

How it's organized

There are three layers:

Raw sources — your original material (articles, PDFs, notes, screenshots). Never modified.
The wiki — markdown notes with YAML frontmatter and [[wikilinks]], kept in the vault.
The skills — the instruction files in skills/ that tell the agent how to maintain the wiki.

What's in this repo

llm-wiki-kit/
├── skills/          20 agent skills
├── vault-template/  an empty vault you can copy and start using
├── .env.example     config (vault path, sources, search)
├── install.sh       links the skills into your agent's skills directory
├── setup-qmd.sh     optional: installs and indexes local search
└── LICENSE

Skills

Skill	What it does
`llm-wiki`	Explains the pattern and retrieval approach. Start here.
`wiki-setup`	Initializes a new vault (structure, system files, config).
`wiki-ingest`	Turns a raw source into wiki notes.
`ingest-url`	Fetches a URL and ingests it.
`data-ingest`	Ingests structured/tabular data.
`claude-history-ingest`	Mines Claude conversation history into the wiki.
`codex-history-ingest`	Mines Codex CLI session history into the wiki.
`wiki-query`	Answers questions from the wiki, with links to source notes.
`wiki-research`	Runs web research and files the results into the wiki.
`wiki-update`	Revises existing notes as things change.
`wiki-synthesize`	Builds notes that connect several existing notes.
`cross-linker`	Finds and adds missing `[[wikilinks]]`.
`tag-taxonomy`	Keeps tags to a controlled list.
`wiki-lint`	Checks for orphans, broken links, contradictions, stale notes.
`wiki-status`	Reports what's new and what's ready to ingest.
`wiki-dashboard`	Generates a vault overview.
`wiki-rebuild`	Rebuilds the index and cache from the notes.
`wiki-export`	Exports the wiki to other formats.
`wiki-capture`	Drops rough notes into the staging area.
`graph-colorize`	Colors the Obsidian graph by category.

Requirements

Obsidian, to read and edit the vault.
An agent that loads skills from a directory — Claudian (the Obsidian + Claude plugin), Claude Code, or similar.
Optional: qmd, a local search engine (BM25 + vector). Without it, the skills use Grep instead, so it's not required. See setup-qmd.sh or run npm install -g @tobilu/qmd.

Setup

Two commands. The installer is interactive and does everything else (links the skills, asks where the vault should live, creates it, writes the config):

git clone https://github.com/Ivy-Chen1999/llm-wiki-kit.git
cd llm-wiki-kit
./install.sh

Then follow the final instructions it prints: open the vault folder in Obsidian, and ask your agent to ingest a source or answer a question.

Notes:

The installer links skills into ~/.claude/skills, which Claude (Claude Code and the Claudian Obsidian plugin) reads automatically. For another agent: ./install.sh --skills-dir <that agent's skills dir>.
It offers to install local search (qmd) at the end. Saying no is fine — the skills fall back to Grep. You can run ./setup-qmd.sh later instead.
Re-running is safe: existing skills, vault, and config are left untouched.

Working with more than one vault

The skills resolve the target vault in this order (highest first):

an OBSIDIAN_VAULT_PATH environment variable (per-session override),
a .env at the current working directory / vault root (vault-scoped),
~/.obsidian-wiki/config (the global default).

So to keep a vault self-contained, drop a .env at its root pointing OBSIDIAN_VAULT_PATH at itself (and give it its own QMD_WIKI_COLLECTION). Work from that vault and the skills target only it — the global default and your other vaults are untouched.

How to use it — just say what you want

You don't memorize skill names. Skills trigger automatically from what you say — Claude matches your message to the right one. Open your vault in Claude and talk normally:

You want to…	Just say something like…
Add an article / file / folder	"add this to my wiki", "ingest these docs"
Add a web page	"save this link to my wiki", paste a URL + "add this"
Ask what you know	"what do I know about X?", "find everything on Y"
Quick answer (no deep read)	"quick answer: …", "just scan for …"
Save the current chat	"save this conversation to my wiki"
Connect related notes	"find missing links", "my wiki feels disconnected"
Find cross-cutting themes	"what concepts keep coming up together?"
Check health / fix issues	"audit my wiki", "what needs fixing?"
See what's been ingested	"what's the status of my wiki?"
Research a topic from the web	"research X and file it", "deep dive on Y"
Tidy up tags	"clean up my tags", "normalize tags"
Color the graph	"color my graph by category"
Start fresh	"archive and rebuild the wiki"

If a phrasing doesn't trigger the right skill, just name it ("use wiki-ingest") — but normal language usually works.

Vault layout

The vault-template/ uses a flat layout: all notes live in notes/, and the category is set in frontmatter (category: concept|entity|reference|insight|synthesis).

my-wiki/
├── notes/        the notes
├── raw/          staging area for unprocessed sources
├── journal/      daily notes
├── Templates/    note and daily-note templates
└── _system/
    ├── index.md  table of contents (maintained by the skills)
    ├── hot.md    short summary of recent activity
    ├── log.md    append-only change log
    └── tags.md   tag list

If you prefer one folder per category, wiki-setup can generate that layout instead.

Make it your own

The 20 skills above are the shared core. The skills that pay off most are usually personal — shaped around how you work. You're encouraged to add your own:

a retro skill that writes a project post-mortem into the vault,
a digest skill that pulls your todos + field news into a dated entry,
your own ingest house-style, or a dispatcher that routes "/wiki do X".

personal-skills/ has a blank template and two worked examples (retro, digest) to copy and adapt. Personal skills live in ~/.claude/skills/, not in this repo — so updating the shared core (git pull) never touches them.

Credits

The pattern comes from Andrej Karpathy's LLM Wiki (the three-layer structure and the idea of processing a source once instead of re-searching it each time). His original write-up is included in skills/llm-wiki/references/karpathy-pattern.md.
Built for Obsidian.
The skills were taken from a personal vault and generalized: hardcoded paths, personal collection names, and non-English text removed.

License

MIT — see LICENSE.

skills/wiki-research/SKILL.md

name: wiki-research description: > Autonomously research a topic via multi-round web search, synthesize findings, and file structured results into the Obsidian wiki. Use this skill when the user says "/wiki-research [topic]", "research X", "find everything about Y", "do a deep dive on Z", "autonomous research on X", or wants comprehensive, web-sourced knowledge on a topic filed directly into their wiki.

Wiki Research — Autonomous Multi-Round Research

You are running an autonomous research loop on a topic, synthesizing what you find, and filing the results into the Obsidian wiki as permanent knowledge.

Before You Start

Resolve the vault path (precedence, highest first): the OBSIDIAN_VAULT_PATH environment variable if set, else a .env in the current working directory (vault-scoped), else ~/.obsidian-wiki/config (global default). Set VAULT to that OBSIDIAN_VAULT_PATH.

Read $VAULT/_system/index.md to understand what's already in the wiki — don't re-research things the wiki covers well
Read $VAULT/_system/hot.md — surfaces recent context and active threads
Check $VAULT/_system/tags.md — use only the tags in the whitelist when writing new notes

Source Verification (mandatory before filing any note): For every claim you intend to write into a wiki page, you must be able to point to a URL you successfully fetched with WebFetch. Search snippets alone are not sufficient. Drop any claim whose source page cannot be verified.

Confirm the research topic with the user if it's ambiguous. Then proceed.

Research Configuration (optional)

If references/research-config.md exists in the vault, read it and apply any rules it defines:

Source preferences (e.g., prefer academic sources, avoid certain domains)
Domains to skip
Confidence scoring adjustments
Topic-specific constraints

If the file doesn't exist, proceed with defaults.

Round 1 — Broad Survey

Goal: Get a wide map of the topic.

Decompose the topic into 3-5 distinct angles (e.g., for "vector databases": what they are, when to use them, leading implementations, trade-offs, production gotchas)
For each angle, run 2-3 WebSearch queries using varied phrasing
For the top 2-3 results per angle, use WebFetch (or defuddle <url> if available — cleaner extraction) to get content
From each fetched page, extract:
- Key claims — what the source explicitly states
- Concepts — ideas, terms, frameworks introduced
- Entities — tools, people, organizations mentioned
- Contradictions — places where sources disagree with each other

Track what's covered and what's missing as you go.

Round 2 — Gap Fill

Goal: Close the holes left by Round 1.

Review what Round 1 produced:

What questions did sources raise but not answer?
Where do sources contradict each other?
Which angles got thin coverage?

Run up to 5 targeted searches specifically addressing these gaps. Prefer primary sources, official documentation, and authoritative analyses over link aggregators.

Add findings to your working set. Update the contradiction list.

Round 3 — Synthesis Check

Goal: Resolve contradictions; confirm depth is sufficient.

If major contradictions remain unresolved:

Run one final targeted pass (2-3 searches) to find authoritative resolution
If resolution is impossible, flag the contradiction explicitly in the synthesis page

If contradictions are minor or the topic feels well-covered after Round 2, skip additional searching and proceed to filing.

Halt condition: Stop when depth is achieved or 3 rounds are complete — do not loop indefinitely.

Filing — Write Wiki Pages

Organize all findings into wiki pages across four output areas:

1. sources/ — One page per major reference

For each significant source (typically 4-8 pages total):

---
title: >-
  <Source title>
category: references
tags: [<2-4 domain tags>]
sources:
  - "<URL>"
source_url: "<URL>"
created: <ISO-8601 timestamp>
updated: <ISO-8601 timestamp>
summary: >-
  <1-2 sentences describing what this source covers, ≤200 chars>
provenance:
  extracted: 0.X
  inferred: 0.X
  ambiguous: 0.X
---

Body: title, URL, what it covers, key claims (with provenance markers), limitations.

2. concepts/ — One page per substantive concept

For each significant concept surfaced across sources:

Standard concept frontmatter + body. Link concepts to each other and to source pages.

3. entities/ — Tools, organizations, people

For each significant entity encountered (tools, libraries, companies, key authors):

Standard entity frontmatter. Link back to concepts that use the entity and sources where it appears.

4. synthesis/Research: [Topic].md — Master synthesis

The primary output: a structured synthesis of everything found.

---
title: >-
  Research: <Topic>
category: synthesis
tags: [<3-5 domain tags>, research]
sources: [<list of source URLs or page paths>]
created: <ISO-8601 timestamp>
updated: <ISO-8601 timestamp>
summary: >-
  Synthesis of <N>-round research on <topic>. Covers <core findings in ≤200 chars>.
provenance:
  extracted: 0.X
  inferred: 0.X
  ambiguous: 0.X
---

# Research: <Topic>

## Overview
<2-4 sentence executive summary of what the research found>

## Key Findings
<Bulleted list of the most important claims, each with a [[source page]] citation>

## Core Concepts
<Links to concept pages created, with one-line descriptions>

## Entities & Tools
<Links to entity pages, with one-line descriptions>

## Contradictions & Open Questions
<Where sources disagree or where the research hit limits>

## Sources Consulted
<Linked list of all source pages>

Cross-linking

After filing all pages:

Every concept page should link to at least 2 source pages
Every source page should link to the concept pages it informed
The synthesis page should link to all concept, entity, and source pages produced

Check index.md for existing pages on the same topics — merge into existing pages rather than creating duplicates.

Update Tracking Files

.manifest.json — Add a research entry:

{
  "type": "research",
  "topic": "<topic>",
  "researched_at": "TIMESTAMP",
  "rounds_completed": 3,
  "sources_fetched": N,
  "pages_created": ["..."],
  "pages_updated": ["..."]
}

index.md — Add all new pages under their respective sections.

log.md — Append:

- [TIMESTAMP] WIKI_RESEARCH topic="<topic>" rounds=N sources_fetched=N pages_created=M

hot.md — Update Recent Activity with the research topic and core finding. Update Active Threads if this is ongoing. Update updated timestamp.

Quality Checklist

[ ] 3 rounds completed (or halted at sufficient depth)
[ ] Synthesis page exists at synthesis/Research: [Topic].md
[ ] Source pages written for major references
[ ] Concept and entity pages written for significant items
[ ] Contradictions flagged in synthesis page
[ ] All pages cross-linked
[ ] index.md, log.md, hot.md, .manifest.json updated

48 files

Advantages

Processes sources once into structured notes rather than performing repeated RAG searches. evidence: README.md
Supports custom personal skills that persist outside the shared core repository. evidence: README.md

Limitations

Requires an agent capable of loading skills from a directory, such as Claude Code or the Claudian plugin. evidence: README.md
Local search functionality requires an optional external dependency (qmd) to avoid falling back to Grep. evidence: README.md

Security

Risk level: Medium risk

This brain runs

Install scripts (1)

install.sh: sh install.sh

Agent instructions that run or fetch something (48)

personal-skills/digest.example/SKILL.md: Run web searches for the user's chosen topics (edit this list to your field).
personal-skills/retro.example/SKILL.md: `git log --oneline -20` (for the development arc)
skills/claude-history-ingest/SKILL.md: ├── audit.jsonl # Audit log — tool calls, file reads, commands run
skills/cross-linker/SKILL.md: **Run after every ingest.** New pages are almost always poorly connected.
skills/graph-colorize/SKILL.md: Tell the user to open it once, then re-run.
skills/graph-colorize/SKILL.md: **New category value appears later**: a category not in the mapping gets no entry and its nodes stay default gray — re-run to pick it up.
skills/graph-colorize/SKILL.md: Re-running is safe: each run creates a new backup, only `colorGroups` is rewritten.
skills/ingest-url/SKILL.md: Read `.manifest.json` to check if this URL was already ingested · uses the network
skills/ingest-url/SKILL.md: **Git remote name** — run `git remote get-url origin 2>/dev/null` from the current working directory.
skills/ingest-url/SKILL.md: **If available:** Use `defuddle <url>` (via Bash) to retrieve a clean, stripped-down markdown version of the page. · uses the network
skills/llm-wiki/SKILL.md: Put a `.env` at a vault's root (or run the skill from inside it) and that vault wins, regardless of the global default.
skills/llm-wiki/SKILL.md: **`~/.obsidian-wiki/config`** — the global default, written by `install.sh`.
skills/wiki-export/SKILL.md: Neo4j Cypher `MERGE` statements — paste into Neo4j Browser or run with `cypher-shell`:
skills/wiki-export/SKILL.md: **Re-running is safe** — all output files are overwritten on each run
skills/wiki-ingest/SKILL.md: If your model doesn't support vision, skip image sources and tell the user which files were skipped so they can re-run with a vision-capable model.
skills/wiki-lint/SKILL.md: Also verify after every lint run: are `_system/index.md` and `_system/hot.md` up to date?
skills/wiki-lint/SKILL.md: Run these checks in order.
skills/wiki-lint/SKILL.md: Run the `cross-linker` skill targeted at the fragmented tag — it will surface and insert the missing links
skills/wiki-lint/SKILL.md: Run the `cross-linker` skill first if affinity scores look stale (e.g., `affinity: {}` on a page with many wikilinks)
skills/wiki-lint/SKILL.md: For each pair, run a quick grep to count pages that link to both:
skills/wiki-lint/SKILL.md: Run `/wiki-synthesize` to automatically discover and fill the top gaps
skills/wiki-lint/SKILL.md: - **#systems** — 7 pages, cohesion=0.06 ⚠️ — run cross-linker on this tag
skills/wiki-lint/SKILL.md: | [[Caching]] × [[Consistency]] | 5 pages | Run `/wiki-synthesize` |
skills/wiki-lint/SKILL.md: | [[Testing]] × [[Observability]] | 3 pages | Run `/wiki-synthesize` |
skills/wiki-rebuild/SKILL.md: They can now run:
skills/wiki-research/SKILL.md: **Source Verification (mandatory before filing any note):** For every claim you intend to write into a wiki page, you must be able to point to a URL you successfully fetched with `WebFetch`. · uses the network
skills/wiki-research/SKILL.md: For each angle, run **2-3 `WebSearch` queries** using varied phrasing
skills/wiki-research/SKILL.md: For the top 2-3 results per angle, use `WebFetch` (or `defuddle <url>` if available — cleaner extraction) to get content · uses the network
skills/wiki-research/SKILL.md: Run **up to 5 targeted searches** specifically addressing these gaps.
skills/wiki-research/SKILL.md: Run one final targeted pass (2-3 searches) to find authoritative resolution
skills/wiki-setup/SKILL.md: Install instructions: see `.env.example` (QMD section).
skills/wiki-setup/SKILL.md: Tell the user about these recommended community plugins (they install manually):
skills/wiki-setup/SKILL.md: Run a quick sanity check:
skills/wiki-setup/SKILL.md: Run `wiki-status` to see what's available to ingest
skills/wiki-setup/SKILL.md: Run `wiki-ingest` to add their first sources
skills/wiki-setup/SKILL.md: Run `claude-history-ingest` to mine their Claude conversations
skills/wiki-setup/SKILL.md: Run `codex-history-ingest` to mine their Codex sessions (if they use Codex)
skills/wiki-setup/SKILL.md: Run `wiki-status` again anytime to check the delta
skills/wiki-status/SKILL.md: **Graph delta since last run.** Compare the current link graph to the snapshot stored in the previous `_insights.md`:
skills/wiki-status/SKILL.md: Flag: pages that were isolated last run but now have incoming links ("newly connected: X, Y")
skills/wiki-status/SKILL.md: Flag: pages that lost incoming links since last run ("link target may have been renamed: A, B")
skills/wiki-status/SKILL.md: At the very end, embed a compact graph snapshot as an HTML comment so the next run can diff against it.
skills/wiki-status/SKILL.md: - **#systems** — 7 pages, cohesion 0.06 ⚠️ run cross-linker on this tag
skills/wiki-status/SKILL.md: ## Graph Delta Since Last Run
skills/wiki-synthesize/SKILL.md: Run this for your top candidate concepts and intersect the result sets.
skills/wiki-synthesize/SKILL.md: This gives the user visibility into what the wiki thinks is worth exploring without forcing every synthesis in one run.
skills/wiki-update/SKILL.md: If `~/.obsidian-wiki/config` doesn't exist, tell the user to run `bash setup.sh` from their obsidian-wiki repo first.
skills/wiki-update/SKILL.md: Use `git log <last_commit>..HEAD --oneline` to see what's new.

Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:28)
Medium risk: runs-instruction-instruction-network (skills/ingest-url/SKILL.md:60)
Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:22)
Medium risk: runs-instruction-instruction-network (skills/wiki-research/SKILL.md:43)
Low risk: scripts-present (install.sh)