AutoGineer
Generated banner for LLM Knowledge Bases
Generated logo for LLM Knowledge Bases

LLM Knowledge Bases

LLM wiki kit

First committed by rvk7895

Published in rvk7895/llm-knowledge-bases. Publisher page

35 GitHub stars

What it is

Claude Code plugin for building LLM-maintained Obsidian wikis from raw research — compile, query, lint, and evolve your personal knowledge base. Inspired by Karpathy's knowledge base workflow.
— the description by rvk7895

AutoGineer overview, from the brain’s own files

LLM Knowledge Bases is a plugin for Claude Code designed to automate the creation and maintenance of a personal knowledge base in Obsidian. By treating the LLM as a knowledge compiler, the system ingests raw materials—such as articles, papers, and transcripts—and organizes them into a structured, interlinked wiki. Users interact with the system by feeding raw data into a designated folder, while the agent handles the compilation, indexing, and health maintenance of the vault. It supports multiple query depths, ranging from quick index lookups to deep, multi-agent research pipelines, ensuring that the knowledge base evolves alongside the user's research interests.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

The system operates by ingesting raw source documents from a raw folder. It then compiles these into an Obsidian vault containing summaries, backlinks, and indexes. Users can query the knowledge base at three levels of depth: quick, standard, or deep. The system also performs automated maintenance tasks like linting for broken links or orphans and provides tools for deep research, which involves generating outlines and deploying parallel agents to research specific topics. All outputs are saved to an output folder or integrated back into the wiki.

Folders
FolderRoleFiles
.claudeagent-config1
docsdocs2
.claude-pluginfolder1
pluginsfolder12

Requirements

Works with: Claude Code, OpenCode

Set it up

  1. This brain runs code automatically: 1 script. Review plugins/kb/setup.sh before opening the folder in Claude Code. No step below turns a hook on for you.

    • plugins/kb/setup.sh: sh plugins/kb/setup.sh
  2. Download the brain as a ZIP and unzip it where you keep your notes or project: https://mp-staging.autogineer.com/agent-brains/rvk7895/llm-knowledge-bases/download

  3. Claude Code reads CLAUDE.md from this folder automatically.

  4. cd llm-knowledge-bases && claude

  5. AutoGineer scanned these files at commit 342686f.

  6. Not scanned by AutoGineerThe author’s README also describes installing it as a Claude Code plugin marketplace. Read its install section: AutoGineer did not scan that path. https://github.com/rvk7895/llm-knowledge-bases/blob/342686f4cdf0e969872601eabbcc10466da91431/README.md#installation

  7. First committed by rvk7895. Published in rvk7895/llm-knowledge-bases. Licence: MIT License.

Usage

AutoGineer overview, from the brain’s own files

  • Initialize a new knowledge base structure
  • Compile raw research documents into an Obsidian wiki
  • Query the knowledge base for information at varying depths
  • Perform automated health checks on the wiki
  • Execute deep research on specific topics using multi-agent pipelines
“/kb compile”
The agent processes all files in the raw/ directory and updates the Obsidian wiki with new summaries and backlinks.
“/research-deep Artificial Intelligence trends”
The agent generates a research outline and launches parallel agents to conduct deep research on each item in the outline.
“/kb lint”
The agent scans the wiki for broken links, orphans, and inconsistencies, reporting them for resolution.

Documentation by rvk7895

LLM Knowledge Bases

A Claude Code plugin that turns raw research material into an LLM-maintained Obsidian wiki -- inspired by Andrej Karpathy's description of using LLMs as knowledge compilers rather than just code manipulators.

You drop source material into raw/, run a single command, and Claude handles the rest: compiling interlinked wiki articles, maintaining indexes and backlinks, answering complex questions at multiple depth levels, and continuously improving the knowledge base over time.

The LLM owns the wiki. You rarely edit it manually -- just explore in Obsidian and keep feeding it raw data.

How It Works

  1. Ingest -- Raw documents (articles, papers, repos, YouTube transcripts, images, datasets) go into raw/

  2. Compile -- Claude builds a structured Obsidian vault with summaries, backlinks, concept articles, and auto-generated indexes

  3. Query -- Three depth levels:

    • Quick -- Answers from wiki indexes and summaries alone

    • Standard -- Cross-references the full wiki, supplements with web search

    • Deep -- Multi-agent research pipeline with parallel web search agents

  4. Output -- Markdown reports, Marp slides, matplotlib charts -- saved to output/ and optionally filed back into the wiki

  5. Maintain -- Automated health checks (broken links, orphans, inconsistencies) and suggestions for new articles and connections

Prerequisites

Obsidian plugins (Dataview, Obsidian Git, etc.) are installed automatically by the setup script during /kb-init.

Installation

In Claude Code, run:

/plugin marketplace add rvk7895/llm-knowledge-bases
/plugin install kb@llm-knowledge-bases

All skills are installed as a single plugin.

Quick Start

# Initialize a new knowledge base
/kb-init

# Add sources to raw/, then compile the wiki
/kb compile

# Query the knowledge base
/kb query

# Run a health check
/kb lint

# Deep multi-agent research on a topic
/research <topic>
/research-deep

Available Skills

SkillPurpose
kb-initOne-time setup: scaffolds directories, generates config, writes project files
kbMain skill with four workflows: compile, query, lint, evolve
researchGenerate a structured research outline for a topic
research-deepLaunch parallel agents for deep research on each outline item
research-add-fieldsAdd field definitions to an existing research outline
research-add-itemsAdd research targets to an existing outline
research-reportCompile deep research results into a markdown report

Directory Structure (after /kb-init)

raw/           -- Raw source documents
wiki/          -- LLM-compiled Obsidian vault
output/        -- Query results, slides, charts, reports
kb.yaml        -- Configuration
CLAUDE.md      -- Project instructions for Claude

X/Twitter Integration (Optional)

To ingest tweets, threads, and bookmarks from X, install Smaug:

npm install -g @steipete/bird
git clone https://github.com/alexknowshtml/smaug && cd smaug && npm install
npx smaug setup  # configures X session cookies

Once configured, paste any x.com link and the kb skill will fetch and compile it automatically. Without Smaug, you can still add X content by pasting tweet text directly or using Thread Reader App for threads.

Note: Smaug uses X session cookies for read-only access to your own data. This is not officially supported by X's TOS. Practical risk for personal use is very low, but be aware.

Attribution

  • Andrej Karpathy -- Original vision for LLM-maintained knowledge bases

  • Weizhena -- Deep Research skills adapted for the research pipeline

plugins/kb/skills/research-deep/SKILL.md

user-invocable: true description: Read research outline and launch independent agents for in-depth research on each item. Task output disabled. allowed-tools: Bash, Read, Write, Glob, WebSearch, Task

Attribution: Originally authored by Weizhena. Included with attribution for use in the Deep query workflow.

Research Deep - Deep Research

Trigger Method

/research-deep

Execution Flow

Step 1: Auto-locate Outline

Find */outline.yaml file in current working directory, read items list and execution configuration (including items_per_agent).

Step 2: Resume From Checkpoint

  • Check completed JSON files in output_dir

  • Skip already completed items

Step 3: Batch Execution

  • Execute in batches by batch_size (need user approval to proceed to next batch after completing one)

  • Each agent handles items_per_agent projects

  • Launch web-search-agent (background parallel, task output disabled)

Parameter Retrieval:

  • {topic}: topic field from outline.yaml

  • {item_name}: name field of item

  • {item_related_info}: complete yaml content of item (name + category + description etc.)

  • {output_dir}: execution.output_dir from outline.yaml (default ./results)

  • {fields_path}: absolute path to {topic}/fields.yaml

  • {output_path}: absolute path to {output_dir}/{item_name_slug}.json (slugify item_name: replace spaces with _, remove special characters)

Hard Constraint: The following prompt must be strictly recited, only replace variables in {xxx}, do not rewrite structure or wording.

Prompt Template:

prompt = f"""## Task
Research {item_related_info}, output structured JSON to {output_path}

## Field Definitions
Read {fields_path} to get all field definitions

## Output Requirements
1. Output JSON according to fields defined in fields.yaml
2. Mark uncertain field values as [不确定]
3. Add uncertain array at end of JSON, listing all uncertain field names
4. All field values must be output in Chinese (research process can use English, but final JSON values in Chinese)

## Output Path
{output_path}

## Validation
After completing JSON output, run validation script to ensure complete field coverage:
python ~/.claude/skills/research/validate_json.py -f {fields_path} -j {output_path}
Task is only complete after validation passes.
"""

One-shot Example (assuming research on GitHub Copilot):

## Task
Research name: GitHub Copilot
category: International Product
description: Developed by Microsoft/GitHub, first mainstream AI programming assistant, market share about 40%, output structured JSON to /home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json

## Field Definitions
Read /home/weizhena/AIcoding/aicoding-history/fields.yaml to get all field definitions

## Output Requirements
1. Output JSON according to fields defined in fields.yaml
2. Mark uncertain field values as [不确定]
3. Add uncertain array at end of JSON, listing all uncertain field names
4. All field values must be output in Chinese (research process can use English, but final JSON values in Chinese)

## Output Path
/home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json

## Validation
After completing JSON output, run validation script to ensure complete field coverage:
python ~/.claude/skills/research/validate_json.py -f /home/weizhena/AIcoding/aicoding-history/fields.yaml -j /home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json
Task is only complete after validation passes.
Step 4: Wait and Monitor

  • Wait for current batch to complete

  • Launch next batch

  • Show progress

Step 5: Summary Report

After all completion, output:

  • Number of completed items

  • Failed/uncertain marked items

  • Output directory

Agent Configuration

  • Background execution: Yes

  • Task Output: Disabled (agent has clear output file when completed)

  • Resume from checkpoint: Yes

21 files

Advantages

  • Automates the maintenance of interlinked wiki articles and indexes, reducing manual effort. evidence: README.md
  • Supports multi-agent research pipelines for deep, parallel investigation of topics. evidence: README.md

Limitations

  • Requires Claude Code CLI and Obsidian to function. evidence: README.md
  • The LLM owns the wiki, and manual editing of articles outside of the provided workflows is discouraged. evidence: CLAUDE.md

Security

Risk level: Medium risk

This brain runs

Install scripts (1)

  • plugins/kb/setup.sh: sh plugins/kb/setup.sh · uses the network

Agent instructions that run or fetch something (44)

  • plugins/kb/skills/kb-init/SKILL.md: After confirming the vault path, run the bundled setup script via Bash tool:
  • plugins/kb/skills/kb-init/SKILL.md: The script is idempotent — safe to re-run.
  • plugins/kb/skills/kb-init/SKILL.md: python -c "import pptx" 2>/dev/null && echo "python-pptx: available" || echo "python-pptx: not installed (needed for slide extraction)"
  • plugins/kb/skills/kb-init/SKILL.md: python -c "import pandas" 2>/dev/null && echo "pandas: available" || echo "pandas: not installed (needed for dataset analysis)"
  • plugins/kb/skills/kb-init/SKILL.md: When the user sets up Smaug (during init or later), save the install path here.
  • plugins/kb/skills/kb-preferences/SKILL.md: If missing, tell the user to run `kb-init` and stop.
  • plugins/kb/skills/kb-preferences/SKILL.md: | Invocation | Mode | Action | |---|---|---| | `/kb-preferences` (bare) | interactive | Show current prefs, ask what to update | | `/kb-preferences init` | init | Re-run the 5 init questions, regenerate all fields from style-guide + user answers | | `/kb-preferences update <field>` | up …[+30 chars]
  • plugins/kb/skills/kb-preferences/SKILL.md: re-ask matching question or accept free-text | | `/kb-preferences add-note "..."` | add-note | Append the quoted string as a new line/paragraph under `notes:` | | `/kb-preferences reflect` | reflect-now | Run the reflection step against the current conversation explicitly | | `/kb-pref …[+164 chars]
  • plugins/kb/skills/kb-preferences/SKILL.md: If missing, tell user to run `/kb-preferences init` first and stop.
  • plugins/kb/skills/kb-preferences/SKILL.md: Confirm loudly: "This will replace all 9 fields with factory defaults and re-run the 5 init questions.
  • plugins/kb/skills/kb/SKILL.md: If missing, tell the user to run `kb-init` and stop.
  • plugins/kb/skills/kb/SKILL.md: Run `/kb-preferences init` to customize."
  • plugins/kb/skills/kb/SKILL.md: After a compile or query finishes writing its output, run a **lightweight reflection step** — only propose preference updates for **high-confidence signals**:
  • plugins/kb/skills/kb/SKILL.md: The user can run `/kb-preferences reflect` manually if they want a forced pass.
  • plugins/kb/skills/kb/SKILL.md: Run lightweight reflection at the end.
  • plugins/kb/skills/kb/SKILL.md: **Detection heuristic:** Run this check for each file:
  • plugins/kb/skills/kb/SKILL.md: Please provide a transcript, or install: pip install openai-whisper"
  • plugins/kb/skills/kb/SKILL.md: Try: `yt-dlp --write-auto-sub --sub-lang en --skip-download --output "/tmp/%(id)s" "<url>"` to get subtitles · uses the network
  • plugins/kb/skills/kb/SKILL.md: Try extracting with python-pptx: `python -c "from pptx import Presentation;
  • plugins/kb/skills/kb/SKILL.md: For Parquet: `python -c "import pandas as pd;
  • plugins/kb/skills/kb/SKILL.md: Run lightweight reflection at the end (after the answer is delivered;
  • plugins/kb/skills/kb/SKILL.md: Run the appropriate pipeline.
  • plugins/kb/skills/kb/SKILL.md: Run the graph health script if available:
  • plugins/kb/skills/kb/SKILL.md: python plugins/kb/scripts/graph_health.py "<vault_path>" --json --fix-suggestions
  • plugins/kb/skills/kb/SKILL.md: Run the evaluation script if available:
  • plugins/kb/skills/kb/SKILL.md: python experiments/evaluate.py "<vault_path>" --json
  • plugins/kb/skills/kb/SKILL.md: If user approves, execute the improvements
  • plugins/kb/skills/kb/SKILL.md: Re-run evaluation to measure improvement
  • plugins/kb/skills/kb/SKILL.md: Run: `npx smaug fetch <tweet_id>` then `npx smaug process`
  • plugins/kb/skills/kb/SKILL.md: **Install Smaug** (recommended) — `git clone https://github.com/alexknowshtml/smaug && cd smaug && npm install`, then `npx smaug setup` to configure X session cookies (`auth_token` + `ct0` from browser DevTools → Cookies → x.com). · uses the network
  • plugins/kb/skills/kb/SKILL.md: After setup, **save the install path to `kb.yaml`** under `integrations.smaug.path`.
  • plugins/kb/skills/kb/SKILL.md: Try to fetch transcript: `yt-dlp --write-auto-sub --sub-lang en --skip-download --output "/tmp/%(id)s" "<url>" 2>&1` · uses the network
  • plugins/kb/skills/kb/SKILL.md: If yt-dlp not installed: try `pip install yt-dlp` then retry
  • plugins/kb/skills/kb/SKILL.md: Extract video metadata: `yt-dlp --dump-json "<url>" 2>/dev/null | python -c "import sys,json; · uses the network
  • plugins/kb/skills/kb/SKILL.md: On the next `/kb compile` run (for compile-time context searches)
  • plugins/kb/skills/kb/SKILL.md: Forgetting to dispatch the auto-evolve subagent after a query -- it must always run, even if it usually exits silently
  • plugins/kb/skills/research-add-items/SKILL.md: Execute simultaneously:
  • plugins/kb/skills/research-deep/SKILL.md: Execute in batches by batch_size (need user approval to proceed to next batch after completing one)
  • plugins/kb/skills/research-deep/SKILL.md: After completing JSON output, run validation script to ensure complete field coverage:
  • plugins/kb/skills/research-deep/SKILL.md: python ~/.claude/skills/research/validate_json.py -f {fields_path} -j {output_path}
  • plugins/kb/skills/research-deep/SKILL.md: After completing JSON output, run validation script to ensure complete field coverage:
  • plugins/kb/skills/research-deep/SKILL.md: python ~/.claude/skills/research/validate_json.py -f /home/weizhena/AIcoding/aicoding-history/fields.yaml -j /home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json
  • plugins/kb/skills/research-report/SKILL.md: **If `kb.yaml` itself is missing:** tell the user to run `kb-init` and stop.
  • plugins/kb/skills/research-report/SKILL.md: Run `python {topic}/generate_report.py`
  • Medium risk: runs-install-script-network-variable-host (plugins/kb/setup.sh)
  • Medium risk: network-variable-host (plugins/kb/setup.sh:81)
  • Medium risk: runs-instruction-instruction-network (plugins/kb/skills/kb/SKILL.md:206)
  • Medium risk: runs-instruction-instruction-network (plugins/kb/skills/kb/SKILL.md:571)
  • Medium risk: runs-instruction-instruction-network (plugins/kb/skills/kb/SKILL.md:596)
  • Medium risk: runs-instruction-instruction-network (plugins/kb/skills/kb/SKILL.md:600)
  • Low risk: network-instructions (.claude/settings.local.json:14)
  • Low risk: scripts-present (plugins/kb/setup.sh)
  • Low risk: network-instructions (plugins/kb/skills/kb/SKILL.md:202)
  • Low risk: autonomy-phrase (plugins/kb/skills/kb/SKILL.md:402)

Attribution and licence

First committed by
GitHub avatar of rvk7895rvk7895
Publisher
Ritvik Aryan Kalra (Community)
Source
rvk7895/llm-knowledge-bases · commit 342686f
Licence
MIT License (files served here)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
ritvik-sarvam, rvk7895

AutoGineer is not affiliated with or endorsed by rvk7895.

Download

The ZIP holds the files AutoGineer serves, with ATTRIBUTION.md and the licence. Download llm-knowledge-bases.zip