ThinkWiki
LLM wiki kit
First committed by wzdavid
Published in wzdavid/ThinkWiki. Publisher page
38 GitHub stars
What it is
Install once, talk to your agent. ThinkWiki turns documents and notes into a local Markdown wiki with inbox review, content knowledge graph, and browsable HTML workspace.
AutoGineer overview, from the brain’s own files
ThinkWiki is an agent-native skill designed to turn scattered information into a structured, durable local workspace. By integrating directly with skills-compatible AI agents, it allows users to manage their knowledge base through natural conversation rather than manual CLI commands. It supports a wide range of input formats, including PDFs, web pages, and various document types, organizing them into a system that features an inbox for review, a local viewer, and an interactive content knowledge graph. The system is built to be local-first, ensuring that Markdown files remain the source of truth, while providing HTML-based interfaces for visualizing relationships between topics, entities, and claims.
How it works
Memory model: LLM wiki
AutoGineer overview, from the brain’s own files
ThinkWiki functions as a skill installed into an AI agent's environment. It uses a set of Python scripts to manage a local directory of Markdown files. The system processes inputs through an inbox workflow, where content is reviewed before being ingested into the wiki. It generates a series of HTML outputs, including a workspace home, an inbox console, a page viewer, and a graph explorer. Users interact with these features by asking their agent to perform tasks like clipping content, building the knowledge graph, or serving the workspace via a local loopback server.
| Folder | Role | Files |
|---|---|---|
| .github | agent-config | 1 |
| docs | docs | 41 |
| scripts | scripts | 34 |
| templates | templates | 13 |
| tests | tests | 1 |
The agent reads
- SKILL.md evidence: README.md
Requirements
Runtimes: Python
API keys (by name): THINKWIKI_EMBED_API_KEY, THINKWIKI_LLM_API_KEY
Set it up
Clone the repository at the commit AutoGineer scanned:
git clone https://github.com/wzdavid/ThinkWiki.git && cd ThinkWiki && git checkout 89f477d7a4095ba2ecf49ef9332e240c6f1208a1cd ThinkWiki && claude
AutoGineer scanned these files at commit 89f477d.
- Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/wzdavid/ThinkWiki/blob/89f477d7a4095ba2ecf49ef9332e240c6f1208a1/README.md#manual-install-optional
First committed by wzdavid. Published in wzdavid/ThinkWiki. Licence: MIT License.
Usage
AutoGineer overview, from the brain’s own files
- Initialize a new local wiki workspace
- Clip web pages, files, or text into an inbox for later processing
- Ingest reviewed items into the permanent wiki structure
- Query the wiki for information with evidence-based responses
- Generate and visualize a content knowledge graph
- Serve the wiki workspace as a local HTML site for browser-based navigation
- “Create a local knowledge base named Research Notes in my workspace.”
- The agent initializes a new ThinkWiki workspace structure in the specified directory.
- “Clip this article into the inbox: https://example.com/article”
- The agent saves the content of the URL into the ThinkWiki inbox for later review.
- “What did we already decide about context budgets?”
- The agent searches the existing wiki pages and provides an answer supported by evidence from the knowledge base.
- “Build the knowledge graph and open the workspace home in my browser.”
- The agent generates the graph visualization and starts the local server to display the workspace in the browser.
Documentation by wzdavid
ThinkWiki
Image: Release Image: License Image: Python
ThinkWiki is an agent-native local knowledge base skill. Talk to your agent to turn scattered documents, web pages, notes, and conversations into a durable Markdown workspace — with inbox review, local browsing, and an interactive content knowledge graph you can open in a browser.
Chinese version: README.zh.md
Repository: https://github.com/wzdavid/ThinkWiki
Works With Skills-Compatible Agents
ThinkWiki follows the open Agent Skills format (SKILL.md + bundled scripts). If your agent can install skills and run local commands, it can run ThinkWiki.
You do not need to memorize CLI commands. Install the skill once, then create and manage your wiki entirely through conversation.
| Agent / Host | Typical use |
|---|---|
| Claude Code | Install the repo as a skill, then ask it to build and query your wiki |
| OpenClaw | Full skill workflow; use serve + the built-in browser to open HTML outputs |
| Trae | Install under .trae/skills, then manage the wiki in chat |
| Hermes Agent | Local skill directory + shell execution |
| OpenAI Codex / Codex CLI | Skills-compatible coding agents with local file access |
| Cursor | Agent mode with project skills |
| Gemini CLI | Skills-compatible CLI agent |
| GitHub Copilot | Agent / coding workflows in VS Code that support skills |
| Other skills hosts | Any product that loads SKILL.md and can run python3 scripts/thinkwiki ... locally |
Requirement: a local environment with Python 3. ThinkWiki bootstraps its own runtime on first use. Pure cloud chat without local file or shell access cannot run a local wiki.
Get Started: Install Through Conversation
Point your agent at this repository and ask it to install the skill for you.
Example prompt:
Please install the ThinkWiki skill from https://github.com/wzdavid/ThinkWiki , bootstrap the runtime, and run a health check to confirm everything works on my machine.
The agent will typically clone or copy the skill into your host's skills directory, run bootstrap, and verify capabilities with doctor. You should not need to run those steps yourself unless you prefer manual setup.
After installation, start using it immediately:
Create a local knowledge base named
My Wikiin my workspace.
Clip this article into the inbox first:
https://example.com/article
Import this PDF into the wiki:
/path/to/file.pdf
Answer from my wiki: what did we already decide about context budgets?
Build the knowledge graph and open the workspace home in my browser.
The agent reads SKILL.md, maps your intent to stable ThinkWiki actions, and reports what changed — including paths to Markdown pages and HTML outputs.
Why ThinkWiki
Agent-first: users talk to an agent, not to a long list of commands.
Local-first: Markdown files remain the source of truth on your machine.
HTML-first: inbox, viewer, graph, and governance pages are real browsable workspaces.
Knowledge-first: the graph is a content knowledge graph — topics, concepts, entities, claims, and governance — not just a file-link map.
What It Produces
ThinkWiki generates a workspace that includes:
output/index.html: unified workspace homeoutput/inbox/index.html: inbox review consoleoutput/viewer/index.html: local page browseroutput/graph/index.html: interactive graph exploreroutput/graph/report.html: graph governance reportoutput/graph/entity-merge-review.html: entity merge review pageoutput/graph/entity-merge-plan.html: entity merge dry-run preview when requested
Preview
Workspace Home (output/index.html)
The unified workspace home summarizes recent changes, recommended next actions, inbox backlog, and a graph snapshot so you can decide whether to read, ingest, or explore.
Image: ThinkWiki workspace home preview
Inbox Review (output/inbox/index.html)
Review clipped web pages, files, and notes before formal ingest. Items are grouped into ready, review, and weak so you can process the highest-value captures first.
Image: ThinkWiki inbox review preview
Local Viewer (output/viewer/index.html)
Browse the entire wiki by page type, confidence, and status. Open any page directly in the viewer without leaving the local HTML workspace.
Image: ThinkWiki local viewer preview
Knowledge Graph (output/graph/index.html)
Explore the content knowledge graph interactively. Switch between knowledge, document, and suggested views to inspect semantic relations and candidate edges.
Image: ThinkWiki knowledge graph preview
Graph Governance Report (output/graph/report.html)
Inspect graph health signals such as isolated pages, hub stubs, fragile bridges, suggested links, and entity merge candidates before making structural changes.
Image: ThinkWiki graph governance report preview
Everyday Commands (In Plain Language)
Once ThinkWiki is installed, these are the kinds of requests users make most often:
| You say | ThinkWiki does |
|---|---|
Create a wiki called Research Notes | Initializes a local workspace |
| Clip / save this URL or file to inbox | Captures content for later review |
| Ingest the ready inbox items | Promotes reviewed captures into wiki pages |
| What does my wiki say about X? | Evidence-first Q&A from existing pages |
| Save this answer as a query page | Persists high-value outputs |
| Show me the knowledge graph | Builds/refreshes graph HTML |
| Review entity merge candidates | Surfaces alias collisions for manual confirmation |
| Open the wiki workspace in my browser | Runs serve and returns http://127.0.0.1:8765/index.html |
Core Capabilities
Initialize a local wiki workspace
Clip web pages, files, or pasted text into an inbox for later review
Import Markdown, PDF, DOCX, XLSX, XLS, PPTX, web pages, and plain text
Answer questions from existing wiki pages with evidence-first behavior
Save high-value outputs as
query,synthesis,decision, orconceptpagesBuild a content knowledge graph with
knowledge,document, andsuggestedviewsGenerate entity governance outputs, including ambiguous alias review and deterministic merge flows
Check runtime health, workspace status, and graph governance state
Content Knowledge Graph
ThinkWiki v1.6.0 builds a schema v2 graph with default_view = knowledge.
The knowledge view can contain:
page-backed nodes such as
source,topic,concept,decision,synthesis,query, andentityclaim nodes extracted from structured wiki content
semantic relations such as
about,belongs_to,depends_on,asserts,supports,contradicts, andsuggests_related_to
This means the graph is not limited to raw file references. It can represent knowledge structure, evidence structure, and entity governance directly.
Browse HTML Outputs
Agent chat UIs usually cannot render ThinkWiki HTML inline. Ask your agent to start the built-in loopback server when you want to inspect inbox, viewer, graph, or governance pages:
Start the ThinkWiki output server and give me the workspace URL.
Or run it yourself:
python3 scripts/thinkwiki serve --root /path/to/my-wikiBy default this serves <wiki-root>/output/ at http://127.0.0.1:8765/. Start from http://127.0.0.1:8765/index.html.
On OpenClaw, after serve is running, the agent can also open the workspace home with:
openclaw browser --browser-profile openclaw open http://127.0.0.1:8765/index.htmlOn other agents, open the same URL in your system browser, or use whatever browser tool the host provides.
Use serve --print-urls when you only need the URL list without starting the server.
Environment Variables
Remote AI features are disabled by default. Configure only what you need:
Content generation (crystallize, digest)
Requires all three variables. Any OpenAI-compatible chat completion endpoint works (MiniMax, OpenAI, DeepSeek, local Ollama, etc.).
| Variable | Required | Purpose |
|---|---|---|
THINKWIKI_LLM_API_KEY | Yes, to enable | API key for the configured LLM provider |
THINKWIKI_LLM_BASE_URL | Yes, to enable | Chat completions URL, e.g. https://api.openai.com/v1/chat/completions |
THINKWIKI_LLM_MODEL | Yes, to enable | Model name, e.g. gpt-4o-mini or MiniMax-M3 |
THINKWIKI_LLM_TEMPERATURE | No | Optional temperature override |
Without a complete LLM configuration, crystallize and digest use local heuristics only.
Entity embedding (entity-merge-review, graph-report, health)
Enabled when THINKWIKI_EMBED_API_KEY is set. Defaults to SiliconFlow BGE-M3; override the endpoint or model if needed.
| Variable | Required | Purpose |
|---|---|---|
THINKWIKI_EMBED_API_KEY | Yes, to enable | API key for the embedding provider |
THINKWIKI_EMBED_BASE_URL | No | Embedding URL(s), comma-separated; default https://api.siliconflow.cn/v1/embeddings |
THINKWIKI_EMBED_MODEL | No | Embedding model; default BAAI/bge-m3 |
Register a free SiliconFlow key at https://siliconflow.cn if you want the default embedding setup.
Manual Install (Optional)
If you prefer to set up the skill yourself instead of asking an agent:
git clone https://github.com/wzdavid/ThinkWiki ThinkWiki
cd ThinkWiki
python3 scripts/thinkwiki bootstrap
python3 scripts/thinkwiki doctor --repo-root .Then install the folder into your agent's skills directory according to that host's documentation.
CLI Reference (For Agents And Power Users)
ThinkWiki keeps one stable entry point that agents call under the hood:
python3 scripts/thinkwiki <command> [args]Common commands: init, clip, ingest, inbox, viewer, graph, graph-report, entity-merge-review, entity-merge-apply, serve, health, status, doctor.
See SKILL.md for the agent behavior contract and intent mapping.
Repository Docs
README.md: concise English project overviewREADME.zh.md: Chinese overviewSKILL.md: skill behavior contract for agent hostsCHANGELOG.md: release history
License
MIT
scripts/ask.py
Shown as plain text.
#!/usr/bin/env python3
from __future__ import annotations
"""
ThinkWiki Script: ask
Purpose:
- Answer a user question from the current wiki using evidence-first retrieval and page ranking.
Usage:
- Prefer `python scripts/thinkwiki ask ...`.
- Run `python scripts/<script> --help` for direct CLI details when the file exposes its own arguments.
"""
import argparse
from datetime import date
from pathlib import Path
import re
from crystallize import first_meaningful_line, text_tokens, write_page
from utils import (
collect_wiki_pages,
extract_summary,
file_uri,
find_repo_root,
normalize_repo_path,
parse_frontmatter,
read_text,
refresh_output_home_if_present,
today_str,
write_text,
)
CONFIDENCE_BONUS = {
"verified": 8,
"extracted": 5,
"mixed": 2,
"inferred": 0,
}
STATUS_BONUS = {
"active": 3,
"stale": -4,
"archived": -7,
"superseded": -9,
}
TYPE_BONUS = {
"source": 5,
"normalized": 4,
"topic": 4,
"concept": 3,
"decision": 2,
"synthesis": 1,
"query": -4,
}
SECTION_PREFERENCE = {
"Summary": 5,
"Decision": 5,
"Answer": 5,
"Key Points": 4,
"Findings": 4,
"Reasoning": 4,
"Details": 3,
"Evidence": 1,
"Consulted Pages": 0,
"Follow-ups": 0,
}
BLOCKED_SECTIONS = {
"Connections",
"Open Questions",
"Consulted Pages",
"Raw Source",
"Extracted Markdown",
"Extracted Excerpt",
}
PLACEHOLDER_PATTERNS = [
re.compile(r"\bnone yet\b", re.IGNORECASE),
re.compile(r"\btodo\b", re.IGNORECASE),
re.compile(r"\bwhat should this source update\b", re.IGNORECASE),
re.compile(r"\bno relevant pages found\b", re.IGNORECASE),
re.compile(r"\bno matching evidence snippets found\b", re.IGNORECASE),
]
META_LINE_PREFIXES = ("来源:", "作者:", "发布日期:", "原文链接:")
NOISE_MARKERS = (
"<ama-doc>",
"文件编号",
"文档版本",
"最后修改日期",
"修订页",
"编 写 人",
"编写时间",
"目录",
)
DEFINITION_HINTS = ("是", "定义", "本质", "指的是", "意味着", "可概括为")
DECISION_HINTS = ("建议", "应", "因此", "因为", "关键", "核心", "推荐")
ANSWER_HINTS = ("定义为", "本质上是", "指的是", "意味着", "换句话说", "可归纳为")
QUESTION_SEGMENT_HINTS = ("研究问题", "问题一", "问题二", "问题三", "问题四")
def parse_iso_date(raw: str) -> date | None:
value = raw.strip()
if not value:
return None
try:
return date.fromisoformat(value[:10])
except ValueError:
return None
def freshness_bonus(updated: str) -> int:
parsed = parse_iso_date(updated)
if parsed is None:
return 0
age_days = max((date.today() - parsed).days, 0)
if age_days <= 30:
return 3
if age_days <= 90:
return 1
if age_days > 365:
return -2
return 0
def candidate_record(root: Path, page: Path) -> dict[str, object]:
meta, body = parse_frontmatter(read_text(page))
title = str(meta.get("title") or page.stem)
summary = extract_summary(meta, body)
confidence = str(meta.get("confidence") or "").strip().lower()
status = str(meta.get("status") or "active").strip().lower()
updated = str(meta.get("updated") or meta.get("created") or "")
page_type = str(meta.get("type") or page.parent.name[:-1])
source_paths = [normalize_repo_path(root, str(item)) for item in meta.get("sources", [])]
return {
"path": page,
"title": title,
"summary": summary,
"confidence": confidence,
"status": status,
"updated": updated,
"type": page_type,
"sources": source_paths,
"body": body,
}
def extract_title_from_body(body: str, fallback: str) -> str:
for raw_line in body.splitlines():
line = raw_line.strip()
if line.startswith("# "):
title = line[2:].strip()
if title:
return title
return fallback
def meaningful_body_summary(body: str) -> str:
for raw_line in body.splitlines():
line = raw_line.strip()
if not line or line.startswith("#"):
continue
if is_noise_line(line):
continue
if line.startswith(("- 来源:", "- 作者:", "- 发布日期:", "- 原文链接:")):
continue
if line.startswith(("更新时间", "更新于", "Published:", "Updated:")):
continue
if line.startswith(("http://", "https://")):
continue
return plain_text(line)[:120]
return "(no summary)"
def normalized_record(root: Path, page: Path) -> dict[str, object]:
body = read_text(page)
title = extract_title_from_body(body, page.stem)
raw_candidates = list((root / "raw").glob(f"**/{page.stem}.*"))
sources = [candidate.relative_to(root).as_posix() for candidate in raw_candidates[:1]]
return {
"path": page,
"title": title,
"summary": meaningful_body_summary(body),
"confidence": "extracted",
"status": "active",
"updated": "",
"type": "normalized",
"sources": sources,
"body": body,
}
def slugify_section_name(value: str) -> str:
slug = re.sub(r"[^a-z0-9\u4e00-\u9fff]+", "-", value.strip().lower())
slug = re.sub(r"-+", "-", slug).strip("-")
return slug or "section"
def is_placeholder_text(text: str) -> bool:
compact = " ".join(text.split()).strip()
if not compact:
return True
if len(compact) <= 4:
return True
for pattern in PLACEHOLDER_PATTERNS:
if pattern.search(compact):
return True
return False
def is_metadata_only_text(text: str) -> bool:
compact = " ".join(text.split()).strip(" -")
if not compact:
return True
parts = [part.strip() for part in compact.split(" - ") if part.strip()]
if not parts:
return True
return all(part.startswith(META_LINE_PREFIXES) for part in parts)
def plain_text(text: str) -> str:
text = re.sub(r"!\[[^\]]*\]\([^)]+\)", "", text)
text = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", text)
text = re.sub(r"https?://\S+", "", text)
return " ".join(text.split()).strip(" -|,;")
def is_table_like_text(text: str) -> bool:
compact = text.strip()
if not compact:
return False
if compact.count("|") >= 4:
return True
lines = [line.strip() for line in compact.splitlines() if line.strip()]
if lines and sum(1 for line in lines if "|" in line) >= max(2, len(lines) // 2 + 1):
return True
return False
def is_toc_like_text(text: str) -> bool:
compact = plain_text(text)
if not compact:
return False
if compact.startswith(("1.", "1.1", "2.", "2.1")) and len(compact) <= 40:
return True
if re.match(r"^\d+(?:\.\d+){0,3}\s*[\u4e00-\u9fffA-Za-z].*\d+$", compact):
return True
return False
def is_noise_line(line: str) -> bool:
compact = plain_text(line)
if not compact:
return True
if compact in {"---", "***"}:
return True
if compact.startswith(NOISE_MARKERS):
return True
if is_table_like_text(compact):
return True
if is_toc_like_text(compact):
return True
if compact.startswith(("**")) and any(marker in compact for marker in NOISE_MARKERS):
return True
return False
def cjk_ngrams(text: str, size: int = 2) -> set[str]:
chars = [char for char in text if "\u4e00" <= char <= "\u9fff"]
if len(chars) < size:
return set(chars)
return {"".join(chars[index : index + size]) for index in range(len(chars) - size + 1)}
def text_match_score(query: str, query_tokens: set[str], text: str) -> int:
normalized = plain_text(text)
if not normalized:
return 0
normalized_tokens = text_tokens(normalized)
overlap = query_tokens.intersection(normalized_tokens)
score = len(overlap) * 4
query_lower = query.strip().lower()
lowered = normalized.lower()
if query_lower and query_lower in lowered:
score += 8
query_ngrams = cjk_ngrams(query)
if query_ngrams:
text_ngrams = cjk_ngrams(normalized)
score += len(query_ngrams.intersection(text_ngrams))
return score
def segment_quality_bonus(section: str, text: str, question: str) -> int:
normalized = plain_text(text)
if not normalized:
return -20
score = SECTION_PREFERENCE.get(section, 0)
if is_table_like_text(text):
score -= 10
if is_toc_like_text(text):
score -= 8
if len(normalized) < 20:
score -= 3
if normalized.endswith(("?", "?")):
score -= 4
if any(hint in normalized for hint in QUESTION_SEGMENT_HINTS):
score -= 6
if any(hint in normalized for hint in ANSWER_HINTS):
score += 6
if any(hint in normalized for hint in DEFINITION_HINTS) and any(token in question for token in ("什么", "定义", "本质")):
score += 8
if any(hint in normalized for hint in DECISION_HINTS) and any(token in question for token in ("为什么", "如何", "建议", "应")):
score += 6
return score
def informative_segments(body: str, limit: int = 6) -> list[dict[str, str]]:
candidates: list[tuple[int, dict[str, str]]] = []
for item in body_segments(body):
snippet = item["snippet"]
section = item["section"]
if is_noise_line(snippet):
continue
quality = segment_quality_bonus(section, snippet, "")
if quality <= -5:
continue
candidates.append((quality, item))
candidates.sort(key=lambda item: (-item[0], item[1]["section"], item[1]["snippet"]))
return [item for _score, item in candidates[:limit]]
def body_segments(body: str) -> list[dict[str, str]]:
segments: list[dict[str, str]] = []
current_section = "Overview"
block_lines: list[str] = []
def flush_block() -> None:
nonlocal block_lines
lines = [line.strip() for line in block_lines if line.strip()]
block_lines = []
if not lines:
return
if all(line.startswith("- [") and "](" in line for line in lines):
return
text = " ".join(line for line in lines if not line.startswith("```")).strip()
text = re.sub(r"^(?:-\s+)+", "", text).strip()
if not text:
return
text = plain_text(text)
if not text:
return
text_without_links = re.sub(r"\[[^\]]+\]\([^)]+\)", "", text).strip(" -|")
if not text_without_links:
return
if current_section in BLOCKED_SECTIONS:
return
if is_placeholder_text(text_without_links):
return
if is_metadata_only_text(text_without_links):
return
if is_noise_line(text_without_links):
return
segments.append({
"section": current_section,
"snippet": text[:400],
})
for raw_line in body.replace("\r\n", "\n").splitlines():
line = raw_line.rstrip()
stripped = line.strip()
if stripped in {"---", "***"}:
continue
if stripped.startswith(":
continue
if stripped.startswith("<") and stripped.endswith(">"):
continue
if stripped.startswith("## "):
flush_block()
current_section = stripped[3:].strip() or "Overview"
continue
if stripped.startswith("### "):
flush_block()
current_section = stripped[4:].strip() or current_section
continue
if not stripped:
flush_block()
continue
if stripped.startswith("#"):
continue
block_lines.append(stripped)
flush_block()
return segments
def candidate_score(question: str, question_tokens: set[str], record: dict[str, object]) -> int:
title = str(record["title"])
summary = str(record["summary"])
body = str(record["body"])
confidence = str(record["confidence"])
status = str(record["status"])
page_type = str(record["type"])
score = text_match_score(question, question_tokens, title) * 2
score += text_match_score(question, question_tokens, summary)
best_segment_score = 0
for item in informative_segments(body, limit=6):
segment_score = text_match_score(question, question_tokens, item["snippet"])
segment_score += segment_quality_bonus(item["section"], item["snippet"], question)
if segment_score > best_segment_score:
best_segment_score = segment_score
score += best_segment_score
score += CONFIDENCE_BONUS.get(confidence, 0)
score += STATUS_BONUS.get(status, 0)
score += TYPE_BONUS.get(page_type, 0)
score += freshness_bonus(str(record["updated"]))
question_lower = question.strip().lower()
joined = " ".join([title.lower(), summary.lower(), plain_text(body).lower()])
if question_lower and question_lower in joined:
score += 6
if question_lower and plain_text(title).lower() in question_lower:
score += 4
if is_table_like_text(summary):
score -= 10
if any(marker in plain_text(summary) for marker in NOISE_MARKERS):
score -= 8
return score
def evidence_score(question: str, question_tokens: set[str], record: dict[str, object], section: str, segment: str, page_score: int) -> int:
if section in BLOCKED_SECTIONS or is_placeholder_text(segment) or is_metadata_only_text(segment):
return 0
score = text_match_score(question, question_tokens, " ".join([section, segment]))
if score <= 0:
return 0
score += segment_quality_bonus(section, segment, question)
score += min(page_score, 18)
score += CONFIDENCE_BONUS.get(str(record["confidence"]), 0)
score += STATUS_BONUS.get(str(record["status"]), 0)
return score
def collect_candidate_records(root: Path) -> list[dict[str, object]]:
records = [candidate_record(root, page) for page in collect_wiki_pages(root)]
normalized_root = root / "normalized"
if normalized_root.exists():
for page in sorted(normalized_root.glob("**/*.md")):
records.append(normalized_record(root, page))
return records
def evidence_items(question: str, question_tokens: set[str], ranked: list[tuple[int, dict[str, object]]], per_page_limit: int = 2, total_limit: int = 5) -> list[dict[str, object]]:
items_by_key: dict[tuple[str, str], dict[str, object]] = {}
for page_score, record in ranked:
segments = body_segments(str(record["body"]))
local_items: list[tuple[int, str, str]] = []
for item in segments:
section = item["section"]
segment = item["snippet"]
score = evidence_score(question, question_tokens, record, section, segment, page_score)
if score <= 0:
continue
local_items.append((score, section, segment))
local_items.sort(key=lambda item: (-item[0], item[2].lower()))
for score, section, segment in local_items[:per_page_limit]:
dedupe_key = (str(record["title"]), segment.casefold())
candidate = {
"page_title": str(record["title"]),
"page_path": Path(str(record["path"])),
"page_type": str(record["type"]),
"section": section,
"section_anchor": slugify_section_name(section),
"confidence": str(record["confidence"] or "n/a"),
"status": str(record["status"] or "n/a"),
"score": score,
"snippet": segment,
}
existing = items_by_key.get(dedupe_key)
if existing is None:
items_by_key[dedupe_key] = candidate
continue
existing_pref = SECTION_PREFERENCE.get(str(existing["section"]), 0)
candidate_pref = SECTION_PREFERENCE.get(section, 0)
if candidate_pref > existing_pref:
items_by_key[dedupe_key] = candidate
continue
if candidate_pref == existing_pref and int(candidate["score"]) > int(existing["score"]):
items_by_key[dedupe_key] = candidate
items = list(items_by_key.values())
items.sort(key=lambda item: (-int(item["score"]), str(item["page_title"]).lower(), str(item["section"])))
return items[:total_limit]
def prioritize_evidence_for_best_record(evidence: list[dict[str, object]], best_record: dict[str, object]) -> list[dict[str, object]]:
best_path = str(best_record["path"])
best_title = str(best_record["title"]).casefold()
prioritized = sorted(
evidence,
key=lambda item: (
0 if str(item["page_path"]) == best_path else 1 if str(item["page_title"]).casefold() == best_title else 2,
-int(item["score"]),
str(item["page_title"]).lower(),
str(item["section"]),
),
)
return prioritized
def expanded_evidence_pool(
ranked: list[tuple[int, dict[str, object]]],
limited: list[tuple[int, dict[str, object]]],
extra_limit: int = 8,
) -> list[tuple[int, dict[str, object]]]:
if not limited:
return limited
best_title = str(limited[0][1]["title"]).casefold()
pool = list(limited)
seen_paths = {str(item[1]["path"]) for item in pool}
for score, record in ranked:
if str(record["path"]) in seen_paths:
continue
if str(record["title"]).casefold() != best_title:
continue
pool.append((score, record))
seen_paths.add(str(record["path"]))
if len(pool) >= len(limited) + extra_limit:
break
return pool
def fallback_evidence_items(ranked: list[tuple[int, dict[str, object]]], total_limit: int = 3) -> list[dict[str, object]]:
items: list[dict[str, object]] = []
seen_titles: set[tuple[str, str]] = set()
type_priority = {"normalized": 0, "source": 1, "topic": 2, "concept": 3, "decision": 4, "synthesis": 5, "query": 6}
best_title = str(ranked[0][1]["title"]).casefold() if ranked else ""
def collect_from_pool(pool: list[tuple[int, dict[str, object]]]) -> None:
for page_score, record in pool:
title_key = (str(record["title"]).casefold(), str(record["type"]))
if title_key in seen_titles:
continue
for item in body_segments(str(record["body"])):
snippet = item["snippet"]
if is_placeholder_text(snippet):
continue
items.append({
"page_title": str(record["title"]),
"page_path": Path(str(record["path"])),
"page_type": str(record["type"]),
"section": item["section"],
"section_anchor": slugify_section_name(item["section"]),
"confidence": str(record["confidence"] or "n/a"),
"status": str(record["status"] or "n/a"),
"score": page_score,
"snippet": snippet,
})
seen_titles.add(title_key)
break
if len(items) >= total_limit:
return
prioritized = sorted(
ranked,
key=lambda item: (
type_priority.get(str(item[1]["type"]), 99),
-item[0],
str(item[1]["title"]).lower(),
),
)
same_title_pool = [item for item in prioritized if str(item[1]["title"]).casefold() == best_title]
collect_from_pool(same_title_pool)
if not items:
collect_from_pool(prioritized)
return items
def bullet_label(record: dict[str, object], score: int, root: Path) -> str:
path = Path(str(record["path"])).relative_to(root).as_posix()
confidence = str(record["confidence"] or "n/a")
status = str(record["status"] or "n/a")
updated = str(record["updated"] or "n/a")
return (
f"- {record['title']} | path: {path} | type: {record['type']} | "
f"confidence: {confidence} | status: {status} | updated: {updated} | score: {score}"
)
def build_answer(question: str, ranked: list[tuple[int, dict[str, object]]], evidence: list[dict[str, object]]) -> str:
if not ranked:
return (
f"- 当前知识库里没有找到足够相关的页面来回答“{question}”。\n"
"- 建议先导入相关资料,或补充一个 topic/source 页面后再查询。"
)
answer_lines = []
best_score, best = ranked[0]
answer_lines.append(
f"- 当前最相关的条目是《{best['title']}》,类型为 {best['type']},"
f"置信度为 {best['confidence'] or 'n/a'},状态为 {best['status'] or 'n/a'}。"
)
answer_lines.append(f"- 该页摘要:{best['summary']}")
if evidence:
top = evidence[0]
answer_lines.append(
f"- 最直接的证据摘录来自《{top['page_title']}》的“{top['section']}”部分:{top['snippet']}"
)
if len(ranked) > 1:
other_titles = ";".join(f"《{record['title']}》" for _score, record in ranked[1:4])
answer_lines.append(f"- 其他可交叉参考的页面有:{other_titles}。")
low_confidence = [
record["title"]
for _score, record in ranked
if str(record["confidence"]) in {"mixed", "inferred", ""}
]
if low_confidence:
answer_lines.append(f"- 需要额外验证的页面包括:{';'.join(f'《{title}》' for title in low_confidence[:3])}。")
stale_pages = [
record["title"]
for _score, record in ranked
if str(record["status"]) in {"stale", "archived", "superseded"}
]
if stale_pages:
answer_lines.append(f"- 其中有较旧或已降权的内容:{';'.join(f'《{title}》' for title in stale_pages[:3])}。")
answer_lines.append(f"- 检索排序优先考虑标题命中、摘要命中、置信度、状态和更新时间;最佳匹配得分为 {best_score}。")
return "\n".join(answer_lines)
def evidence_label(item: dict[str, object], root: Path) -> str:
path = Path(str(item["page_path"])).relative_to(root).as_posix()
ref = f"{path}#{item['section_anchor']}"
return (
f"- 《{item['page_title']}》 | path: {path} | type: {item['page_type']} | "
f"section: {item['section']} | ref: {ref} | confidence: {item['confidence']} | status: {item['status']} | score: {item['score']} | "
f"snippet: {item['snippet']}"
)
def answer_with_evidence(answer: str, evidence: list[dict[str, object]], root: Path) -> str:
if not evidence:
return answer
lines = [answer, "", "### Evidence", ""]
lines.extend(evidence_label(item, root) for item in evidence)
return "\n".join(lines)
def sanitize_saved_query_page(page_path: Path, root: Path) -> None:
page_text = read_text(page_path)
if not page_text:
return
self_repo_path = page_path.relative_to(root).as_posix()
page_name = page_path.name
lines = page_text.splitlines()
cleaned: list[str] = []
in_sources = False
in_consulted = False
for line in lines:
stripped = line.strip()
if stripped.startswith("sources:"):
in_sources = True
in_consulted = False
cleaned.append(line)
continue
if stripped.startswith("## "):
in_sources = False
in_consulted = stripped == "## Consulted Pages"
cleaned.append(line)
continue
if in_sources and stripped.startswith("-"):
if self_repo_path in stripped:
continue
cleaned.append(line)
continue
if in_sources and stripped.startswith("-") is False and stripped.startswith("tags:"):
in_sources = False
cleaned.append(line)
continue
if in_sources and stripped.startswith("-") is False and stripped:
in_sources = False
if in_consulted and f"]({page_name})" in stripped:
continue
cleaned.append(line)
write_text(page_path, "\n".join(cleaned))
def main() -> int:
parser = argparse.ArgumentParser(description="Ask the wiki a question using lightweight page ranking.")
parser.add_argument("--root", default=".", help="Wiki root path")
parser.add_argument("--question", required=True, help="Question to ask the wiki")
parser.add_argument("--limit", type=int, default=5, help="Maximum number of pages to consult")
parser.add_argument("--save", action="store_true", help="Persist the ask result into wiki/queries")
parser.add_argument("--title", default="", help="Optional query page title when --save is used")
parser.add_argument("--follow-up", action="append", default=[], help="Optional follow-up item when --save is used")
parser.add_argument("--slug", default="", help="Explicit query page slug when --save is used")
parser.add_argument("--update", action="store_true", help="Update an existing query page when --save is used")
parser.add_argument("--merge-mode", choices=["append", "replace", "dedupe"], default="replace", help="How to merge fields when --save is used")
args = parser.parse_args()
root = find_repo_root(Path(args.root))
question_tokens = text_tokens(args.question)
ranked: list[tuple[int, dict[str, object]]] = []
for record in collect_candidate_records(root):
score = candidate_score(args.question, question_tokens, record)
if score <= 0:
continue
ranked.append((score, record))
ranked.sort(key=lambda item: (-item[0], str(item[1]["title"]).lower()))
non_query_ranked = [item for item in ranked if str(item[1]["type"]) != "query"]
primary_ranked = non_query_ranked or ranked
limited = primary_ranked[: max(args.limit, 1)]
evidence_pool = expanded_evidence_pool(primary_ranked, limited)
evidence = evidence_items(args.question, question_tokens, evidence_pool)
if not evidence:
evidence = fallback_evidence_items(evidence_pool)
if limited and evidence:
evidence = prioritize_evidence_for_best_record(evidence, limited[0][1])
answer = build_answer(args.question, limited, evidence)
consulted_paths = [Path(str(record["path"])).relative_to(root).as_posix() for _score, record in limited]
lines = [
"# Ask Result",
"",
f"- Date: {today_str()}",
f"- Question: {args.question}",
f"- Consulted: {len(limited)}",
"",
"## Answer",
"",
answer,
"",
"## Evidence",
"",
]
if evidence:
lines.extend(evidence_label(item, root) for item in evidence)
else:
lines.append("- No matching evidence snippets found")
lines.extend([
"",
"## Consulted Pages",
"",
])
if limited:
lines.extend(bullet_label(record, score, root) for score, record in limited)
else:
lines.append("- No relevant pages found")
if args.save:
title = args.title.strip() or args.question.strip()
summary = first_meaningful_line(answer, title)
saved_content = answer_with_evidence(answer, evidence, root)
page_path, action = write_page(
root=root,
kind="query",
title=title,
summary=summary,
content=saved_content,
source_paths=consulted_paths or ["index.md"],
related_paths=[],
follow_ups=args.follow_up,
findings=[],
tensions=[],
key_points=[],
action_label="ask",
slug=args.slug,
update=args.update,
merge_mode=args.merge_mode,
)
sanitize_saved_query_page(page_path, root)
lines.extend([
"",
"## Query Page",
"",
f"- {action}: {page_path.relative_to(root).as_posix()}",
])
output_home = refresh_output_home_if_present(root)
if output_home is not None:
lines.extend([
"",
"## Output Pages",
"",
"- Open output/index.html to quickly access the local viewer and graph pages.",
f"- file URI: {file_uri(output_home)}",
])
print("\n".join(lines))
return 0
if __name__ == "__main__":
raise SystemExit(main())
98 files · 5 not served here
- .github/workflows/cross-platform.yml
- .gitignore
- ATTRIBUTION.md
- CHANGELOG.md
- LICENSE
- README.md
- README.zh.md
- SKILL.md
- docs/assets/graph-preview.png (not served here)
- docs/assets/graph-report-preview.png (not served here)
- docs/assets/inbox-preview.png (not served here)
- docs/assets/output-hub-preview.png (not served here)
- docs/assets/viewer-preview.png (not served here)
- docs/demo-wiki/.wiki-schema.md
- docs/demo-wiki/index.md
- docs/demo-wiki/log.md
- docs/demo-wiki/normalized/articles/eval-loops.md
- docs/demo-wiki/normalized/articles/platform-spec.md
- docs/demo-wiki/normalized/articles/review-checklist.md
- docs/demo-wiki/normalized/assets/inbox/2026-06-20-team-retrospective/cover.txt
- docs/demo-wiki/normalized/inbox/2026-06-20-team-retrospective.json
- docs/demo-wiki/normalized/inbox/2026-06-20-team-retrospective.md
- docs/demo-wiki/output/graph/graph.json
- docs/demo-wiki/output/graph/graph.md
- docs/demo-wiki/output/graph/index.html
- docs/demo-wiki/output/graph/report.html
- docs/demo-wiki/output/graph/report.json
- docs/demo-wiki/output/graph/report.md
- docs/demo-wiki/output/inbox/index.html
- docs/demo-wiki/output/index.html
- docs/demo-wiki/output/viewer/index.html
- docs/demo-wiki/output/viewer/viewer.json
- docs/demo-wiki/overview.md
- docs/demo-wiki/purpose.md
- docs/demo-wiki/raw/articles/eval-loops.md
- docs/demo-wiki/raw/articles/platform-spec.md
- docs/demo-wiki/raw/articles/review-checklist.md
- docs/demo-wiki/raw/inbox/2026-06-20-team-retrospective.md
- docs/demo-wiki/wiki/concepts/ai-native-team.md
- docs/demo-wiki/wiki/concepts/execution-spec.md
- docs/demo-wiki/wiki/decisions/context-budget.md
- docs/demo-wiki/wiki/decisions/execution-policy.md
- docs/demo-wiki/wiki/queries/what-makes-ai-native-team.md
- docs/demo-wiki/wiki/sources/eval-loops.md
- docs/demo-wiki/wiki/sources/platform-spec.md
- docs/demo-wiki/wiki/sources/review-checklist.md
- docs/demo-wiki/wiki/syntheses/delivery-system.md
- docs/demo-wiki/wiki/topics/eval-loops.md
- docs/demo-wiki/wiki/topics/model-platform.md
- requirements.txt
- scripts/ai_config.py
- scripts/ask.py
- scripts/batch_clip.py
- scripts/batch_ingest.py
- scripts/bootstrap_runtime.py
- scripts/build_demo_wiki.py
- scripts/build_graph.py
- scripts/build_inbox.py
- scripts/build_viewer.py
- scripts/capture_readme_screenshots.py
- scripts/clip.py
- scripts/convert_source.py
- scripts/correct.py
- scripts/crystallize.py
- scripts/digest.py
- scripts/doctor.py
- scripts/embed_client.py
- scripts/entity_merge_apply.py
- scripts/entity_merge_review.py
- scripts/graph_report.py
- scripts/health.py
- scripts/ingest.py
- scripts/init_wiki.py
- scripts/lint_wiki.py
- scripts/llm_client.py
- scripts/query_wiki.py
- scripts/rebuild_index.py
- scripts/runtime_capabilities.py
- scripts/serve_outputs.py
- scripts/status.py
- scripts/thinkwiki
- scripts/url_safety.py
- scripts/utils.py
- scripts/workspace_status.py
- templates/pages/concept.md
- templates/pages/decision.md
- templates/pages/entity.md
- templates/pages/query.md
- templates/pages/source.md
- templates/pages/synthesis.md
- templates/pages/topic.md
- templates/root/.wiki-schema.md
- templates/root/AGENTS.md
- templates/root/index.md
- templates/root/log.md
- templates/root/overview.md
- templates/root/purpose.md
- tests/test_thinkwiki.py
Advantages
- Enables natural language management of a knowledge base without requiring the user to memorize complex CLI commands. evidence: README.md
- Maintains Markdown files as the local source of truth, ensuring data portability and ownership. evidence: README.md
- Provides a rich, interactive content knowledge graph that maps semantic relations like claims and entities rather than just file links. evidence: README.md
Limitations
- Requires a local environment with Python 3 and cannot function in pure cloud-based chat environments without local file or shell access. evidence: README.md
- Agent chat UIs typically cannot render the generated HTML interfaces inline, requiring the use of a separate browser or a local loopback server. evidence: README.md
Security
Risk level: No risk found
This brain runs
Agent instructions that run or fetch something (5)
SKILL.md:If the user wants a new workspace, run `init`.SKILL.md:After `viewer`, `graph`, `inbox`, or `graph-report`, run `serve` when the user wants to browse the results.SKILL.md:If a long-running `serve` process is not practical in the current session, run `serve --print-urls` and tell the user to start `serve` locally, then open the printed workspace URL.SKILL.md:After starting `serve`, give the user the workspace home URL first (`http://127.0.0.1:8765/index.html` by default) and, in OpenClaw, offer to open it with `openclaw browser --browser-profile openclaw open <url>`.· uses the networkSKILL.md:If the host cannot keep a long-running process alive, run `serve --print-urls` to show the URLs and ask the user to start `serve` in a local terminal or background job.
Attribution and licence
- First committed by
wzdavid
- Publisher
- Wei Zhang (Community)
- Source
- wzdavid/ThinkWiki · commit 89f477d
- Licence
- MIT License (files served here)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
- Contributors
- wzdavid
AutoGineer is not affiliated with or endorsed by wzdavid.
Install from source
5 of its files are not served here, so there is no ZIP. wzdavid/ThinkWiki at 89f477d