Generated banner for ThinkWiki
Generated logo for ThinkWiki

ThinkWiki

LLM wiki kit

First committed by wzdavid

Published in wzdavid/ThinkWiki. Publisher page

38 GitHub stars

What it is

Install once, talk to your agent. ThinkWiki turns documents and notes into a local Markdown wiki with inbox review, content knowledge graph, and browsable HTML workspace.
— the description by wzdavid

AutoGineer overview, from the brain’s own files

ThinkWiki is an agent-native skill designed to turn scattered information into a structured, durable local workspace. By integrating directly with skills-compatible AI agents, it allows users to manage their knowledge base through natural conversation rather than manual CLI commands. It supports a wide range of input formats, including PDFs, web pages, and various document types, organizing them into a system that features an inbox for review, a local viewer, and an interactive content knowledge graph. The system is built to be local-first, ensuring that Markdown files remain the source of truth, while providing HTML-based interfaces for visualizing relationships between topics, entities, and claims.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

ThinkWiki functions as a skill installed into an AI agent's environment. It uses a set of Python scripts to manage a local directory of Markdown files. The system processes inputs through an inbox workflow, where content is reviewed before being ingested into the wiki. It generates a series of HTML outputs, including a workspace home, an inbox console, a page viewer, and a graph explorer. Users interact with these features by asking their agent to perform tasks like clipping content, building the knowledge graph, or serving the workspace via a local loopback server.

Folders
FolderRoleFiles
.githubagent-config1
docsdocs41
scriptsscripts34
templatestemplates13
teststests1

The agent reads

Requirements

Runtimes: Python

API keys (by name): THINKWIKI_EMBED_API_KEY, THINKWIKI_LLM_API_KEY

Set it up

  1. Clone the repository at the commit AutoGineer scanned:

    git clone https://github.com/wzdavid/ThinkWiki.git && cd ThinkWiki && git checkout 89f477d7a4095ba2ecf49ef9332e240c6f1208a1
  2. cd ThinkWiki && claude

  3. AutoGineer scanned these files at commit 89f477d.

  4. Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/wzdavid/ThinkWiki/blob/89f477d7a4095ba2ecf49ef9332e240c6f1208a1/README.md#manual-install-optional
  5. First committed by wzdavid. Published in wzdavid/ThinkWiki. Licence: MIT License.

Usage

AutoGineer overview, from the brain’s own files

  • Initialize a new local wiki workspace
  • Clip web pages, files, or text into an inbox for later processing
  • Ingest reviewed items into the permanent wiki structure
  • Query the wiki for information with evidence-based responses
  • Generate and visualize a content knowledge graph
  • Serve the wiki workspace as a local HTML site for browser-based navigation
“Create a local knowledge base named Research Notes in my workspace.”
The agent initializes a new ThinkWiki workspace structure in the specified directory.
“Clip this article into the inbox: https://example.com/article”
The agent saves the content of the URL into the ThinkWiki inbox for later review.
“What did we already decide about context budgets?”
The agent searches the existing wiki pages and provides an answer supported by evidence from the knowledge base.
“Build the knowledge graph and open the workspace home in my browser.”
The agent generates the graph visualization and starts the local server to display the workspace in the browser.

Documentation by wzdavid

ThinkWiki

Image: Release Image: License Image: Python

ThinkWiki is an agent-native local knowledge base skill. Talk to your agent to turn scattered documents, web pages, notes, and conversations into a durable Markdown workspace — with inbox review, local browsing, and an interactive content knowledge graph you can open in a browser.

Chinese version: README.zh.md

Repository: https://github.com/wzdavid/ThinkWiki

Works With Skills-Compatible Agents

ThinkWiki follows the open Agent Skills format (SKILL.md + bundled scripts). If your agent can install skills and run local commands, it can run ThinkWiki.

You do not need to memorize CLI commands. Install the skill once, then create and manage your wiki entirely through conversation.

Agent / HostTypical use
Claude CodeInstall the repo as a skill, then ask it to build and query your wiki
OpenClawFull skill workflow; use serve + the built-in browser to open HTML outputs
TraeInstall under .trae/skills, then manage the wiki in chat
Hermes AgentLocal skill directory + shell execution
OpenAI Codex / Codex CLISkills-compatible coding agents with local file access
CursorAgent mode with project skills
Gemini CLISkills-compatible CLI agent
GitHub CopilotAgent / coding workflows in VS Code that support skills
Other skills hostsAny product that loads SKILL.md and can run python3 scripts/thinkwiki ... locally

Requirement: a local environment with Python 3. ThinkWiki bootstraps its own runtime on first use. Pure cloud chat without local file or shell access cannot run a local wiki.

Get Started: Install Through Conversation

Point your agent at this repository and ask it to install the skill for you.

Example prompt:

Please install the ThinkWiki skill from https://github.com/wzdavid/ThinkWiki , bootstrap the runtime, and run a health check to confirm everything works on my machine.

The agent will typically clone or copy the skill into your host's skills directory, run bootstrap, and verify capabilities with doctor. You should not need to run those steps yourself unless you prefer manual setup.

After installation, start using it immediately:

Create a local knowledge base named My Wiki in my workspace.

Clip this article into the inbox first: https://example.com/article

Import this PDF into the wiki: /path/to/file.pdf

Answer from my wiki: what did we already decide about context budgets?

Build the knowledge graph and open the workspace home in my browser.

The agent reads SKILL.md, maps your intent to stable ThinkWiki actions, and reports what changed — including paths to Markdown pages and HTML outputs.

Why ThinkWiki

  • Agent-first: users talk to an agent, not to a long list of commands.

  • Local-first: Markdown files remain the source of truth on your machine.

  • HTML-first: inbox, viewer, graph, and governance pages are real browsable workspaces.

  • Knowledge-first: the graph is a content knowledge graph — topics, concepts, entities, claims, and governance — not just a file-link map.

What It Produces

ThinkWiki generates a workspace that includes:

  • output/index.html: unified workspace home

  • output/inbox/index.html: inbox review console

  • output/viewer/index.html: local page browser

  • output/graph/index.html: interactive graph explorer

  • output/graph/report.html: graph governance report

  • output/graph/entity-merge-review.html: entity merge review page

  • output/graph/entity-merge-plan.html: entity merge dry-run preview when requested

Preview
Workspace Home (output/index.html)

The unified workspace home summarizes recent changes, recommended next actions, inbox backlog, and a graph snapshot so you can decide whether to read, ingest, or explore.

Image: ThinkWiki workspace home preview

Inbox Review (output/inbox/index.html)

Review clipped web pages, files, and notes before formal ingest. Items are grouped into ready, review, and weak so you can process the highest-value captures first.

Image: ThinkWiki inbox review preview

Local Viewer (output/viewer/index.html)

Browse the entire wiki by page type, confidence, and status. Open any page directly in the viewer without leaving the local HTML workspace.

Image: ThinkWiki local viewer preview

Knowledge Graph (output/graph/index.html)

Explore the content knowledge graph interactively. Switch between knowledge, document, and suggested views to inspect semantic relations and candidate edges.

Image: ThinkWiki knowledge graph preview

Graph Governance Report (output/graph/report.html)

Inspect graph health signals such as isolated pages, hub stubs, fragile bridges, suggested links, and entity merge candidates before making structural changes.

Image: ThinkWiki graph governance report preview

Everyday Commands (In Plain Language)

Once ThinkWiki is installed, these are the kinds of requests users make most often:

You sayThinkWiki does
Create a wiki called Research NotesInitializes a local workspace
Clip / save this URL or file to inboxCaptures content for later review
Ingest the ready inbox itemsPromotes reviewed captures into wiki pages
What does my wiki say about X?Evidence-first Q&A from existing pages
Save this answer as a query pagePersists high-value outputs
Show me the knowledge graphBuilds/refreshes graph HTML
Review entity merge candidatesSurfaces alias collisions for manual confirmation
Open the wiki workspace in my browserRuns serve and returns http://127.0.0.1:8765/index.html

Core Capabilities

  • Initialize a local wiki workspace

  • Clip web pages, files, or pasted text into an inbox for later review

  • Import Markdown, PDF, DOCX, XLSX, XLS, PPTX, web pages, and plain text

  • Answer questions from existing wiki pages with evidence-first behavior

  • Save high-value outputs as query, synthesis, decision, or concept pages

  • Build a content knowledge graph with knowledge, document, and suggested views

  • Generate entity governance outputs, including ambiguous alias review and deterministic merge flows

  • Check runtime health, workspace status, and graph governance state

Content Knowledge Graph

ThinkWiki v1.6.0 builds a schema v2 graph with default_view = knowledge.

The knowledge view can contain:

  • page-backed nodes such as source, topic, concept, decision, synthesis, query, and entity

  • claim nodes extracted from structured wiki content

  • semantic relations such as about, belongs_to, depends_on, asserts, supports, contradicts, and suggests_related_to

This means the graph is not limited to raw file references. It can represent knowledge structure, evidence structure, and entity governance directly.

Browse HTML Outputs

Agent chat UIs usually cannot render ThinkWiki HTML inline. Ask your agent to start the built-in loopback server when you want to inspect inbox, viewer, graph, or governance pages:

Start the ThinkWiki output server and give me the workspace URL.

Or run it yourself:

python3 scripts/thinkwiki serve --root /path/to/my-wiki

By default this serves <wiki-root>/output/ at http://127.0.0.1:8765/. Start from http://127.0.0.1:8765/index.html.

On OpenClaw, after serve is running, the agent can also open the workspace home with:

openclaw browser --browser-profile openclaw open http://127.0.0.1:8765/index.html

On other agents, open the same URL in your system browser, or use whatever browser tool the host provides.

Use serve --print-urls when you only need the URL list without starting the server.

Environment Variables

Remote AI features are disabled by default. Configure only what you need:

Content generation (crystallize, digest)

Requires all three variables. Any OpenAI-compatible chat completion endpoint works (MiniMax, OpenAI, DeepSeek, local Ollama, etc.).

VariableRequiredPurpose
THINKWIKI_LLM_API_KEYYes, to enableAPI key for the configured LLM provider
THINKWIKI_LLM_BASE_URLYes, to enableChat completions URL, e.g. https://api.openai.com/v1/chat/completions
THINKWIKI_LLM_MODELYes, to enableModel name, e.g. gpt-4o-mini or MiniMax-M3
THINKWIKI_LLM_TEMPERATURENoOptional temperature override

Without a complete LLM configuration, crystallize and digest use local heuristics only.

Entity embedding (entity-merge-review, graph-report, health)

Enabled when THINKWIKI_EMBED_API_KEY is set. Defaults to SiliconFlow BGE-M3; override the endpoint or model if needed.

VariableRequiredPurpose
THINKWIKI_EMBED_API_KEYYes, to enableAPI key for the embedding provider
THINKWIKI_EMBED_BASE_URLNoEmbedding URL(s), comma-separated; default https://api.siliconflow.cn/v1/embeddings
THINKWIKI_EMBED_MODELNoEmbedding model; default BAAI/bge-m3

Register a free SiliconFlow key at https://siliconflow.cn if you want the default embedding setup.

Manual Install (Optional)

If you prefer to set up the skill yourself instead of asking an agent:

git clone https://github.com/wzdavid/ThinkWiki ThinkWiki
cd ThinkWiki
python3 scripts/thinkwiki bootstrap
python3 scripts/thinkwiki doctor --repo-root .

Then install the folder into your agent's skills directory according to that host's documentation.

CLI Reference (For Agents And Power Users)

ThinkWiki keeps one stable entry point that agents call under the hood:

python3 scripts/thinkwiki <command> [args]

Common commands: init, clip, ingest, inbox, viewer, graph, graph-report, entity-merge-review, entity-merge-apply, serve, health, status, doctor.

See SKILL.md for the agent behavior contract and intent mapping.

Repository Docs

  • README.md: concise English project overview

  • README.zh.md: Chinese overview

  • SKILL.md: skill behavior contract for agent hosts

  • CHANGELOG.md: release history

License

MIT

scripts/digest.py

Shown as plain text.

#!/usr/bin/env python3
from __future__ import annotations

"""
ThinkWiki Script: digest

Purpose:
- Create a synthesis page that combines multiple source pages into one reusable artifact.

Usage:
- Prefer `python scripts/thinkwiki digest ...`.
- Run `python scripts/<script> --help` for direct CLI details when the file exposes its own arguments.
"""


import argparse
import re
import sys
from pathlib import Path

from crystallize import first_meaningful_line, write_page
from ai_config import llm_is_configured, resolve_llm_config
from llm_client import llm_digest
from utils import file_uri, find_repo_root, normalize_repo_path, parse_frontmatter, read_text, refresh_output_home_if_present

def ordered_unique(items: list[str]) -> list[str]:
    seen: set[str] = set()
    result: list[str] = []
    for item in items:
        clean = " ".join(item.split()).strip()
        if not clean or clean in seen:
            continue
        seen.add(clean)
        result.append(clean)
    return result


def extract_title(body: str, fallback: str) -> str:
    for raw in body.splitlines():
        line = raw.strip()
        if line.startswith("# "):
            return line[2:].strip() or fallback
    return fallback


def source_page_companion(root: Path, page_path: Path) -> Path | None:
    if page_path.parent.name != "sources":
        return None
    text = read_text(page_path)
    meta, _body = parse_frontmatter(text)
    source_values = meta.get("sources", [])
    if not isinstance(source_values, list) or not source_values:
        return None
    raw_path = Path(str(source_values[0]))
    if len(raw_path.parts) < 3 or raw_path.parts[0] != "raw":
        return None
    normalized_path = root / Path("normalized", *raw_path.parts[1:]).with_suffix(".md")
    if normalized_path.exists():
        return normalized_path
    return None


def resolve_input_path(root: Path, value: str) -> Path:
    path = Path(value)
    if path.is_absolute():
        return path
    normalized = normalize_repo_path(root, value)
    return root / normalized


def collect_related_paths(root: Path, body: str, page_path: Path) -> list[str]:
    related: list[str] = []
    for match in re.findall(r"\[[^\]]+\]\(([^)]+)\)", body):
        if match.startswith(("http://", "https://", "mailto:", "#")):
            continue
        target = (page_path.parent / match).resolve()
        try:
            related.append(target.relative_to(root.resolve()).as_posix())
        except ValueError:
            continue
    return ordered_unique([item for item in related if item.startswith("wiki/") and item.endswith(".md")])


def source_record(root: Path, raw_path: Path, summary_kind: str = "synthesis") -> dict[str, object]:
    text = read_text(raw_path)
    if not text:
        raise SystemExit(f"Cannot read source content: {raw_path}")
    title = raw_path.stem
    source_paths: list[str] = []
    related_paths: list[str] = []

    if raw_path.suffix.lower() == ".md" and "wiki" in raw_path.parts:
        meta, body = parse_frontmatter(text)
        title = str(meta.get("title") or raw_path.stem)
        source_values = meta.get("sources", [])
        if isinstance(source_values, list):
            source_paths = [normalize_repo_path(root, str(item)) for item in source_values]
        related_paths = collect_related_paths(root, body, raw_path)
    else:
        body = text
        title = extract_title(body, raw_path.stem)
        if raw_path.is_relative_to(root):
            source_paths = [raw_path.relative_to(root).as_posix()]

    companion = source_page_companion(root, raw_path)
    companion_text = read_text(companion) if companion else ""
    primary_body = companion_text or body

    return {
        "path": raw_path,
        "title": title,
        "source_paths": ordered_unique(source_paths),
        "related_paths": ordered_unique(related_paths),
        "source_data": {"title": title, "body": primary_body},
    }


_HEURISTIC_BLOCKED_SECTIONS = {
    "Connections",
    "Related Pages",
    "Open Questions",
    "Consulted Pages",
    "Sources",
    "Raw Source",
    "Extracted Markdown",
    "Extracted Excerpt",
}
_HEURISTIC_META_PREFIXES = ("来源:", "作者:", "发布日期:", "原文链接:")
_HEURISTIC_LOW_VALUE_SUMMARY_PATTERNS = (
    "以下为",
    "点击查看",
    "原文链接",
)
_HEURISTIC_TENSION_HINTS = ("但", "不过", "仍", "需要", "风险", "问题", "挑战", "难", "冲突", "取舍", "边界", "验证", "确认")
_HEURISTIC_LOW_VALUE_LINE_HINTS = (
    "本报告旨在回答",
    "研究问题",
    "副院长",
    "资深专家",
    "日期:",
    "日期:",
    "page ",
)
_HEURISTIC_DEFINITION_HINTS = ("定义为", "本质上是", "指的是", "意味着", "可概括为")
_HEURISTIC_DECISION_HINTS = ("不适合", "应按", "应采用", "建议", "推荐", "换句话说", "关键在于", "核心判断")
_HEURISTIC_STRATEGY_HINTS = ("缓解策略", "首要评判标准", "产品管理思维", "开发者体验(DevEx)", "开发者体验(DevEx)")
_HEURISTIC_ROLE_HINTS = ("典型角色包括", "职责包括", "角色包括")
_HEURISTIC_SYNTHESIS_HINTS = ("这意味着", "因此", "换句话说", "核心判断", "说明", "首先是一种", "本质上是")
_HEURISTIC_ORGANIZATION_HINTS = ("团队", "组织", "平台", "协作", "运行机制", "交付系统")
_HEURISTIC_CONTINUATION_ENDINGS = tuple("的了和与及并而按把将向在于为是小会度案等其")


def _heuristic_normalize_text(text: str) -> str:
    return " ".join(text.replace("\r\n", "\n").replace("\r", "\n").split())


def _heuristic_plain_text(text: str) -> str:
    text = re.sub(r"!\[[^\]]*\]\([^)]+\)", "", text)
    text = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", text)
    text = re.sub(r"\[\[\d+\]\]\([^)]+\)", "", text)
    text = re.sub(r"\[\[\d+\]\]\(?", "", text)
    text = re.sub(r"https?://\S+", "", text)
    text = re.sub(r"[*_`~#>]+", " ", text)
    return _heuristic_normalize_text(text).strip(" -|,;")


def _heuristic_short_text(text: str, limit: int = 180) -> str:
    value = _heuristic_plain_text(text)
    if len(value) <= limit:
        return value
    window = value[: limit - 1]
    cut = max(window.rfind("。"), window.rfind(";"), window.rfind("."), window.rfind(";"), window.rfind(" "))
    if cut >= max(20, limit // 3):
        window = window[:cut]
    return window.rstrip() + "…"


def _heuristic_looks_like_placeholder(text: str) -> bool:
    compact = _heuristic_normalize_text(text).lower()
    if not compact:
        return True
    return compact in {"none yet", "todo", "- todo", "(no summary)"}


def _heuristic_cleaned_line(raw: str) -> str:
    return raw.strip().lstrip("-* ").strip()


def _heuristic_is_metadata_line(text: str) -> bool:
    return _heuristic_cleaned_line(text).startswith(_HEURISTIC_META_PREFIXES)


def _heuristic_is_link_only(text: str) -> bool:
    clean = _heuristic_cleaned_line(text)
    if clean.startswith(("http://", "https://", "<http://", "<https://")):
        return True
    return bool(re.fullmatch(r"\d+\.\s*https?://\S+", clean))


def _heuristic_low_value_summary(text: str) -> bool:
    clean = _heuristic_plain_text(text)
    if not clean:
        return True
    if _heuristic_is_metadata_line(clean) or _heuristic_is_link_only(clean):
        return True
    lowered = clean.lower()
    return any(clean.startswith(prefix) for prefix in _HEURISTIC_LOW_VALUE_SUMMARY_PATTERNS) or any(hint in lowered for hint in _HEURISTIC_LOW_VALUE_LINE_HINTS)


def _heuristic_split_sentences(text: str) -> list[str]:
    normalized = text.replace("\r\n", "\n").replace("\r", "\n")
    merged_lines: list[str] = []
    buffer = ""
    for raw in normalized.split("\n"):
        line = raw.strip()
        if not line:
            continue
        if not buffer:
            buffer = line
            continue
        if buffer.endswith(("。", "!", "?", ".", "!", "?", ";", ";", ":", ":")):
            merged_lines.append(buffer.strip())
            buffer = line
            continue
        if re.match(r"^(?:[-*]|\d+[.)、])\s*", line):
            merged_lines.append(buffer.strip())
            buffer = line
            continue
        buffer = f"{buffer} {line}"
    if buffer:
        merged_lines.append(buffer.strip())

    parts: list[str] = []
    for chunk in merged_lines:
        parts.extend(re.split(r"(?<=[。!?!?\.])\s+", chunk))
    return [part.strip(" -") for part in parts if part.strip(" -")]


def _heuristic_looks_incomplete_sentence(text: str) -> bool:
    clean = _heuristic_plain_text(text)
    if not clean:
        return True
    if clean.endswith(("。", "!", "?", ".", "!", "?")):
        return False
    if clean.endswith(("…", ";", ";", ":", ":")):
        return True
    if clean[-1] in _HEURISTIC_CONTINUATION_ENDINGS:
        return True
    if len(clean) < 30:
        return True
    return False


def _heuristic_is_low_value_sentence(text: str) -> bool:
    clean = _heuristic_plain_text(text)
    lowered = clean.lower()
    if len(clean) < 12:
        return True
    if _heuristic_looks_like_placeholder(clean):
        return True
    if _heuristic_is_metadata_line(clean) or _heuristic_is_link_only(clean):
        return True
    if any(hint in lowered for hint in _HEURISTIC_LOW_VALUE_LINE_HINTS):
        return True
    if clean.endswith((":", ":")):
        return True
    return False


def _heuristic_sentence_priority(text: str) -> int:
    clean = _heuristic_plain_text(text)
    score = 0
    if any(hint in clean for hint in _HEURISTIC_DEFINITION_HINTS):
        score += 8
    if any(hint in clean for hint in _HEURISTIC_DECISION_HINTS):
        score += 7
    if "定义为" in clean:
        score += 8
    if "不适合" in clean or "应采用" in clean or "推荐采用" in clean:
        score += 10
    if "本质上是" in clean:
        score += 4
    if "指的是" in clean:
        score += 2
    if len(clean) >= 30:
        score += 2
    if len(clean) >= 80:
        score += 2
    if len(clean) > 200:
        score -= 2
    if "?" in clean or "?" in clean:
        score -= 6
    return score


def _heuristic_summary_candidate_score(text: str) -> int:
    clean = _heuristic_plain_text(text)
    score = _heuristic_sentence_priority(clean)
    if len(clean) < 24:
        score -= 6
    elif len(clean) <= 140:
        score += 4
    elif len(clean) <= 220:
        score += 1
    else:
        score -= 4
    has_terminal_punctuation = clean.endswith(("。", "!", "?", ".", "!", "?"))
    if has_terminal_punctuation:
        score += 4
    else:
        score -= 14
    if clean.endswith(("…", ";", ";", ":", ":")):
        score -= 6
    if clean and clean[-1] in _HEURISTIC_CONTINUATION_ENDINGS:
        score -= 30
    if re.search(r"\d+$", clean):
        score -= 4
    return score


def _heuristic_kind_summary_score(text: str, kind: str, title: str = "") -> int:
    clean = _heuristic_plain_text(text)
    score = _heuristic_summary_candidate_score(clean)
    normalized_kind = kind.strip().lower()
    title_clean = _heuristic_plain_text(title)

    if normalized_kind == "concept":
        if any(hint in clean for hint in _HEURISTIC_DEFINITION_HINTS):
            score += 18
        if "首先是一种" in clean or "可一句话概括" in clean:
            score += 10
        if "关键不在于" in clean or "围绕可执行规格" in clean:
            score += 10
        if "软件交付团队" in clean or "其关键不在于" in clean:
            score += 12
        if title_clean and (clean.startswith(title_clean) or clean.startswith(f"{title_clean}(")):
            score += 14
        if title_clean and title_clean in clean and any(
            hint in clean for hint in ("定义为", "本质上是", "指的是", "意味着", "首先是一种")
        ):
            score += 8
        if clean.startswith("本报告不将") or "并列的独立概念来讨论" in clean:
            score -= 24
        if "而不是另一套平行的方法论" in clean:
            score -= 10
        if "本报告" in clean and ("系统论证" in clean or "应运而生" in clean):
            score -= 16
        if any(hint in clean for hint in _HEURISTIC_STRATEGY_HINTS):
            score -= 20
        if any(hint in clean for hint in _HEURISTIC_ROLE_HINTS):
            score -= 10
        if "团队应" in clean or "建议" in clean:
            score -= 8
    elif normalized_kind == "decision":
        if any(hint in clean for hint in _HEURISTIC_DECISION_HINTS):
            score += 18
        if "不适合" in clean or "应按" in clean or "应采用" in clean or "推荐采用" in clean:
            score += 14
        if any(hint in clean for hint in _HEURISTIC_DEFINITION_HINTS):
            score -= 6
        if clean.startswith(("缓解策略", "建议采用")) and "不适合" not in clean and "应按" not in clean:
            score -= 12
    else:
        if any(hint in clean for hint in _HEURISTIC_SYNTHESIS_HINTS):
            score += 12
        if any(hint in clean for hint in _HEURISTIC_DEFINITION_HINTS):
            score += 8
        if any(hint in clean for hint in _HEURISTIC_DECISION_HINTS):
            score += 8
        if "首先是一种组织模式" in clean or "组织模式而非工具清单" in clean:
            score += 16
        if "关键不在于" in clean and "团队是否" in clean:
            score += 10
        if "不能只理解为" in clean or "真正难点并不在于" in clean:
            score += 12
        if clean.startswith("本报告不将") or "并列的独立概念来讨论" in clean:
            score -= 20
        if "而不是另一套平行的方法论" in clean:
            score -= 10
        if any(hint in clean for hint in _HEURISTIC_STRATEGY_HINTS):
            score -= 18
        if any(hint in clean for hint in _HEURISTIC_ROLE_HINTS):
            score -= 10
        if any(hint in clean for hint in _HEURISTIC_ORGANIZATION_HINTS):
            score += 4
        if "团队应" in clean or "建议采用" in clean:
            score -= 8
        if len(clean) > 180:
            score -= 12
        if len(clean) > 240:
            score -= 10
        if clean.count("。") >= 3:
            score -= 6

    if clean.count(";") + clean.count(";") >= 2:
        score -= 4
    return score


def _heuristic_summary_sentence_candidates(text: str, summary_kind: str, title: str, limit: int = 12) -> list[str]:
    candidates: list[tuple[int, int, str]] = []
    for index, part in enumerate(_heuristic_split_sentences(text)):
        clean = _heuristic_plain_text(part).strip(" -")
        if clean.startswith("#") or _heuristic_is_low_value_sentence(clean):
            continue
        candidates.append((_heuristic_kind_summary_score(clean, summary_kind, title), index, clean))
    candidates.sort(key=lambda item: (-item[0], item[1], item[2]))
    return [item[2] for item in candidates[:limit]]


def _heuristic_choose_best_summary(summary: str, primary_body: str, title: str, summary_kind: str = "synthesis") -> str:
    candidates: list[str] = []
    summary_clean = _heuristic_plain_text(summary)
    if summary_clean and not _heuristic_low_value_summary(summary_clean):
        candidates.append(summary_clean)
    candidates.extend(_heuristic_summary_sentence_candidates(primary_body, summary_kind, title, limit=12))
    candidates = ordered_unique(candidates)
    if not candidates:
        return title
    scored = sorted(
        ((_heuristic_kind_summary_score(candidate, summary_kind, title), index, candidate) for index, candidate in enumerate(candidates)),
        key=lambda item: (-item[0], item[1], item[2]),
    )
    return scored[0][2]


def _heuristic_extract_sections(body: str) -> dict[str, str]:
    sections: dict[str, list[str]] = {}
    current = "Overview"
    sections[current] = []
    for raw in body.splitlines():
        line = raw.rstrip()
        stripped = line.strip()
        if stripped.startswith("## "):
            current = stripped[3:].strip() or "Overview"
            sections.setdefault(current, [])
            continue
        if stripped.startswith("### "):
            current = stripped[4:].strip() or current
            sections.setdefault(current, [])
            continue
        sections.setdefault(current, []).append(line)
    return {name: "\n".join(lines).strip() for name, lines in sections.items()}


def _heuristic_clean_content_lines(text: str) -> list[str]:
    lines: list[str] = []
    for raw in text.splitlines():
        line = raw.strip()
        if not line:
            continue
        if line in {"---", "***"}:
            continue
        if line.startswith("#"):
            continue
        if line.startswith("![]("):
            continue
        if _heuristic_is_metadata_line(line):
            continue
        if line.startswith(("更新时间", "更新于", "Published:", "Updated:")):
            continue
        if _heuristic_is_link_only(line):
            continue
        lines.append(line)
    return lines


def _heuristic_meaningful_sentences(text: str, limit: int = 8) -> list[str]:
    candidates: list[tuple[int, int, str]] = []
    for index, part in enumerate(_heuristic_split_sentences(text)):
        clean = _heuristic_plain_text(part).strip(" -")
        if clean.startswith("#") or _heuristic_is_low_value_sentence(clean):
            continue
        if _heuristic_looks_incomplete_sentence(clean):
            continue
        candidates.append((_heuristic_sentence_priority(clean), index, clean))
    candidates.sort(key=lambda item: (-item[0], item[1], item[2]))
    return [item[2] for item in candidates[:limit]]


def _heuristic_likely_tension(text: str) -> bool:
    clean = _heuristic_plain_text(text)
    if len(clean) < 16:
        return False
    return any(hint in clean for hint in _HEURISTIC_TENSION_HINTS)


def _heuristic_source_record(root: Path, raw_path: Path, summary_kind: str = "synthesis") -> dict[str, object]:
    text = read_text(raw_path)
    if not text:
        raise SystemExit(f"Cannot read source content: {raw_path}")
    record_type = "normalized" if "normalized" in raw_path.parts else "page"
    title = raw_path.stem
    summary = ""
    source_paths: list[str] = []
    related_paths: list[str] = []

    if raw_path.suffix.lower() == ".md" and "wiki" in raw_path.parts:
        meta, body = parse_frontmatter(text)
        title = str(meta.get("title") or raw_path.stem)
        summary = str(meta.get("summary") or "").strip()
        source_values = meta.get("sources", [])
        if isinstance(source_values, list):
            source_paths = [normalize_repo_path(root, str(item)) for item in source_values]
        related_paths = collect_related_paths(root, body, raw_path)
    else:
        body = text
        title = extract_title(body, raw_path.stem)
        summary_candidates = _heuristic_clean_content_lines(body)
        summary = _heuristic_short_text(summary_candidates[0]) if summary_candidates else ""
        if raw_path.is_relative_to(root):
            source_paths = [raw_path.relative_to(root).as_posix()]

    companion = source_page_companion(root, raw_path) if record_type == "page" else None
    companion_text = read_text(companion) if companion else ""
    primary_body = companion_text or body
    companion_summary = (_heuristic_meaningful_sentences(primary_body, limit=1) or [""])[0]
    sections = _heuristic_extract_sections(primary_body)
    key_texts: list[tuple[int, str]] = []
    tension_texts: list[str] = []

    for section_name, section_body in sections.items():
        if section_name in _HEURISTIC_BLOCKED_SECTIONS:
            continue
        for line in _heuristic_meaningful_sentences(section_body, limit=6):
            clean = _heuristic_cleaned_line(line)
            if _heuristic_is_low_value_sentence(clean):
                continue
            if section_name in {"Open Questions", "Tensions", "Follow-ups"}:
                tension_texts.append(clean)
            else:
                key_texts.append((_heuristic_sentence_priority(clean), clean))

    key_sentences = _heuristic_meaningful_sentences(primary_body, limit=10)
    if not tension_texts:
        tension_texts.extend([item for item in _heuristic_meaningful_sentences(primary_body, limit=12) if _heuristic_likely_tension(item)][:3])
    summary = _heuristic_short_text(_heuristic_choose_best_summary(summary or companion_summary, primary_body, title, summary_kind))

    return {
        "path": raw_path,
        "title": title,
        "summary": summary,
        "type": record_type,
        "source_paths": ordered_unique(source_paths),
        "related_paths": ordered_unique(related_paths),
        "findings": ordered_unique([item for _score, item in sorted(key_texts, key=lambda value: (-value[0], value[1]))] + key_sentences),
        "tensions": ordered_unique(tension_texts),
    }


def _heuristic_auto_summary(records: list[dict[str, object]], fallback: str, summary_kind: str = "synthesis") -> str:
    parts = [str(record["summary"]).strip() for record in records if str(record["summary"]).strip()]
    parts = ordered_unique(parts)
    if not parts:
        return fallback
    if len(parts) == 1:
        return _heuristic_short_text(parts[0], limit=220)
    ranked = sorted(
        ((_heuristic_kind_summary_score(part, summary_kind, fallback), index, part) for index, part in enumerate(parts)),
        key=lambda item: (-item[0], item[1], item[2]),
    )
    lead = ranked[0][2]
    if summary_kind == "synthesis":
        preferred = next(
            (
                part
                for score, _index, part in ranked
                if len(part) <= 180
                and part.endswith(("。", "!", "?", ".", "!", "?"))
                and (
                    any(hint in part for hint in _HEURISTIC_DECISION_HINTS)
                    or "首先是一种组织模式" in part
                    or "关键不在于" in part
                    or "不能只理解为" in part
                    or "软件交付团队" in part
                )
                and score >= ranked[0][0] - 6
            ),
            "",
        )
        if preferred:
            lead = preferred
    support = next((part for _score, _index, part in ranked[1:] if part != lead), "")
    if support:
        return _heuristic_short_text(f"{lead} 这一判断也得到其他来源的补充支持,说明相关组织结论具有跨材料一致性。", limit=220)
    return _heuristic_short_text(lead, limit=220)


def _heuristic_auto_findings(records: list[dict[str, object]], limit: int = 6) -> list[str]:
    findings: list[str] = []
    for record in records:
        title = str(record["title"])
        for item in list(record["findings"])[:3]:
            item = _heuristic_short_text(item, limit=150)
            prefix = f"{title}: "
            findings.append(prefix + item if not item.startswith(title) else item)
            if len(findings) >= limit:
                return ordered_unique(findings)
    return ordered_unique(findings)


def _heuristic_auto_tensions(records: list[dict[str, object]], limit: int = 4) -> list[str]:
    tensions: list[str] = []
    for record in records:
        title = str(record["title"])
        for item in list(record["tensions"])[:2]:
            item = _heuristic_short_text(item, limit=150)
            tensions.append(f"{title}: {item}")
            if len(tensions) >= limit:
                return ordered_unique(tensions)
        if not list(record["tensions"]):
            hinted = [item for item in list(record["findings"]) if _heuristic_likely_tension(item)][:2]
            for item in hinted:
                item = _heuristic_short_text(item, limit=150)
                tensions.append(f"{title}: {item}")
                if len(tensions) >= limit:
                    return ordered_unique(tensions)
    summaries = ordered_unique([str(record["summary"]).strip() for record in records if str(record["summary"]).strip()])
    if len(summaries) > 1:
        tensions.append("不同来源强调的重点并不完全一致,需要人工确认哪些结论应上升为稳定知识。")
    if not tensions and len(records) >= 2:
        tensions.append("当前 digest 主要做信息汇总,是否存在冲突、时效差异或证据缺口仍需进一步审阅。")
    if not tensions:
        tensions.append("当前输入来源较少,尚未形成明显冲突,但仍需要继续补充资料与交叉验证。")
    return ordered_unique(tensions[:limit])


def _heuristic_auto_content(summary: str, findings: list[str], tensions: list[str]) -> str:
    lines = [summary.strip()]
    if findings:
        lines.extend(["", "Findings:"])
        lines.extend(f"- {item}" for item in findings[:4])
    if tensions:
        lines.extend(["", "Tensions:"])
        lines.extend(f"- {item}" for item in tensions[:3])
    return "\n".join(line for line in lines if line is not None).strip()


def _fallback_digest(root: Path, source_paths: list[str], title: str, content_fallback: str = "") -> dict[str, object]:
    records: list[dict[str, object]] = []
    for raw in source_paths:
        path = resolve_input_path(root, raw)
        if not path.exists():
            continue
        records.append(_heuristic_source_record(root, path, summary_kind="synthesis"))
    fallback_str = first_meaningful_line(content_fallback, title)
    findings = _heuristic_auto_findings(records)
    tensions = _heuristic_auto_tensions(records)
    summary = _heuristic_auto_summary(records, fallback_str, summary_kind="synthesis")
    body = _heuristic_auto_content(summary, findings, tensions)
    return {
        "summary": summary,
        "key_points": [],
        "body": body,
        "findings": findings,
        "tensions": tensions,
    }


def main() -> int:
    parser = argparse.ArgumentParser(description="Persist a multi-page digest into wiki/syntheses.")
    parser.add_argument("--root", default=".", help="Wiki root path")
    parser.add_argument("--title", required=True, help="Digest title")
    parser.add_argument("--summary", default="", help="Short summary stored in frontmatter")
    parser.add_argument("--content", default="", help="Optional long-form digest body")
    parser.add_argument("--source-path", action="append", default=[], help="Consulted wiki page path")
    parser.add_argument("--related-path", action="append", default=[], help="Related wiki page path")
    parser.add_argument("--finding", action="append", default=[], help="Key finding")
    parser.add_argument("--tension", action="append", default=[], help="Open tension or conflict")
    parser.add_argument("--slug", default="", help="Explicit target slug")
    parser.add_argument("--update", action="store_true", help="Update an existing digest page with the same title or slug")
    parser.add_argument("--merge-mode", choices=["append", "replace", "dedupe"], default="dedupe", help="How to merge fields when --update is used")
    args = parser.parse_args()

    root = find_repo_root(Path(args.root))
    records: list[dict[str, object]] = []
    if args.source_path:
        for raw in args.source_path:
            path = resolve_input_path(root, raw)
            if not path.exists():
                raise SystemExit(f"Source path not found: {raw}")
            records.append(source_record(root, path, summary_kind="synthesis"))

    auto_source_paths = ordered_unique([
        source_path
        for record in records
        for source_path in list(record["source_paths"]) or ([record["path"].relative_to(root).as_posix()] if Path(record["path"]).is_relative_to(root) else [])
    ])
    auto_related_paths = ordered_unique([item for record in records for item in list(record["related_paths"])])

    source_data_list = [record["source_data"] for record in records]
    if llm_is_configured():
        llm_config = resolve_llm_config()
        print(
            f"Notice: sending source content to configured LLM at {llm_config.base_url} "
            f"(model: {llm_config.model}).",
            file=sys.stderr,
        )
        try:
            llm_result = llm_digest(source_data_list, args.title, raise_on_failure=True)
        except Exception:
            print("Warning: LLM failed, falling back to heuristic mode.", file=sys.stderr)
            llm_result = _fallback_digest(root, args.source_path, args.title, args.content)
    else:
        llm_result = _fallback_digest(root, args.source_path, args.title, args.content)

    summary = args.summary.strip() or str(llm_result["summary"])
    content = args.content.strip() or str(llm_result["body"])
    page_path, action = write_page(
        root=root,
        kind="synthesis",
        title=args.title,
        summary=summary,
        content=content,
        source_paths=ordered_unique(args.source_path + auto_source_paths) or ["index.md"],
        related_paths=ordered_unique(args.related_path + auto_related_paths),
        follow_ups=[],
        findings=ordered_unique(args.finding + list(llm_result["findings"])),
        tensions=ordered_unique(args.tension + list(llm_result["tensions"])),
        key_points=[],
        action_label="digest",
        slug=args.slug,
        update=args.update,
        merge_mode=args.merge_mode,
    )
    output_home = refresh_output_home_if_present(root)
    print(f"{action.title()} {page_path.relative_to(root).as_posix()}")
    if output_home is not None:
        print("Output hub: output/index.html")
        print(f"Output hub URI: {file_uri(output_home)}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
98 files · 5 not served here

Advantages

  • Enables natural language management of a knowledge base without requiring the user to memorize complex CLI commands. evidence: README.md
  • Maintains Markdown files as the local source of truth, ensuring data portability and ownership. evidence: README.md
  • Provides a rich, interactive content knowledge graph that maps semantic relations like claims and entities rather than just file links. evidence: README.md

Limitations

  • Requires a local environment with Python 3 and cannot function in pure cloud-based chat environments without local file or shell access. evidence: README.md
  • Agent chat UIs typically cannot render the generated HTML interfaces inline, requiring the use of a separate browser or a local loopback server. evidence: README.md

Security

Risk level: No risk found

This brain runs

Agent instructions that run or fetch something (5)

  • SKILL.md: If the user wants a new workspace, run `init`.
  • SKILL.md: After `viewer`, `graph`, `inbox`, or `graph-report`, run `serve` when the user wants to browse the results.
  • SKILL.md: If a long-running `serve` process is not practical in the current session, run `serve --print-urls` and tell the user to start `serve` locally, then open the printed workspace URL.
  • SKILL.md: After starting `serve`, give the user the workspace home URL first (`http://127.0.0.1:8765/index.html` by default) and, in OpenClaw, offer to open it with `openclaw browser --browser-profile openclaw open <url>`. · uses the network
  • SKILL.md: If the host cannot keep a long-running process alive, run `serve --print-urls` to show the URLs and ask the user to start `serve` in a local terminal or background job.

Attribution and licence

First committed by
GitHub avatar of wzdavidwzdavid
Publisher
Wei Zhang (Community)
Source
wzdavid/ThinkWiki · commit 89f477d
Licence
MIT License (files served here)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
wzdavid

AutoGineer is not affiliated with or endorsed by wzdavid.

Install from source

5 of its files are not served here, so there is no ZIP. wzdavid/ThinkWiki at 89f477d