business-context-os

Wiki Zone Architecture

How docs/_wiki/ works in CLEAR Context OS — what it holds, how content gets in, how it stays healthy, how its vocabularies grow as the business grows, and how it stays distinct from the other zones (data points, collections, inbox, planned, archive). Designed to be AI-friendly today and graph/RAG-ready tomorrow.

For the sibling zone mechanics see collections-zone.md and content-routing.md. For the philosophical placement of zones see system-design.md. For the canonical zone registry that names wiki alongside its eleven siblings, see context-zones.md — wiki is one zone among many; the registry is what every cross-zone consumer reads.

Net summary in one line: keep pin-llm-wiki's clever mechanics (HTML-comment DSL, banner citations, three structural shapes, fetch protocols, soft-delete cascade), and replace its loose semantics (tags, related, product, version-less frontmatter, advisory-only lint) with BCOS's CLEAR ownership, frontmatter hook, propagation-based staleness, scheduled maintenance jobs, and a schema-driven governance layer so vocabularies grow with the business instead of being chiseled in stone.

Why a Wiki Zone Exists

BCOS is built around canonical, exclusively-owned data points: each fact has exactly one home, and we link instead of duplicating. That model is excellent for what is true and terrible for everything else.

The wiki is BCOS's universal long-form / cross-cutting content destination — canonical per plugin-storage-contract.md Rule 2. It holds three broad categories of content that don't fit the data-point model:

1. Operational truth (authority: canonical-process) — how the business actually runs:

• Runbooks (rebuilding document-index.md when frontmatter is corrupt; deploying the dashboard)

• How-tos (onboarding a new client to BCOS; running discovery calls)

• SOPs and playbooks (sales follow-up cadence; the meeting-transcript ingestion pipeline)

• Decision logs (why we picked Postgres over Mongo in 2025)

• Post-mortems (the March pricing migration incident)

• Scripts-with-context (filename-sanitization logic for the Zapier transcript ingestor, with the WHY around it)

2. Internal reference (authority: internal-reference) — derivative explainers for skimming:

• Glossaries that define "cluster," "data point," "document"

• FAQs reformatted from the brand-voice data point

• Tutorials onboarding newcomers to a domain

3. Plugin cross-cutting content and external sources — content that crosses plugin boundaries or originates outside:

• Meeting transcripts (informational, not legal evidence)

• WhatsApp / Slack exports

• Customer-call notes, email captures, research clippings

• Charters tied to plugin records

• External source summaries captured from websites, GitHub repos, YouTube talks (authority: external-reference)

These pages explain, operate on, reference, or summarize for skimming the canonical truths in docs/*.md — or capture the surrounding context plugins and consumers need. They are derivative by design: they cite upstream data points via builds-on:, never replace them. That's what docs/_wiki/ is for.

What does NOT belong in the wiki:

• Canonical data points — those live in docs/*.md. If a wiki page tries to own canonical content, the duplication-vs-data-point lint catches it.

• Plugin-private structured data (initiative records, task JSON, SQLite) — those live in docs/_<plugin>/ per Rule 1.

• Legal-weight evidence (signed contracts, regulatory filings, audited reports) — those live in _collections/ per Rule 3. Informational artifacts (a meeting transcript, an email export) → wiki. Legal artifacts → collections.

Where Wiki Sits Among the Zones

Each underscore-prefixed zone in BCOS carries a different truth-mode. The wiki zone slots in alongside the others without overlap.

The four tightest distinctions:

1. Wiki vs. data point. Data point: "Our target audience is small fintech teams in EU." Wiki: "How we run discovery calls to validate audience assumptions."

2. Wiki vs. collection. Collection: the signed Acme contract PDF. Wiki: a how-to titled "Reading Our Acme Contract."

3. Wiki vs. _planned/. _planned/ = future business state. Wiki = current operational knowledge.

4. Wiki vs. _inbox/. _inbox/ = temporary triage. Wiki = permanent with review cycle.

Folder Layout

Pages flat under _wiki/pages/. Page-type is the only discriminator (frontmatter, not folder). External URL captures keep their own subdirectory because they have a distinct lifecycle (URL-fetched, structural shapes, refresh-must-rediscover applies).

docs/_wiki/
├── .config.yml                   # Wiki-zone runtime config (detail level, cadence, source types)
├── .schema.yml                   # Vocabulary registry (page-types, statuses, detail-levels — see "Schema and Governance")
├── README.md                     # Zone overview + decision tree (auto-maintained)
├── index.md                      # Page table — DERIVED ARTIFACT (regenerated by refresh_wiki_index.py; never hand-edited)
├── overview.md                   # Cross-page synthesis with [[wikilinks]]
├── log.md                        # Append-only ingest/refresh log
├── queue.md                      # Pending URLs (Path A) — renamed from pin-llm-wiki's inbox.md to disambiguate from docs/_inbox/
│
├── pages/                        # All internally-authored pages (flat; page-type discriminator in frontmatter)
│   ├── onboarding-flow.md        # page-type: how-to
│   ├── data-point-glossary.md    # page-type: glossary
│   ├── q1-pricing-incident.md    # page-type: post-mortem
│   └── …
│
├── source-summary/               # External URL captures (Path A only); has umbrella/sub/unified shape rules
│   ├── langchain-com.md
│   ├── langchain-com-langgraph.md
│   └── …
│
├── raw/
│   ├── web/<slug>.md             # Pin-llm-wiki native: web captures
│   ├── github/<slug>.md          # Pin-llm-wiki native: GitHub repo captures
│   ├── youtube/<slug>.md         # Pin-llm-wiki native: YouTube transcripts
│   ├── local/<slug>.md           # BCOS extension: from Path B (inbox-promoted or user-provided)
│   └── <type>/_manifest.md       # Per-type manifest (fetch date, source URL/path, hash) — same pattern as _collections/<type>/_manifest.md
│
└── .archive/                     # Soft-deleted pages with dangling-reference scan history
    └── <archived-slug>.md

Schema fragments overlay (.schema.d/)

The wiki schema is layered. _wiki/.schema.yml is the base, owned by BCOS and refreshable per install. BCOS-enabled plugins (initiatives-os, executions-os, future siblings) extend the vocabulary via fragments dropped at:

docs/_wiki/
├── .schema.yml                       # base — user-owned, BCOS-template-seeded
└── .schema.d/                        # overlay — plugin-owned
    ├── initiatives-os.yml            # one file per plugin; shipped via plugin install_here.py
    ├── executions-os.yml
    └── <future-plugin>.yml

Wiki tooling (frontmatter validation hook, /wiki schema list, future lint passes) calls into .claude/scripts/_wiki_schema_merge.py which reads base + every .schema.d/*.yml and returns a merged registry. The base remains plugin-agnostic. update.py never touches .schema.d/ — fragments survive every framework update.

What a fragment may declare. Two extensible registries: cross-references: (plugin-defined frontmatter ref fields like initiative-refs, task-refs, stakeholder-refs) and raw-source-types: (plugin-defined raw subtypes like meeting, whatsapp, email). Fragment must also declare plugin: and plugin-version: for diagnostics.

What a fragment must NOT declare. Anything base-owned: page-types, statuses, detail-levels, authority-values, lint-checks, auto-fixes, clusters, thresholds, schema-version, migrations. The merger emits a fragment error if a base-owned key appears.

Conflict resolution. Two fragments declaring the same cross-reference field with identical attribute maps is a no-op (allowed — useful when plugins overlap on shared fields like stakeholder-refs). Two fragments declaring the same field with differing attributes is a hard merge error surfaced via /wiki schema list and the hook's _fragment-errors diagnostic.

Anti-patterns.

• Plugin's install_here.py hand-edits _wiki/.schema.yml directly. Causes plugin↔plugin collision (last-writer-wins on the shared file) and breaks clean uninstall. Use .schema.d/<plugin>.yml instead.

• Plugin-specific fields are added to upstream _wiki.schema.yml.tmpl. Pollutes BCOS core with vocabulary that is dead weight in non-plugin repos. Stays in plugin's fragment file.

• Fragments stored in any folder other than _wiki/.schema.d/. The merger only reads that one location; orphaned fragments elsewhere are invisible.

An example fragment is shipped at ../templates/_wiki.schema.d.example.yml.tmpl.

Why no per-page-type sub-folders. BCOS's standard is "flat in docs/ root with cluster: frontmatter for grouping, not subdirectories." Per-page-type folders (how-to/, runbook/, etc.) would duplicate the page-type: discriminator and force five places to touch when adding a new page-type: (1) frontmatter enum, (2) folder, (3) lint rules, (4) schema docs, (5) README. Flat keeps page-type: as the single source of truth — adding a new page-type is one schema edit. Folder is convenience; frontmatter is authority.

Why source-summary/ keeps its own folder. Source-summary pages are the only ones that have umbrella/sub/unified topology, are produced by a different ingest path, follow a different review cadence, and aligns with pin-llm-wiki's upstream layout. Co-locating them with internal pages would hide that asymmetry.

Why queue.md not inbox.md. Pin-llm-wiki's inbox.md is a single-file URL queue; BCOS's _inbox/ is a folder for triage. Different concepts, distinguishable names.

Why raw/ is nested inside _wiki/. Raw captures exist because of wiki ingest. _collections/ is for user-uploaded evidence with value independent of any wiki page. Co-location keeps the wiki ecosystem self-contained.

index.md is a derived artifact, not authored truth. It is a registry of pages — title, page-type, builds-on, last-reviewed — regenerated from frontmatter by refresh_wiki_index.py (shipped with the framework, wired into job-index-health.md). Hand-editing is wrong; if index.md looks stale, the fix is to run the refresh script, not to edit the file. This follows the same derived-artifact pattern as .claude/quality/ecosystem/state.json (lesson L-INIT-20260404-009: "State files record what discovery found, not the definition itself."). overview.md and log.md are NOT derived — they are authored by ingest (cross-source synthesis + append-only history).

Frontmatter Schema

BCOS's required fields, plus wiki-only discriminators. All fields kebab-case.

---
# ─── BCOS-required (same bar as data points) ────────────────────────────
version: 1.0.0
created: 2026-04-30                # immutable
last-updated: 2026-04-30           # bumped on every change
cluster: brand-identity            # primary cluster — must match an entry in cluster vocabulary (see Schema)
owner: gunti
status: active                     # value must come from .schema.yml `statuses:` list

# ─── BCOS ownership spec (Level 1 minimum) ──────────────────────────────
domain: "How to apply our brand voice on LinkedIn"
exclusively-owns:
  - LinkedIn-specific tone adjustments
  - LinkedIn formatting conventions
strictly-avoids:
  - The brand voice itself (lives in brand-voice.md)
  - Audience definitions (live in target-audience.md)

# ─── BCOS relationships (replaces pin-llm-wiki's `related:` and `tags:`) ─
# See "Reference format rule" — slugs intra-zone, paths cross-zone.
builds-on:                         # cross-zone (data points): relative paths with .md
  - ../brand-voice.md
  - ../target-audience.md
references:                        # intra-zone (other wiki pages): bare slugs, no .md
  - linkedin-best-practices
provides: []                       # rare for wiki — usually consumers, not producers

# ─── Wiki-specific ──────────────────────────────────────────────────────
page-type: how-to                  # value must come from .schema.yml `page-types:` registry
last-reviewed: 2026-04-30          # separate from last-updated; resets on human review

# ─── Provenance (NEW — for Path B local content; mirrors queue.md DSL for URLs) ──
provenance:
  kind: inbox-promotion            # url-fetch | inbox-promotion | local-document
  source-path: docs/_inbox/2026-04-15_acme-onboarding-notes.md  # original location
  captured-on: 2026-04-30
  promoted-by: gunti
  notes: ""                         # optional

# ─── Pin-llm-wiki shape discriminators (mutually exclusive — see "Three Structural Shapes") ──
# These are populated ONLY for page-type: source-summary
source-url: https://...
companion-urls: []
raw-files: []
subpages: []
parent-slug: null
detail-level: standard             # value must come from .schema.yml `detail-levels:` list
last-fetched: 2026-04-30
---

Page-type-specific frontmatter rules are defined in _wiki/.schema.yml (next-section). The frontmatter validation hook (PostToolUse on Edit/Write) reads the schema at validation time, so adding a new page-type is one schema edit — no code change.

Authority Semantics (schema 1.2)

Every wiki page declares an authority: value (or has one mechanically derived by the 1.1 -> 1.2 migration). Authority answers exactly one question:

How much trust does this page deserve when its claims conflict with canonical docs/*.md or with another wiki page?

Four values, ordered by trust-tier:

Mechanical default mapping

The 1.1 -> 1.2 migration writes authority for every existing page using path + page-type + provenance.kind:

external-evidence is never assigned mechanically. It must be declared explicitly. Most evidence belongs in _collections/, not _wiki/ — this value exists for the rare case where a wiki page IS the evidence (e.g., regulatory filings where paraphrasing risks compliance).

Override rule

The migration is non-clobbering: if a page already declares authority:, the migration leaves it alone. When a user-declared value disagrees with the mechanical default — for example, a source-summary/ page tagged authority: canonical-process — /wiki lint emits authority-default-questionable (INFO). The lint asks "is this intentional?" but doesn't block.

The full audit trail (every per-page derivation) is appended to .claude/quality/migration-log.jsonl, so reverse migration can reconstruct original frontmatter.

Anti-patterns

• Do not add a universal "always check the wiki first" rule. The wiki is layered; _wiki/source-summary/ is reference, not source-of-truth. The wake-up-context surfaces the authority hierarchy explicitly so consumers (LLM agents, builders, lint passes) read context with the right priority.

• Do not treat external-evidence as a synonym for "I'm pasting verbatim text." Most paste-verbatim content is external-reference. Evidence implies legal/regulatory weight where the file IS the artifact.

Temporal Semantics (schema 1.2)

Most apparent contradictions between sources are not contradictions — they are temporal supersession. 2024 OpenAI pricing vs 2025 vs 2026 are each correct as of their time. Schema 1.2 introduces three optional fields on source-summary to encode this:

A page can be a chain head (declares supersedes:) OR a chain tail (declares superseded-by:), not both. Lint enforces this via both-supersedes-and-superseded-by (ERROR).

Bidirectional rule

When ingest writes supersedes: [<predecessor-slug>] on page Y, it must also write superseded-by: <Y-slug> on the predecessor page. This mirrors the existing references: bidirectional rule in ingest.md Step 2a — a one-way link is a smell.

Cycle detection

A -> supersedes B -> supersedes A is rejected as supersession-cycle (ERROR). Cycles can only form via concurrent ingest of two pages that each claim the other as predecessor.

Class B triage uses these fields

When a new ingest enters a cluster where another source-summary already covers the same source-url with a different source-published, the triage detector classifies as Class B (temporal supersession candidate) and auto-writes the supersedes link with confidence >= 0.85. No callout, no interrupt — the wiki encodes the timeline cleanly.

Class D ("stale-canonical") complements this

When a newly ingested external source contains a fact that diverges from a canonical doc whose last-updated is older than the threshold, the daily wiki-canonical-drift job emits a wiki-canonical-drift-suggestion finding into the morning digest. The job NEVER edits canonical docs — it only suggests review.

Citation Rules

Adopt pin-llm-wiki's banner; combine with BCOS's link-don't-duplicate discipline.

Banner citation (top of every wiki page, immediately after H1 if any):

• For page-type: source-summary:

  All claims below are sourced from ../raw/<type>/<slug>.md unless otherwise noted.

• For all other page types:

  Canonical knowledge: brand-voice | target-audience — see those for the source of truth.

Per-paragraph inline citation only when a paragraph deviates from the banner sources.

Wikilink discipline:

• [[wiki-slug]] — references to other wiki pages (zone-internal graph)

• Markdown links [text](../brand-voice.md) — references to data points (cross-zone)

• Both are graph edges. Lint can reverse-build a citation graph from these.

Reference format rule (normative — enforced by the frontmatter validator and lint):

Slugs survive renames within the zone (refactor-safe via /wiki schema rename); paths are necessary cross-zone because data points live in docs/, not _wiki/. Mixing the two — e.g., putting .md on intra-zone references — is a lint error (reference-format-mismatch).

Hard rule: a wiki page that restates a fact owned by a data point is a lint error (duplication-vs-data-point). Replace the restatement with a wikilink/markdown-link.

Three Structural Shapes (source-summary only)

Pin-llm-wiki's clean topology for external sources, kept verbatim. Discriminators are mutually exclusive — lint enforces this.

Lint check frontmatter-shape:

• A page MUST be exactly one shape.

• Subs must point to an existing umbrella (parent-slug resolves).

• Umbrellas must not declare companion-urls: or raw-files:.

• Unified pages must not declare subpages: or parent-slug:.

• These rules apply only when page-type: source-summary.

Two Ingest Paths

There are two real paths through the wiki ingest pipeline, distinguished by where the content originates from. Both converge on the same page format, the same lint, the same log.md entry.

Path A — URL via queue (pin-llm-wiki native)

Trigger: "Add this URL to the wiki" / a URL appears in docs/_wiki/queue.md.

Flow: queue.md accumulates URLs → /wiki run (batch) or /wiki run <url> (single) → source-type detection (web/github/youtube) → fetch via gh, yt-dlp, or WebFetch → write verbatim capture to _wiki/raw/<type>/<slug>.md → ingest into _wiki/source-summary/<slug>.md → update overview.md and log.md → run refresh_wiki_index.py (regenerates index.md from frontmatter) → mark queue line <!-- ingested YYYY-MM-DD -->.

Use it for: external knowledge. The truth lives somewhere else and we want a citable, durable copy plus a wiki page.

Path B — Local content (covers _inbox/ promotions AND direct uploads)

Trigger: any local content the user wants in the wiki — pasted text, drag-dropped PDF/.docx, or a previously-captured file in docs/_inbox/.

Flow: /wiki create from <path-or-paste> → if path starts with docs/_inbox/, treat as inbox-promotion; if path is anywhere else (or content is pasted), treat as local-document; if file is binary (PDF, .docx, .zip), the binary lands at _wiki/raw/local/<slug>.<ext> AND a markdown extraction lands at _wiki/raw/local/<slug>.md; if pasted text or text file, it goes straight to _wiki/raw/local/<slug>.md → page authored at _wiki/pages/<slug>.md with full BCOS frontmatter including provenance: block recording the origin → standard finalization → if origin was _inbox/, the original _inbox/ file is deleted (its content now lives immutably in raw/local/).

Wiki ingest never writes to _collections/. _collections/ is the user's evidence zone — files-as-received with value independent of any wiki page. If the user explicitly wants the binary to also live in _collections/ as standalone evidence (e.g., a signed contract that's also the source for a wiki how-to), that's a separate /collections add operation. The wiki page can then builds-on: the _collections/ path. Implicit cross-zone writes blur the boundary; explicit ones don't.

Why one path, not two. The earlier draft split inbox-promotion and local-document into separate paths. They share 95% of the pipeline; the only real difference is the trigger. Collapsing to one path with provenance.kind: distinguishing the trigger is simpler. The triage signal coming into Path B can vary; the path itself doesn't.

Triage signals during context-ingest that suggest Path B (wiki-bound):

• Content describes a how, process, or steps rather than a fact

• Content is a post-incident narrative

• Content is a glossary-shaped definition list

• Content is a decision narrative (the why behind a choice)

Path Comparison

The HTML-Comment DSL on queue.md (Path A lifecycle)

Pin-llm-wiki's best mechanic: encode the entire lifecycle of a URL on one Markdown line, git-diffable.

## Pending
- [ ] https://blog.langchain.dev/state-of-ai-agents/         <!-- detail:deep -->
- [ ] https://github.com/langchain-ai/langchain              <!-- detail:standard branch:main clone -->
- [ ] https://youtube.com/watch?v=zjkBMFhNj_g                <!-- detail:standard -->
- [ ] https://example.com/docs                               <!-- companion:github.com/example/repo -->
- [ ] https://broken.example.com                             <!-- skip note:404 last checked 2026-04-15 -->

## Completed
- [x] https://anthropic.com/news/claude-3-7                  <!-- ingested 2026-04-22 -->
- [x] https://github.com/karpathy/llm-os                     <!-- ingested 2026-04-18 refresh -->
- [x] https://example.com/old-docs                           <!-- ingested 2026-03-30 -->

Tag vocabulary:

The queue is the state machine. Git history shows every transition without a separate registry file.

Provenance for Path B (local content)

Path B has no equivalent of the queue.md state machine because there's no queue — the content arrives via direct command. Instead, every Path B page carries a provenance: frontmatter block that records the origin in a git-diffable way:

provenance:
  kind: inbox-promotion            # url-fetch | inbox-promotion | local-document
  source-path: docs/_inbox/2026-04-15_acme-onboarding-notes.md
  captured-on: 2026-04-30
  promoted-by: gunti
  notes: "From discovery call with Acme on 2026-04-15"

Refresh logic for Path B content:

• kind: inbox-promotion: source has been deleted from _inbox/ after promotion. Refresh = N/A; the raw capture in _wiki/raw/local/ IS the durable copy. To "refresh," user provides new content via /wiki create from <path> and existing page is updated, with last-updated bumped and an entry in log.md.

• kind: local-document (file still on disk somewhere accessible): refresh re-reads the source file, hash-compares against the raw capture, and updates the wiki page if changed. If the file is no longer accessible, lint reports provenance-source-missing (INFO).

• kind: url-fetch: see "Refresh-Must-Rediscover" below; this kind only appears on _wiki/source-summary/ pages.

The provenance: block makes Path B's lifecycle just as auditable as Path A's queue.md.

Detail Level as Cost Knob

Three levels with explicit cost expectations — forces honest tradeoffs at queue time. Values come from _wiki/.schema.yml detail-levels: list (default [brief, standard, deep]).

Token guard at 200k: any single fetch that would cross 200k tokens halts and asks the user to confirm or downgrade detail level. Generalized to BCOS bulk operations (context-mine on long transcripts, context-audit on 100+ doc bases).

Operational Discipline — Sub-Agent Isolation

Big-body wiki work happens in .claude/agents/wiki-fetch/AGENT.md, not in the main thread. The principle: expensive reads should produce small returns. A 100k-token URL fetched into main saturates the context window before a single page lands; the same is true for deep PDFs, .docx exports, and full-tier source-summary refreshes.

Every Path-A and Path-B body fetch dispatches via the Task tool with the wiki-fetch sub-agent. The sub-agent does the read + extract + suggest, writes the raw body to disk under _wiki/raw/<type>/, and returns a structured ≤4000-token summary (title, h2-outline, key-sentences, suggested-page-type, suggested-cluster, raw-file-pointer, citation-banner-fields). Main never holds the full body.

Cumulative budget guard. Across N parallel sub-agent invocations in /wiki run batch mode, the calling skill consults _wiki_budget.decide_dispatch_strategy() before dispatching. The rule: cumulative projected return tokens (N × 4000) must stay ≤ 80% of main-context limit. Above that, sub-agents serialize — the next batch waits for the previous batch's results to free up budget.

Result validation. Every sub-agent return passes through _wiki_fetch_contract.validate_result() before the calling skill writes anything. Malformed outputs (missing required fields, oversized payload, type mismatches) are rejected with a clear error rather than producing a malformed wiki page.

The anti-inline-fetch rule. run.md Step 4 contains an explicit "do not fetch inline" directive. If a future edit brings inline-fetching back, test_wiki_fetch.py::WikiFetchSkillDispatchWiringTests fails — the wiring discipline is mechanically enforced.

The 15-test regression at test_wiki_fetch.py covers: agent contract shape, dispatch wiring across run/refresh/promote/create, cumulative budget math, and result-schema validation across canonical / error / malformed / oversized variants.

Refresh-Must-Rediscover (Path A only)

When a source-summary page is fully refreshed, the fetch protocol must re-run discovery from scratch — not assume the prior topology is still valid:

• A site that was single-product may have launched a second product → must re-detect multi-product split

• A site that was unified may have moved docs to a separate repo → must re-detect

• A landing page may now link to an llms.txt it didn't have before → must re-check

This is an anti-shortcut rule. Without it, refresh becomes "re-fetch the URLs we already had," and the wiki silently drifts from the source's actual structure. If topology changed, the prior wiki page is superseded (status: archived; new page(s) created).

Two-tier cadence (token-cost optimization). Full rediscover is expensive. To bound cost on large wikis, the wiki-source-refresh job runs in two tiers:

1. Quick-check tier (default cadence: stale_threshold_days / 4, e.g. 7 days when the default is 28). Issue an HTTP HEAD against the umbrella URL via _wiki_http.head_check. Compare the returned etag / last-modified against the values stored in the source-summary page's frontmatter via head_signals_unchanged(). If unchanged, bump last-fetched and skip. If validators differ or are missing, escalate to full rediscover immediately, regardless of where we are in the default tier. The stored content-hash (schema 1.1) is NOT consulted at this tier — a body hash is a full-tier validator, not a HEAD-response signal.

2. Default tier (stale_threshold_days, e.g. 28 days). Full refresh-must-rediscover. After wiki-fetch returns the body, the main thread calls _wiki_http.body_content_unchanged() against the stored content-hash — this catches the "ETag drifted but bytes identical" case. Runs whether or not the quick-check has fired in the interim.

The HEAD-check is a freshness signal, not a substitute for rediscover. Topology changes (new sub-products, llms.txt added, companion repo split) often happen without ETag/Last-Modified moving — so the default tier catches them. Weak ETags (W/"...") are normalized for comparison; servers that omit ETag fall back to Last-Modified; missing both validators escalate to full rediscover (the helper never auto-decides "unchanged" without evidence).

Layered validators (schema 1.1): etag + last-modified are HEAD-tier (cheap, no body); content-hash is full-tier (computed once a body is fetched). They live in the same source-summary frontmatter but answer different questions and run at different cadences.

A manual <!-- refresh --> queue tag always triggers the default tier (skips quick-check).

Maintenance Pipeline

Audit of pin-llm-wiki's 12 lint checks

New BCOS-specific checks

Auto-fixes (whitelisted in schedule-config.json)

• Run refresh_wiki_index.py to regenerate _wiki/index.md from frontmatter (replaces the obsolete "insert overview/log link" auto-fix — the regenerator always includes those links)

• Rewrite citation paths when data points are renamed (when redirect map provided)

• Mark queue lines <!-- ingested YYYY-MM-DD --> after successful Path A run

• Move expired post-mortems to status: archived when auto-archive-after-days (per-page-type schema setting) elapses

Scheduled Jobs

Four jobs added to .claude/quality/schedule-config.json (and template). All wire into schedule-dispatcher.

{
  "jobs": {
    "wiki-stale-propagation": {
      "enabled": true,
      "schedule": "daily",
      "_about": "When a data point's last-updated advances, flag every wiki page whose builds-on includes that data point as needs-review (until human resets last-reviewed)."
    },
    "wiki-source-refresh": {
      "enabled": true,
      "schedule": "weekly",
      "_about": "Two-tier refresh on source-summary pages. (1) Quick-check tier (default cadence: stale_threshold_days/4): HEAD-check upstream; if hash unchanged, bump last-fetched and skip. (2) Default tier (stale_threshold_days): full refresh-must-rediscover regardless of HEAD result — re-runs source-type detection, product-discovery, companion-repo discovery from scratch. HEAD-check is a freshness signal, not a substitute for rediscover."
    },
    "wiki-graveyard": {
      "enabled": true,
      "schedule": "monthly",
      "_about": "Pages with last-reviewed > 12 months AND no recent inbound traffic → AskUserQuestion about archive."
    },
    "wiki-coverage-audit": {
      "enabled": true,
      "schedule": "quarterly",
      "_about": "Daydream-adjacent. Two outputs: (1) which active data points lack a wiki explainer? (2) Vocabulary suggestions: any frequently-used `<!-- note: -->` keywords or recurring page tags that should be promoted to a registered page-type in .schema.yml?"
    }
  },
  "auto_fix": {
    "whitelist": [
      "wiki-index-refresh",
      "wiki-mark-queue-ingested",
      "wiki-archive-expired-post-mortem",
      "wiki-rewrite-citations-on-rename"
    ]
  }
}

Cadence rationale:

• Daily for wiki-stale-propagation — propagation events are the highest-value signal; cheap to compute (read frontmatter only, no fetches).

• Weekly for wiki-source-refresh — fetch cost is non-trivial; daily is overkill, monthly is too slow. HEAD-check skips unchanged sources to keep the cost bounded.

• Monthly for wiki-graveyard — slow-moving by nature; running daily would generate noise.

• Quarterly for wiki-coverage-audit — daydream-territory, expensive synthesis; the vocabulary-suggestion sub-check needs accumulated usage data.

All cadences and thresholds (graveyard age, stale-day count, coverage-audit lookback) are tunable per repo via _wiki/.schema.yml and _wiki/.config.yml. The schedule-tune skill is the user-facing surface for changing them.

/wiki Command Surface

Two ingest entry points + management + schema governance.

# Ingest
/wiki run                          # Batch process all "Pending" entries in queue.md (Path A)
/wiki run <url>                    # Single-URL ingest, ad-hoc (Path A)
/wiki create from <path-or-paste>  # Path B — auto-detects if origin is _inbox/ and adjusts provenance

# Management
/wiki review <page>                # Bump last-reviewed; optionally re-pull builds-on data points to verify
/wiki archive <page>               # Soft-delete to _wiki/.archive/ with dangling-reference scan
/wiki refresh <page>               # Re-fetch a source-summary page (refresh-must-rediscover applies)
/wiki lint                         # Run wiki-specific checks on demand (also runs via schedule)

# Schema governance (see "Schema and Governance" section)
/wiki schema list                  # Show current vocabulary
/wiki schema add page-type <name>  # Add new page-type via AskUserQuestion-driven prompts
/wiki schema rename page-type <from> <to>     # Migrate frontmatter on all pages; record in log.md
/wiki schema retire page-type <name>          # Mark deprecated; lint existing pages with migration hint
/wiki schema validate              # Walk every page; verify against current schema
/wiki schema migrate <from-version> <to-version>  # Versioned migration with dry-run diff

All decision points use AskUserQuestion per BCOS's decision-point UX rule.

Inbox Reconciliation

Two inboxes is the wrong answer. The reconciliation:

• docs/_inbox/ stays the unified BCOS triage zone for anything unclassified

• docs/_wiki/queue.md is an intentional URL queue (Path A only)

• context-ingest gets Path 7 in its triage tree: when classifying an _inbox/ item as wiki-bound, it hands off to /wiki create from <_inbox/path> (Path B with kind: inbox-promotion provenance), or extracts URLs from the capture and routes them into _wiki/queue.md

This preserves the single-entry-point principle while letting the wiki own its specialized URL pipeline.

What We Drop from pin-llm-wiki

• AGENTS.md and the cross-tool adapter pattern. BCOS has one CLAUDE.md.

• The tags: [] and related: [] fields. Replaced by directional builds-on, references, provides and categorical cluster.

• The product: field. Replaced by cluster: (BCOS native).

• Pin-llm-wiki's interview-style init flow. Replaced by AskUserQuestion-driven config.

• Versionless frontmatter. Replaced by BCOS's version: + last-updated: discipline.

• Per-page-type sub-folder structure. Page-type lives in frontmatter; folder layout is flat (pages/).

What We Adopt as-is from pin-llm-wiki

• HTML-comment DSL on queue.md (Path A lifecycle) — git-diffable state machine

• Banner citation pattern — extended to BCOS's builds-on model

• Three structural shapes (standalone / unified / umbrella+sub) for source-summary

• Detail-level cost knob with token + dollar estimates and 200k token guard

• Refresh-must-rediscover anti-shortcut rule

• Source-type fetch protocols — gh, yt-dlp, web crawler with companion + llms.txt discovery

• Soft-delete with dangling-reference scan

• Append-only wiki/log.md

Boundary with lessons.json

BCOS's lessons.json (maintained by lessons-consolidate) and the wiki's post-mortem and decision-log page-types overlap in domain (institutional knowledge). The boundary:

The flow: a post-mortem (wiki page) is a narrative source. Reading the post-mortem produces zero or more atomic lessons that lessons-consolidate extracts into lessons.json. The lesson cites the post-mortem in its provenance; the post-mortem links back via wikilink/reference.

daydream and wiki-coverage-audit cross-reference both: a wiki post-mortem with no extracted lessons is a candidate for lessons-consolidate; a recurring lesson that points to no post-mortem is a candidate for new wiki-page authoring.

Schema and Governance

Vocabularies in BCOS should grow with the business, not be chiseled in stone. The wiki zone exposes its vocabulary via a single registry file, _wiki/.schema.yml, that drives validation, lint, and tooling. Every type, category, and constraint lives in one place. Schema-versioned. Migrating between versions is a defined operation.

_wiki/.schema.yml registry

schema-version: 1
last-updated: 2026-04-30
last-migrated: null                  # set by `/wiki schema migrate`

page-types:
  how-to:
    description: "Procedural — steps for accomplishing a task"
    required-fields: [builds-on]
    review-cadence-days: 180
  glossary:
    description: "Term definitions for newcomers"
    required-fields: []
    review-cadence-days: 365
  source-summary:
    description: "External source captured via Path A (URL via queue)"
    required-fields: [source-url, last-fetched, detail-level]
    structural-shapes: [standalone, unified, umbrella-sub]
    review-cadence-days: 90
  # add page-types as you discover them; suggested next:
  # runbook, post-mortem, decision-log, faq, playbook, tutorial, meeting-notes

statuses: [active, under-review, archived]
detail-levels: [brief, standard, deep]
provenance-kinds: [url-fetch, inbox-promotion, local-document]

# Cluster vocabulary inherits from BCOS (single source of truth)
cluster-source: docs/document-index.md
allow-cluster-not-in-index: false    # strict; or true to permit wiki-only clusters

# Lint: forbid building on non-canonical sources
forbid-builds-on-paths:
  - _planned/
  - _inbox/
  - _archive/

# What `bcos init` should scaffold in a new repo
bootstrap-defaults:
  page-types: [how-to, glossary, source-summary]
  note: "Start small. Add page-types as you discover real categories you keep wanting."

Governance operations

Migration safety net (dry-run diff)

$ /wiki schema rename page-type how-to procedure

Migration plan:
  Update frontmatter:   12 pages       (page-type: how-to → page-type: procedure)
  Update schema:        .schema.yml    (page-types.how-to → page-types.procedure)
  Update lint config:   1 reference
  Update README:        2 mentions
  Log entry:            _wiki/log.md   (migration: schema 1 → 2)

Proceed? [Yes / Show diff / No]    ← AskUserQuestion

The log entry records the inverse migration so it's reversible.

Bootstrap behavior on bcos init in a new repo

• .schema.yml is scaffolded with sensible defaults — initially just how-to, glossary, source-summary (the three universal categories)

• Header comment: "Review this. Add page-types as you discover real categories you keep wanting."

• The wiki-coverage-audit quarterly job gains a sub-check: vocabulary suggestion — "You keep tagging pages with <!-- note:meeting-notes -->. Add meeting-notes as a registered page-type?" Real-usage drives schema growth, not upfront speculation.

Same pattern, other registries

For consistency across BCOS, the schema-driven approach applies beyond the wiki:

• _collections/.schema.yml — collection sub-types, manifest-schema column requirements, naming conventions, archive policy. See collections-zone.md.

• .claude/quality/lint-checks.json — registry of lint check IDs, severity, scope. New checks register here without code changes to the dispatcher.

• .claude/quality/schedule-config.json — already exists; we add a required _about field so the catalog is self-documenting.

The principle: adding a category should be one edit, not five.

Retrieval — Today and Tomorrow

The wiki zone is now a read store, not just a write store. Mechanical cross-zone search shipped in P2 of the wiki-missing-layers plan (2026-05-04). The graph/embeddings layers below are still future-looking and ship behind P6 (conditional on real precision needs).

What's shipped — mechanical cross-zone search (P2)

/context search "<query>" and the wiki-scoped sugar /wiki search "<query>" both run mechanical BM25-style ranking over .claude/quality/context-index.json — the canonical corpus that context_index.py builds from docs/. The engine lives at context_search.py; the skill surface lives at context-routing/SKILL.md and bcos-wiki/search.md.

Three rules are load-bearing:

1. Mechanical-first (D-10 strict). The default path is pure Python, deterministic, ms-latency. The engine never decides on its own to call the LLM. The only LLM-touching path is --semantic, which the user opts into explicitly. There is no auto-trigger on 0-hit, no fallback, no hidden cost.

2. Stable citation-ids. Every hit returns citation-id: zone:slug (e.g., wiki:linkedin-tone, active:pricing, planned:wiki-missing-layers). Path-derived; survives whitespace edits; cited back by other agents and by P5 bundle output.

3. Empty zones are graceful. When _wiki/ is not initialized in a fresh repo, search returns [] for the wiki zone and surfaces it under zones-skipped-not-present. Not a hard failure.

The 15-test regression at test_context_search.py covers the shared schema, ranking, zone filter, top-K cap, citation stability, the D-10 escalation rules, the empty-corpus path, and CLI smoke tests.

Graph readiness

The frontmatter is already a typed-edge graph specification:

A graph generator can walk _wiki/ + docs/*.md + _collections/ and emit nodes (with frontmatter as attributes) + edges (with provenance). The Context Atlas (.claude/scripts/bcos-dashboard/atlas_collectors.py) already does this for data points; extending to wiki + collections is mechanical.

Chunking readiness

• Consistent H2 structure, no horizontal rules between sections → each ## H2 is a chunk candidate

• Banner citation at top → every chunk inherits provenance via the page-level banner

• Frontmatter is page-level metadata → each chunk carries cluster, page-type, builds-on, last-updated for filtering

• Stable kebab-case slugs → chunk IDs don't change when content edits happen

• No H1 inside body (filename = title) → no ambiguity about which heading is the page title

Embedding-update logic

Same trigger logic as wiki-stale-propagation:

• A data point's last-updated advances → re-embed every wiki page that builds-on it (chunk-granular)

• A raw file is overwritten → re-embed every wiki page whose raw-files: includes it

• Pages older than last-reviewed > 12 months flagged for re-embed regardless

The lint pipeline that flags review candidates IS the re-embedding queue.

Provenance-grounded retrieval

A retrieval that returns a wiki chunk can cite, in order: (1) the chunk's section heading; (2) the page-level banner (builds-on: data points); (3) the page-level raw-files: (verbatim source); (4) the page-level last-reviewed (freshness). No chunk is orphaned from its provenance chain.

What's deliberately NOT pre-built

We don't add explicit chunk-boundary markers, pre-computed embeddings, or a graph-store schema now. The data is already shaped so when those layers land, they slot in without re-formatting the existing wiki.

Boundary with Strict Ownership

Wiki pages still declare domain + exclusively-owns (within the wiki's universe of explanation). A how-to owns "LinkedIn-specific tone adjustments" exclusively — no other how-to should claim that. But a wiki page's exclusively-owns is about explanation, not canonical truth. Two pages can explain the same data point from different angles (e.g., linkedin-tone.md and customer-success-emails.md both build on brand-voice.md) — they own different applications of the canonical truth.

The duplication-vs-data-point lint catches the failure mode where a wiki page accidentally tries to own canonical content (which already lives in a data point).

Adoption Plan

The pin-llm-wiki repo at https://github.com/ndjordjevic/pin-llm-wiki provides Path A end-to-end. Adoption steps:

1. Skill scaffolding (code-free — pin-llm-wiki is pure markdown instructions):

• Copy skills/pin-llm-wiki/ → .claude/skills/wiki-manage/

• Patch templates: inbox.md → queue.md; wiki/sources/ → _wiki/source-summary/; raw/ → _wiki/raw/; raw/<type>/README.md → raw/<type>/_manifest.md (cross-zone consistency with _collections/)

• Replace per-page-type folder writes with flat _wiki/pages/<slug>.md

• Rename config: .pin-llm-wiki.yml → _wiki/.config.yml

• Add _wiki/.schema.yml scaffold with bootstrap-defaults

• Add Path B as new sub-flow (referencing Path A's ingest pipeline as the shared transformation, with provenance: frontmatter populated from origin detection)

• Drop AGENTS.md template; merge instructions into wiki-manage/SKILL.md

• Replace prose-style init with AskUserQuestion-driven config

2. Frontmatter migration:

• Replace pin-llm-wiki's loose schema with the BCOS-aligned schema above

• Extend the existing PostToolUse hook to read _wiki/.schema.yml and enforce schema-defined rules

• Add provenance: block validation for Path B pages

3. Lint migration:

• Implement the keep/replace/drop matrix

• Add the seven new BCOS-specific checks (including forbidden-builds-on-target, provenance-source-missing, schema-violation)

• Wire into .claude/quality/schedule-config.json

4. context-ingest extension:

• Add Path 7 to the triage tree (content-routing.md): wiki promotion

• Wire _inbox/ items classified as wiki-bound to /wiki create from <_inbox/path> (Path B with inbox-promotion provenance)

5. Schema governance:

• Implement /wiki schema command surface

• Wire schema-driven validation into the PostToolUse hook

• Add quarterly vocabulary suggestion sub-check to wiki-coverage-audit

6. Documentation updates:

• ✅ Add _wiki/ row to CLAUDE.md Folder Trust Levels — done

• ✅ Add _wiki/ row to folder-conventions.md — done

• Add Path 7 (wiki promotion) to content-routing.md triage tree

• Update metadata-system.md with page-type enum + wiki-specific frontmatter fields (or, better: point to _wiki/.schema.yml as the source of truth)

7. Bootstrap defaults for new repos:

• bcos init scaffolds _wiki/.schema.yml with [how-to, glossary, source-summary] only

• Header comment instructs the user to grow the vocabulary as real categories emerge

• Quarterly wiki-coverage-audit surfaces vocabulary suggestions from real usage

8. License / attribution:

• pin-llm-wiki is Apache 2.0

• Attribution preserved in adapted skill's SKILL.md header

Out of scope for v1:

• Atlas / galaxy node-type extension to render wiki pages distinctly

• Pre-computed graph export to a vector/graph store

• Cross-tool adapter (AGENTS.md) — revisit only if BCOS-wide need emerges

See Also

• collections-zone.md — Sibling zone for verbatim evidence. Wiki ingest (Path B) does NOT write to _collections/ — that's reserved for explicit user /collections add actions. _wiki/raw/<type>/ uses the same _manifest.md pattern as _collections/<type>/_manifest.md (cross-zone consistency); both zones use .schema.yml registries.

• content-routing.md — How content flows into the data-point zone (docs/*.md)

• metadata-system.md — Canonical frontmatter field reference (the wiki adds wiki to the type enum and a wiki-only fields section)

• system-design.md — Why zones exist; the philosophical underpinning

• ../guides/folder-conventions.md — Folder-level rules across all zones

• ../methodology/document-standards.md — Frontmatter + ownership-spec authoritative reference

• ../templates/_wiki.schema.yml.tmpl — Wiki vocabulary registry template

• pin-llm-wiki upstream — https://github.com/ndjordjevic/pin-llm-wiki

Status: v3 design — schema-governance layer added; canonical (parallel draft-a retired 2026-05-01); ready for skill scaffolding. Created: 2026-04-30 Last Updated: 2026-05-13 Version: 3.2.0

v3.2.0 — 2026-05-13: Reframed the "Why Wiki Zone Exists" opening to explicitly anchor the wiki as BCOS's universal long-form / cross-cutting content destination per plugin-storage-contract.md Rule 2. Three content categories surfaced (operational truth, internal reference, plugin cross-cutting + external sources) instead of a single "operational/explanatory" framing. Added explicit "what does NOT belong" callouts (canonical data points, plugin-private structured records, legal-weight evidence). Diagnosed via a downstream session where an agent inferred "wiki = URLs only" from the prior framing and dumped a runbook into docs/operations/. See .claude/quality/sessions/20260513_103523_wiki-reframe-autoinit/ for the full plan.