data-olympus

Adoption guide: bring your own KB

This guide explains how to create, validate, and serve a data-olympus knowledge bundle from your own content. It covers the full path from a fresh directory to a running MCP server your agents can query.

1. Install

From the repo root:

uv venv && uv pip install -e '.[dev]'

Or skip the venv step and use uv run to drive the CLI directly. Every command in this guide works with either approach; the examples below use uv run.

2. Author a bundle

A bundle is a directory of markdown files with YAML frontmatter. The layout mirrors example-bundle/ in this repo.

Scaffolding a fresh bundle

Instead of copying example-bundle/ by hand, scaffold a new bundle with:

uv run data-olympus init <dir>

This creates the tier directories (--tiers, default universal,tech-stacks, projects,decisions,workflows,tooling), a root index.md carrying the spec_version/okf_version frontmatter, a template.md authoring scaffold, and one example document per supported type (standard, decision, workflow, project, memory, reference), including a real superseded/superseded_by/supersedes pair and applies_when trigger metadata, so data-olympus lint, data-olympus index, and kb search all have something to demo against immediately. The command refuses to write into a non-empty directory. Scaffold a subset of tiers with, for example, --tiers decisions,workflows.

Required frontmatter fields

Every concept file (every .md file that is not index.md, log.md, or template.md) must include:

• id: a stable, globally unique identifier for the concept (e.g. STD-U-001, ADR-007, WF-onboarding).

• type: one of standard, decision, workflow, project, memory, reference.

• status: one of draft, active, deprecated, superseded, proposed, accepted, rejected.

• tier: one of T1, T2, T3, T4, meta.

Recommended frontmatter fields

Including these fields improves search quality and the generated index:

• title: human-readable name.

• description: one-sentence summary.

• tags: list of keywords.

• generated: who produced the content and when it last meaningfully changed, as { by, at } (OKF v0.2). Use by: human:<your-id> for content you write by hand, and an ISO 8601 datetime with an offset for at, for example "2026-06-24T00:00:00Z". A legacy timestamp date is still accepted, so existing documents need no change.

• applies_when: list of trigger phrases describing the coding intents this document governs (see below).

See SPEC.md for the full schema, cross-linking conventions, and tier definitions.

Authoring applies_when

applies_when is a list of short verb phrases in an agent's own vocabulary, not the document's internal terminology: "reading a .env file", not "security". It matters because it is the highest-weight field in the reference search index, tied with title, ahead of description and body text: a standard titled "Secrets Handling" is retrieved when an agent is mid-task on "logging a request body" only because that phrase is listed in applies_when, not because the title or body happen to share vocabulary with the query.

applies_when:
  - "reading a .env file"
  - "committing a credential or API key"
  - "logging a request or response body"

Add it to every standard and decision document you expect an agent to hit mid-task, phrased as that moment, not as a topic label.

Reserved files

Three filenames have special meaning and must not carry concept frontmatter:

• index.md: generated navigation index (root index carries spec_version and okf_version frontmatter; see below).

• log.md: human-maintained changelog for the bundle, date-grouped, newest entry first.

• template.md: an authoring-scaffold file for a directory (not itself a governed concept).

Cross-links

Reference other concepts with bundle-relative paths (absolute from the bundle root, e.g. /universal/foundation/STD-U-001.md). This applies to markdown links in the document body.

The supersedes, superseded_by, contradicts, and derived_from frontmatter fields are different: they hold concept IDs, not paths (e.g. supersedes: ADR-002, not supersedes: /decisions/ADR-002.md). Shapes per field: supersedes, contradicts and derived_from accept a single ID or a list of IDs (both normalize to a list); superseded_by is a single scalar ID only, never a list. kb lint cross-checks these fields across the bundle (malformed shapes, self-references, supersession cycles, dangling/asymmetric targets, path-shaped values, and in-force contradicts pairs are all reported; see SPEC.md section 4.2 for the exact error/warning list). Since format 0.4, a supersedes or superseded_by target that does not resolve to a document is an error, so create or import the document being superseded before the one that supersedes it. Since format 0.5, derived_from names the documents a concept's guidance was drawn from; it follows the same resolution rule, and when a source leaves force kb lint warns and kb_get names the in-force dependents for a person to review, without filtering anything. The reference implementation's indexer also extracts them into an edges table consumed by later retrieval features.

Directory structure

There is no required directory layout beyond the reserved file names. The example-bundle/ convention (tiers as top-level directories: universal/, tech-stacks/, decisions/, workflows/, projects/) is a proven pattern and is described in SPEC.md, but you can adapt it to your context.

3. Migrating an existing rule corpus

If you already have agent rules in flat files (CLAUDE.md, AGENTS.md, GEMINI.md, .cursorrules), adr-tools ADR directories, or an OKF-produced bundle, the import command migrates them into governed drafts instead of requiring you to hand-write frontmatter for each concept:

uv run data-olympus import <source> --kind <kind> --tier <tier> [options]

• <source>: a flat rule file (for the flat kinds) or a directory (for adr / okf).

• --kind: one of claude-md, agents-md, gemini-md, cursorrules, adr, okf.

• --tier: the governance tier stamped on every draft (T1..T4, meta, or a bare digit like 3).

• --out <dir>: output bundle directory (default: <source>/imported-<kind> or, for a flat file, a sibling imported-<kind>/ directory).

• --category <cat>: optional category stamped on every draft.

• --id-prefix <prefix>: id prefix for generated ids (default: per-kind, e.g. CLAUDE, AGENTS, OKF; ADR imports always use ADR-NNNN).

• --force: overwrite an output directory that was already used as an import target (see re-run behavior below).

• --json: emit the import report as JSON instead of the human-readable form.

What the importer does

• Flat rule files are split into candidate concepts: markdown headings first (each heading starts a section), and for heading-less files, blank-line separated bullet clusters. Each concept becomes a draft with a generated id, type: standard, status: draft, and a title/description/tags derived from the source. The original text is preserved verbatim as the body; the importer never rewrites your words. Sections too short to be a real concept are skipped and listed in the report.

• ADR directories (doc/adr/NNNN-title.md) map the number and title to an ADR-NNNN id and title, parse the ## Status section into supersedes / superseded_by references (type: decision), and preserve the ADR body. The parsed adr-tools status (e.g. Accepted, Superseded by ...) is preserved in a non-activating source_status field for the reviewer; the concept itself is imported as status: draft so it does not become in-force before review.

• OKF bundles are normalized into the governance profile: alias field names are renamed to the canonical schema keys, missing required fields are filled with draft-safe defaults, and every inference is reported. When a document uses two aliases for one field, the canonical key wins, then identifier over uid, kind over doctype and updated over date; the dropped value is reported.

The OKF path is covered by executable interoperability checks against official Google OKF v0.2 commit ad30107c31c06aec8a7d5636e0d1058118604e6f in GoogleCloudPlatform/open-knowledge-format. The pinned official Bitcoin sample imports as nine lint-clean governed drafts, all nine are indexed, and search plus retrieval complete successfully. The importer keeps a source's OKF v0.2 generated record and legacy timestamp as written, and stamps generated naming the data-olympus tool only when a document records neither. The reverse check runs Google's reference visualization consumer over every concept in example-bundle. To repeat both checks from a source checkout:

uv run python scripts/okf_conformance.py verify-pin
uv run python scripts/okf_conformance.py consume-data-olympus \
  --upstream-root to-delete/okf-reference
uv run python scripts/okf_conformance.py consume-upstream

The first and third commands are offline. The second requires the official repository checked out at the exact SHA recorded in tests/okf/reference.json. The evidence is scoped to that revision and those fixtures.

Safety and next steps

• Everything lands as status: draft. Nothing auto-activates, and the importer never commits to git: it only writes files into the output directory. Imported ADRs keep their original status in source_status for the reviewer.

• Duplicate ids are refused. If two imported concepts (or an imported concept and one already in the output dir) would share an id, the import aborts rather than write a structurally unsafe bundle.

• The command runs the existing lint machinery over the output; the happy path produces lint-clean drafts. Any finding is included in the report. An ADR that supersedes one that is not part of the imported set produces an unresolved target error and the command exits 1, but the drafts, report and marker are still written. Pass --resolve-root <your-bundle-dir> so targets already in your bundle resolve, or --unresolved-targets warn to report them as warnings. An invalid --resolve-root exits 1 before anything is written.

• Re-run behavior: the importer refuses to write into an output directory it already used as an import target (or any directory that already holds governed files), so a second run cannot silently duplicate or clobber. Pass --force to re-run: it deletes exactly the files the previous import wrote (ids restart deterministically) and leaves any file you added by hand untouched. Or choose a fresh --out directory.

• Converging with an existing KB: the report's next-steps point at the dedup pass (the kb_cleanup_plan MCP tool / POST /api/v1/onboarding/cleanup-plan), which classifies each draft against your committed KB as imported_duplicate / partial_overlap / unique and proposes thin-pointer replacements for exact duplicates. Run it before activating the drafts so imports converge with the KB instead of duplicating it.

Example: import a CLAUDE.md into a new draft bundle and inspect the result:

uv run data-olympus import ./CLAUDE.md --kind claude-md --tier T3 --out ./imported
uv run data-olympus lint ./imported

4. Validate

uv run data-olympus lint <your-bundle-dir>

Expected output when the bundle is conformant (the trailing count is how many concept files were linted; N is the number of concept docs in your bundle):

0 errors across 0 files (N linted)

The linter checks required frontmatter fields, controlled-vocabulary values, and reserved-file constraints. Fix any reported errors before proceeding. lint exits non-zero when it finds no concept files to lint (for example, an empty or mis-pathed bundle), so a clean pass always means real files were checked.

A supersedes or superseded_by target that does not resolve is an error (format 0.4). When linting a subdirectory of a larger bundle, resolve targets against the whole bundle:

uv run data-olympus lint <your-bundle-dir>/projects/acme --resolve-root <your-bundle-dir>

Upgrading a bundle that has unresolved targets: list them without failing, fix or remove each one, then drop the flag.

uv run data-olympus lint <your-bundle-dir> --unresolved-targets warn

5. Generate navigation and graph

uv run data-olympus index <your-bundle-dir>

This regenerates index.md files at each directory level. The root index.md carries spec_version and okf_version frontmatter; preserve those lines if you edit the root index by hand.

uv run data-olympus visualize <your-bundle-dir> -o <your-bundle-dir>/viz.html --name "My KB"

This generates an interactive HTML graph of all concepts and their cross-links. Open the file in a browser to explore the knowledge graph.

6. Serve to agents

Local (development)

./scripts/run-local.sh

The script copies example-bundle/ to /tmp/data-olympus-demo-kb, git-inits the copy, and starts the MCP server at http://localhost:8080. It is a demo helper only: it always wipes and recreates its target directory, so it does not read KB_MAIN_PATH and must not be pointed at your own bundle.

To serve your own bundle instead, invoke the MCP server directly:

KB_MAIN_PATH=/path/to/your-bundle \
  KB_INDEX_PATH=/tmp/your-kb.db \
  KB_REMOTE_URL="" \
  uv run data-olympus-mcp

Your bundle must be a git repository (run git init inside it first).

Docker

docker compose -f deploy/docker/compose.yaml up --build

Bind-mount your bundle to serve real content:

docker compose -f deploy/docker/compose.yaml run \
  -e KB_MAIN_PATH=/my-kb \
  -v /path/to/your-bundle:/my-kb:ro \
  data-olympus-mcp

Kubernetes

Apply the kustomize manifests in deploy/k8s/:

kubectl apply -k deploy/k8s/
sops exec-file deploy/k8s/secret.sops.yaml 'kubectl apply -f {}'

See docs/serving.md for the full serving model, including the single-replica / single-writer constraint and how to run read-only replicas for higher read throughput.

Write and push

To enable the write pipeline (agent-proposed edits committed and pushed to a remote), set two environment variables before starting the server:

• KB_REMOTE_URL: the git remote URL the server should push to. With the published container image this must be an SSH URL (for example [e-mail address on the upstream page](https://github.com/knaisoma/data-olympus/blob/6d46033f7a3fc08acfff3f148e143077a9984eef/docs/adoption.md):org/kb.git): from 0.11.3 the image's system gitconfig refuses git over http, https, ftp and ftps, so an HTTPS remote fails with "transport 'https' not allowed". A server installed from PyPI uses the host's git configuration and is not affected.

• The deploy key or credential must be available to the server process (SSH key on disk, or a credential helper).

If your repository's trunk is not called main, also set KB_GIT_BRANCH (for example master). It names the branch that is fetched, fast-forwarded, rebased onto and pushed to; see the core configuration reference in serving.md.

Without KB_REMOTE_URL, the server runs in read-only mode: the pull loop runs but does not push, and the write pipeline is disabled. REST write routes (/api/v1/propose/*, /resolve/{id}, /onboarding/bootstrap) return 503 {"error": "write_pipeline_disabled"} and the MCP write tools return {"status": "write_pipeline_disabled"} rather than crashing. Read routes work normally.

7. Wire an agent

Register the MCP endpoint with your agent. The server exposes MCP over streamable HTTP at http://<host>:8080/mcp (or the port you configured).

Easiest: the setup wizard

data-olympus setup

The wizard probes the endpoint, detects which of Claude Code, Codex, Gemini, and OpenCode are installed, writes each agent's MCP registration (with a timestamped backup of any file it edits, and idempotent re-runs), optionally installs the enforcement hooks, and prints a doctor summary.

Flags:

• --endpoint <url>: server endpoint to wire (default $KB_ENDPOINT or http://localhost:8080).

• --check: read-only doctor summary (also the update-check path); changes nothing.

• -y / --yes: non-interactive; accept defaults (register detected agents, skip hook prompts).

• --no-version-check: skip the PyPI/GitHub latest-version lookup (fully offline).

When you point agents at a server, use GET /readyz (not /api/v1/health) as the readiness probe: it returns 200 once the process is up and the index is loaded, independent of data staleness. See docs/serving.md for the full health/readiness/liveness split.

Manual per-agent registration

If you prefer to wire agents by hand, use the CURRENT surface for each. These are the same commands/files the wizard uses. Replace http://localhost:8080 with your endpoint.

Claude Code (use the claude mcp CLI; it writes ~/.claude.json, not settings.json):

claude mcp add --transport http data-olympus http://localhost:8080/mcp

Codex (use the codex mcp CLI; it writes [mcp_servers.*] in ~/.codex/config.toml):

codex mcp add data-olympus --url http://localhost:8080/mcp

Gemini / Antigravity (merge into ~/.gemini/settings.json; note the explicit "type": "http"):

{
  "mcpServers": {
    "data-olympus": {
      "url": "http://localhost:8080/mcp",
      "type": "http"
    }
  }
}

OpenCode (no native remote-HTTP transport; wrap via mcp-remote under the top-level mcp key in ~/.config/opencode/opencode.json):

{
  "mcp": {
    "data-olympus": {
      "type": "local",
      "command": ["npx", "-y", "mcp-remote", "http://localhost:8080/mcp", "--allow-http"],
      "enabled": true
    }
  }
}

Restart each agent's session after registering. Verify by asking it to call kb_search.

The kb CLI in bin/kb is a thin client for the same HTTP API. Set KB_ENDPOINT and use it from the shell or from agent-invoked scripts:

export KB_ENDPOINT=http://localhost:8080
./bin/kb search "module structure"
./bin/kb get /tech-stacks/backend-nestjs/STD-BN-001-module-structure.md
./bin/kb outline

The CLI supports -o plain for machine-readable output and -o json for structured output suitable for piping into other tools.