data-olympus

The data-olympus OKF profile

This document is the field-by-field reference for data-olympus as an Open Knowledge Format (OKF) v0.2 profile: which frontmatter fields are stable and lint-guarded, which are runtime-only serving-envelope fields that never touch frontmatter, which are experimental candidates tracked against open OKF discussions, and how each maps to OKF baseline compatibility. It exists so an adjacent OKF implementer (or a data-olympus operator) can compare field names and semantics without reading the codebase, and so nobody (including us) overclaims beyond the exact interoperability evidence recorded for issue #82.

This is a profile document, not a new specification. SPEC.md remains the authoritative source; where this document restates a rule, SPEC.md wins on conflict.

1. OKF baseline inherited conventions

data-olympus inherits, by construction, OKF's directory structure, frontmatter basics, reserved filenames, link model, and unknown-key tolerance (SPEC.md sections 3-5). Specifically:

• Documents are UTF-8 markdown with a YAML frontmatter block delimited by a bare --- line (SPEC.md section 4.1).

• The reserved filenames index.md, log.md, and template.md are exempt from concept-schema validation in every directory (SPEC.md section 3; enforced as format.validate.RESERVED and checked first thing in validate_document).

• Links are untyped directed edges using bundle-relative or relative markdown hyperlink syntax; consumers MUST tolerate broken links (SPEC.md section 5).

• Consumers MUST tolerate unknown type values, unknown extra frontmatter keys, missing optional fields, and broken links (SPEC.md sections 4.1 and 9). This is the tolerance rule every extension in this document depends on: it is what lets an OKF consumer silently ignore supersedes, validity, and every other data-olympus addition.

On the conformance wording, deliberately careful: CI pins official Google OKF v0.2 commit ad30107c31c06aec8a7d5636e0d1058118604e6f in the dedicated GoogleCloudPlatform/open-knowledge-format repository and verifies the fixture and Apache 2.0 license checksums from tests/okf/reference.json. The okf/ copy in GoogleCloudPlatform/knowledge-catalog, pinned before, is a frozen snapshot and is no longer accepted as a pin. Google's reference visualization consumer reads every concept in example-bundle. In the reverse direction, the data-olympus OKF importer converts the pinned official Bitcoin sample into governed drafts, then the linter, index, search, and retrieval paths consume all nine concepts. These are executable interoperability checks for the named revision and fixtures. They are not a formal certification, a round-trip fidelity claim, or evidence for every possible OKF bundle. A scheduled workflow reports upstream drift of the vendored FIXTURE by maintaining one GitHub issue and never updates the pin automatically; a change confined to the upstream reference consumer, SPEC, or license is not detected by that workflow.

2. Stable governance extensions

These fields are shipped, schema-checked by data-olympus lint (src/data_olympus/format/lint.py via validate_document and the cross-file lifecycle pass), and part of the current SPEC.md (version 0.5). "Stable" here means the field name and semantics are not expected to change shape; it does not mean the field is required in every bundle unless stated, and two fields (applies_when, owner) are stable documented conventions with no lint coverage; the table's lint column is authoritative per field.

status, and the note on issue #114

status is listed as a required field in SPEC.md section 4.2 today, and format.validate.REQUIRED = ("id", "type", "status", "tier") has always enforced it as a lint error when absent; this is not new in v0.4.0.

What v0.4.0 adds (per issue #114, merged in PR #128) is write-path enforcement plus a staged migration story, not a change of required-field semantics:

• New documents: kb_propose_edit now rejects a postimage that creates a NEW document without status (rejected_invalid_document, reason missing_status, write_gate.validate_postimage step 2b). Previously the write path was more permissive than kb lint: the enum check only fired when status was present and invalid, so a brand-new status-less document slipped through.

• Existing documents: editing an EXISTING status-less document still requires no status (write_gate._target_path_exists classifies the write), so a legacy corpus is never locked out of incremental fixes. Reserved filenames and memory-inbox documents are exempt (every server-rendered memory is already stamped status: proposed per issue #109).

• Migration vehicle: the maintenance ledger (issue #113, src/data_olympus/maintenance.py) computes status_present_in_all_kb_entries and a capped missing_status_paths list at every index build and nags through the pending_actions CTA (HealthResponse.pending_actions, ConsultResponse.pending_actions) until the corpus is clean. Legacy status-less documents keep being served by kb_search/kb_get, but are never in force (absent status is not in IN_FORCE_STATUSES) and never surfaced by kb_consult. See docs/operations.md section 5.1 for the operator runbook.

• No SPEC.md version bump: SPEC.md section 10 explains why: status has been required (and a lint error when absent) since the 0.1 draft, so this is a reference-implementation enforcement-timing change, not a format change.

applies_when: authoring guidance

applies_when is a YAML list of short, verb-phrase trigger strings describing the coding intents a standard or decision document governs (SPEC.md section 4.2). It is the single highest-weight indexed field in kb_search (matched and boosted alongside title, ahead of description and body text), and it feeds the abstention gate: a query must match title, tags, or applies_when before retrieval proceeds. A non-list value is silently parsed as empty (matching tags' parsing), but unlike tags, kb lint does not currently warn on a missing or malformed applies_when, per SPEC.md section 9 states this explicitly as the one recommended field with no lint coverage today.

supersedes / superseded_by / contradicts: typed lifecycle edges

Shipped in issue #110 across two slices (both merged to release/0.4.0 as of this document):

• Slice 1 (parse + cross-file lint): flat frontmatter fields, normalized to a canonical shape at parse time (format/lint.py's _normalize_multi_ref / _normalize_single_ref). Targets must be stable concept IDs, never paths; _path_shaped warns on a / or .md-suffixed value. Cross-file checks run over an in-memory ID map built from the discovered file list (_cross_file_lifecycle_findings): malformed shape, self-supersession, supersession cycles, and (since format 0.4, issue #259) a supersedes or superseded_by target that does not resolve are errors; dangling contradicts targets, asymmetric pairs, superseded_by set on an in-force document, status: superseded with no superseded_by, path-shaped targets that resolve, and in-force contradicts pairs are warnings. The write path refuses a write that introduces an unresolved supersession target (see docs/serving.md).

• Slice 2 (executable retrieval policy): the indexer materializes both fields into an edges table. An in_force=true query additionally excludes any document that is the target of a supersedes edge whose source document is itself in force (the "in-force-source guard") (format.validate.graph_excluded_ids_sql, is_in_force). A mutually supersessive cycle is not special-cased: every in-force member independently satisfies the exclusion rule. kb_get always resolves regardless of graph-exclusion status and returns the union of a document's own superseded_by claim and any reverse supersedes edge naming it; one consistent computed shape (GetResponse.superseded_by). contradicts never filters or ranks anywhere in the retrieval path; it is annotation surfaced on both docs so an agent reconciles or escalates rather than silently picking a side.

This decision directly answers OKF issue #148's typed-relationships thread: data-olympus keeps the flat scalar/list shape (rather than adopting a nested relationships: block) until OKF settles one, precisely because the important property (the edge is executable retrieval policy, not just navigation metadata) does not require a nested shape. See section 4 below for the conditional migration path if OKF #148 standardizes something else.

derived_from (format 0.5, issue #300) joins the same edges table under its own rel and the same OKF #148 staged-migration commitment (section 7). Unlike supersedes, it is not retrieval policy: when a source leaves force (graph-excluded, deprecated, superseded, rejected or expired), kb lint warns on both ends and kb_get names the in-force dependents (dependents_to_review) and the retired sources (derived_from_retired), so a person decides. rejected counts as retired deliberately, because deriving from an explicitly rejected document is worth surfacing; draft and proposed never governed and do not count. Nothing is filtered or demoted, and every edges query filters on rel, so the new rows cannot enter graph exclusion. The fixture bundle tests/fixtures/okf-derived-from/ carries the field in both forms, and CI feeds it to the pinned OKF reference consumer, so tolerance is exercised rather than asserted.

validity: freshness metadata with hard expiry semantics

Shipped in issue #107, merged to release/0.4.0. An optional nested object, concept-level only:

All dates normalize to ISO YYYY-MM-DD at index/lint time (normalize_validity_date). kb lint treats a malformed validity value (an unparsable date, or validity present but not a mapping) as absent (fail open) while still emitting a warning (_validity_findings in validate.py). The content-change time (generated.at, or the legacy timestamp) is explicitly not a staleness signal: SPEC.md section 4.2 states tooling MUST NOT derive expiry or freshness from it or from a file's last_modified. OKF v0.2's own stale_after and verified families are preserved on import and not interpreted by data-olympus.

3. Runtime-only serving envelope fields

These fields are computed at request time and never written to frontmatter (SPEC.md section 8's "runtime envelope" note). A bundle's on-disk conformance (section 9 of SPEC.md) is judged entirely on frontmatter; these fields are additive and orthogonal, visible only in served MCP/REST responses.

Every row above is shipped and present in src/data_olympus/models.py at the code state this document describes. One further envelope addition, the per-session recap (N committed, M demoted to pending) surfaced through the same pending_actions envelope and through the kb_session_recap tool so a governed-lane demotion is never silent, shipped with issue #112; see the #112 note below.

kb_consult's retrieval is hard-filtered to the in-force class (SPEC.md section 8): the rules it returns are restricted to IN_FORCE_STATUSES, within their validity window, and never a memory-inbox document, so an unreviewed proposed memory or a retired/expired/upcoming document is never presented as a governing rule. This is also where issue #109's decision landed: server- rendered memories (kb_propose_memory) are stamped type: memory / status: proposed so the existing status filter and deviation-only status emission apply with no new code path, and a document under the memory-inbox prefix is never in force regardless of claimed status (format.validate.is_inbox_path / memory_inbox_prefix).

Provenance fields (issue #109, shipped): ProposeMemoryRequest / ProposeEditRequest accept an optional evidence: list[str], echoed on PendingEntry.evidence and AuditEvent.evidence (redacted item-by-item by the same scan that covers reason). source_session and reason were already persisted in pending meta and are now returned by kb_list_pending (PendingEntry.source_session, PendingEntry.reason).

Note on issue #112 (governed-lane protection, shipped). A non-operator-confirmed write may not set or change status into the in-force class, and any agent-proposed edit targeting an already in-force document is demoted to a pending entry regardless of confidence (machine-readable demotion_reason; target lookups fail closed when the index cannot verify the target). An advisory injection-pattern scan annotates pending entries for the reviewer and never blocks or demotes by itself. The whole mechanism is on by default and can be disabled with KB_GOVERNED_LANE_PROTECTION=off. Neither mechanism adds a frontmatter field: the runtime surface is the per-write demotion notice, the kb_session_recap tool and kb session-recap CLI, and the recap item in the pending_actions envelope described above.

4. Experimental / candidate extensions

None of the following are implemented. They are tracked against specific OKF ecosystem threads and are documented here so an adjacent implementer can see where data-olympus currently stands on each axis.

• Structured relationships: block (OKF #148). Conditional: SPEC.md section 4.2 says a nested block (grouping supersedes/superseded_by/contradicts under one key, possibly with per-edge reason/decided_at/source) may be adopted in a future spec version if the upstream OKF discussion settles on a standard shape. Any such change would land as a staged migration (the existing scalar-or-list normalization for supersedes already tolerates more than one authored shape), not a breaking removal of the flat fields.

• Version-keyed validity (OKF #159). Deferred per the #107 decision comment pending that thread settling; today's validity object is concept-level only, with no per-version validity windows.

• Confidence/reliability axis (OKF #151). Deferred per the #107 decision comment. data-olympus's public position on this thread (see the issue #108 context links) is that lifecycle/validity ("is this claim currently allowed to govern behavior") is a first-class sibling axis to reliability/confidence ("how much do we believe this claim"), not a field that should be folded into one object; the two questions have different failure modes and a document can be simultaneously low-confidence and in-force, or high-confidence and retired.

• authority_state / allowed_use enum: explicitly REJECTED. The issue #109 decision considered and rejected adding a parallel authority/allowed-use vocabulary derived from status. Reasoning: a second, derived vocabulary that must track status forever is exactly the "two fields that can only agree or disagree" failure mode that also rules out allowed_use as a standalone field: it can drift from its source of truth with no structural guarantee. The decision fixed the real gaps (kb_consult in-force filtering, memory stamping, the mandatory-status direction in #114, computed in_force) at the root instead of adding a vocabulary to paper over them. origin_provider (constant inside data-olympus) and asserted_by (duplicate of agent_identity) were rejected for the same not-worth-a-parallel-field reasoning. The authority-forgery risk that motivated the original ask was explicitly split out of the #109 slice and assigned to governed-lane protection (issue #112, shipped in 0.4.0; see the #112 note in section 3). It is a write-path control that prevents an unreviewed write from minting in-force status, rather than a frontmatter field describing the risk after the fact.

5. Per-field compatibility table

"OKF-baseline readable" means an OKF consumer that tolerates unknown keys (per section 1's inherited rule) can safely ignore the field. Every data-olympus extension is readable this way by construction. The table also includes the OKF-recommended fields this profile constrains further (type, title, description, tags, timestamp), so a single table answers both "will an OKF consumer choke on this" and "what does data-olympus tooling do with it".

6. Relationship to adjacent projects

The fuller comparison against OKF itself, enterprise catalogs, markdown KB tools, agent-context conventions, RAG/memory layers, and ADR tooling lives in docs/comparison.md; this section covers only the three adjacent projects surfaced in the OKF ecosystem threads that motivated this document (issue #108) and does not duplicate that comparison's structure.

• inkxel/throughline is a commit-triggered memory scaffold for a single code repo (journal, decisions, wiki, research folders under knowledge/) that exports to OKF on demand, validated against Google's reference tooling. It targets a different unit of governance than data-olympus: a per-repo, largely single-writer memory layer authored through a git hook, versus data-olympus's cross-project, multi- agent, server-enforced propose/pending/commit pipeline over a live MCP endpoint. Throughline's author is an active participant in the same OKF lifecycle/relationships threads this document tracks (OKF #148, #151) and has converged on a structurally similar supersedes/contradicts shape independently.

• inkxel/dotKnowledge (early RFC, v0.0.x) is a portable "sealed bundle" format scoped to a subject (person/org/brand/project) with a capsule.yaml/BOUNDARY manifest declaring a rises: policy (what may be synthesized out of the bundle) and a draft "convergence protocol" for mounting multiple bundles under one boundary-respecting layer. It names Throughline as its reference engine. data-olympus has no subject-typed bundle concept and no boundary/rises policy; its closest analog is the write-path's structural rule set (path prefix allow-list, KB_WRITE_BLOCK_TIERS/KB_WRITE_BLOCK_PATHS), which governs what may be written into a bundle rather than what may be synthesized out of one: a different axis, not a competing implementation of the same one.

• dynamicfeed/signed-okf adds a tamper-evident, Ed25519-signed manifest (okf.manifest.json) over an OKF bundle: it hashes every file and lets a holder of the issuer's public key verify the bundle was signed by the claimed issuer and is unaltered. This is a provenance/integrity concern, orthogonal to the governance extensions in this document: data-olympus's write-path integrity guarantees come from git history and the single-writer MCP pipeline's commit provenance (agent identity, session, evidence; section 3 above), not from cryptographic signing of the bundle at rest. The two are compatible in principle (a data-olympus bundle could be signed with signed-okf tooling as an additional, external layer) but nothing in data-olympus verifies or produces a signed-okf manifest today.

• Generic memory/RAG MCP servers (Cognee, Zep/Graphiti, Mem0, Letta, Supermemory) are covered in depth in docs/comparison.md. The short version for this document's purposes: none of them expose an OKF-readable frontmatter schema or a lint-checked lifecycle vocabulary, they are self-mutating semantic memory, not curated, reviewed, versioned knowledge, so they are not directly comparable on the field-by-field axis this document covers.

7. Migration policy

Extension fields graduate out of this profile in one of two directions:

• Into the core OKF spec, if an upstream OKF discussion standardizes a shape data-olympus's flat fields currently approximate. The clearest candidate is supersedes/superseded_by/contradicts/derived_from under OKF #148: if OKF adopts a structured relationships: block, SPEC.md section 4.2 already commits to accepting the upstream shape via the same importer-style normalization that today accepts both a scalar and a list for supersedes (the bundled ADR importer emits a bare scalar for one entry, a list for more than one). A staged migration, never a breaking removal of the flat fields in the same release.

• Out of "experimental" and into "stable governance extension", when a candidate in section 4 ships with lint coverage and a SPEC.md normative section, exactly as validity (issue #107) and the typed lifecycle edges (issue #110) did in this release. The rejected candidates in section 4 (authority_state/allowed_use) do not have a graduation path: they were evaluated and declined, not deferred.

Format version bumps follow SPEC.md section 10's semver-style rule: a new optional field or enum value is a minor bump (existing bundles unaffected); a required-field change or removed field is a major bump requiring a staged migration. Issue #114 is the instructive counter-example: it changed write- path enforcement of an already-required field, not the field's semantics, so SPEC.md section 10 explicitly records it as a reference-implementation enforcement-timing change with no version bump (see section 2 above).

See also

• SPEC.md, especially sections 4 (frontmatter), 8 (serving model, runtime envelope), 9 (conformance), 10 (versioning), and 11 (relationship to other formats).

• docs/comparison.md for the full competitive landscape.

• docs/enforcement.md for the kb_consult gate this document's section 3 summarizes.

• docs/operations.md for the maintenance ledger's operator- facing workflow.

• Issue #82: OKF conformance test tracker, referenced throughout section 1.