Vault-LD

Brain spec

First committed by dfreytag

Published in The-Knowledge-Graph-Guys/vault-ld. Publisher page

246 GitHub starsUpdated 2026-07-15

What it is

Vault-LD: an open spec for Markdown vaults as linked data. YAML-LD frontmatter + a shared `@context` = an RDF knowledge graph. Prose for humans and LLMs, triples for machines.

— the description by dfreytag

AutoGineer overview, from the brain’s own files

Vault-LD provides a standardized way to bridge human-readable Markdown notes with machine-readable RDF knowledge graphs. By utilizing YAML-LD in the frontmatter of Markdown files and a shared context, it allows for a seamless roundtrip between prose and structured data. This enables AI agents and other systems to reason over a vault's contents as a graph while maintaining a format that remains easy for humans to edit and version control in Git. The specification includes reference tools for exporting vaults to RDF and ingesting RDF into vault structures, ensuring full fidelity in both directions.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

Vault-LD organizes information in a folder-based structure of Markdown files. Each file uses YAML-LD frontmatter to define its properties and relationships, which are resolved against a shared context file. The system uses reference scripts to convert these Markdown files into RDF triples (schema and data layers) and vice versa, allowing the vault to function as both a human-editable wiki and a structured knowledge graph.

Folders
Folder	Role	Files
.github	agent-config	11
docs	docs	39
images	attachments	3
scripts	scripts	7
Vault-LD Example	folder	14

The agent writes

.github/ evidence: AGENTS.md

Requirements

Works with: OpenAI Codex, GitHub Copilot, Cursor, Kiro, OpenCode, Amp, Cline, JetBrains Junie, Devin (ex-Windsurf)

Set it up

Clone the repository at the commit AutoGineer scanned:

git clone https://github.com/The-Knowledge-Graph-Guys/vault-ld.git && cd vault-ld && git checkout 025e71be8d810e387dc451c920e05472d1c47170

Point Claude Code at AGENTS.md: it does not read that file on its own.
cd vault-ld && claude
AutoGineer scanned these files at commit 025e71b.
First committed by dfreytag. Published in The-Knowledge-Graph-Guys/vault-ld. Licence: Apache License 2.0.

Usage

AutoGineer overview, from the brain’s own files

Defining business semantics and ontologies within a Markdown-based wiki.
Enabling AI agents to perform graph-based reasoning over local note collections.
Synchronizing external RDF data sources with a human-friendly Markdown vault.
Maintaining a version-controlled knowledge base that is simultaneously readable by humans and machines.

“List all recipes in the vault that require chickpeas.”: The agent queries the RDF graph derived from the Markdown frontmatter to identify all notes with type 'Recipe' and 'requiresIngredient' set to 'Chickpeas'.
“Update the prep time for the Hummus recipe to 30 minutes.”: The agent modifies the 'prepTimeMinutes' field in the YAML frontmatter of the Hummus.md file, ensuring the change is reflected in both the prose and the underlying knowledge graph.

Documentation by dfreytag

Vault-LD

An open format for knowledge that two very different readers can share at once:<br>a person editing notes, and a machine reasoning over a graph.

Image: CI Image: Release Image: Docs Image: License Image: GitHub stars Image: Issues Image: Contributions welcome

Specification · Docs site · Example vault · Contributing

</div>

RDF in the Vault

Karpathy's "LLM wiki" idea points at something real: a directory of plain Markdown notes is one of the most powerful ways to work with LLMs. It is readable by you, chunkable by a model, and diffable in git, with no database required. The YAML frontmatter at the centre of this approach carries a hidden pattern: it can map onto YAML-LD (JSON-LD with a YAML serialization). Resolve that frontmatter through one shared context and your notes stop being tagged text. They become linked data.

This matters for anyone already invested in RDF-based ontologies. Your business semantics (your classes, properties, and controlled vocabularies) drop straight into the wiki your team already uses, much as schema.org gave web pages a shared vocabulary without asking authors to leave HTML. The benefit runs both ways: business semantics arrive inside the wiki, and the ontology gains a human-friendly surface, where classes and properties can be read and edited as notes rather than as raw Turtle.

Both directions work because of the roundtrip. Since the frontmatter is YAML-LD, a note and an ontology definition are the same kind of object, and either can be projected losslessly to RDF and back. An ontology can enter as Turtle, be edited as Markdown, and leave as Turtle again, or the reverse, with no canonical "real" form privileged over the other.

Image: Vault-LD

What's here

SPEC.md: the normative reference. It defines how frontmatter becomes a knowledge graph (§4) and how any RDF graph round-trips through the vault format with full fidelity (§5), along with terminology, directory structure, conformance criteria, and a compatibility profile for lifting OKF-style Markdown bundles into linked data (Appendix B).
Vault-LD Example/: a complete, copyable vault that demonstrates every rule in the spec. It holds one ontology (Culinary), one controlled vocabulary (Difficulty Levels), and one instance (hummus). Its root context.jsonld composes the Culinary ontology's own context, showing how multiple contexts compose (SPEC §4.2).
scripts/vault_to_rdf.py: a reference exporter that projects a vault to RDF — see Exporting to RDF below.
scripts/rdf_to_vault.py: the reference ingester running the other way, RDF → vault — see Ingesting RDF below. Together the two tools close the roundtrip.

Exporting to RDF

The repo ships scripts/vault_to_rdf.py, a small Python tool that reads a vault and emits Turtle, split by layer into two files:

schema.ttl — the schema layer (classes, properties, concepts), each ontology in its own namespace (cul:, diff:, …),
data.ttl — the instance layer (typed notes) under the vault base, each IRI minted from the note's vault-relative path.

It classifies each note by its folder, mints each subject under the @base of its governing context (schema notes from their file name, instances from their vault-relative path — SPEC §4.5), resolves frontmatter through the vault's composed @context, and flags anything it can't map rather than dropping it. Export the example vault to build/ in one line:

pip install rdflib pyyaml && python scripts/vault_to_rdf.py "Vault-LD Example" --out-dir build

See EXPORT.md for the full usage guide, flags, and how layer classification and context composition work.

Ingesting RDF

The reverse direction, scripts/rdf_to_vault.py, projects any RDF graph into the vault format (SPEC §5.5) — one note per subject, hierarchy in frontmatter, IRI objects as wiki links. Point it at an existing vault and it updates notes in place, preserving bodies, tags:, and folder structure, and leaving unchanged notes untouched; point it at an empty directory and it builds the whole vault, synthesizing a composed context.jsonld when none exists. Predicates the context doesn't map get a short name coined and added to the right context document; anything the format can't express (blank nodes, language tags) is flagged, never dropped (SPEC §5.6). Import a foreign ontology in one line:

python scripts/rdf_to_vault.py MyVault foreign-ontology.ttl

See INGEST.md for the full usage guide and the fidelity rules. Because the two tools are inverses, vault → RDF → vault is a no-op and RDF → vault → RDF is graph-isomorphic.

A taste

Read this note:

---
type: "[[Recipe]]"
requiresIngredient: "[[Chickpeas]]"
prepTimeMinutes: 25
---

# Hummus
A smooth purée of chickpeas, tahini, lemon, and garlic.

You have just read these triples:

@prefix : <https://example.org/> .

:Hummus a :Recipe ;
            :requiresIngredient :Chickpeas ;
            :prepTimeMinutes 25 .

The frontmatter is the graph; the body is prose for human and machine readers. One shared context.jsonld supplies the IRIs, datatypes, and prefixes, so authors write short, clean names while the semantics are supplied for them. See the Vault-LD Specification for the full rules.

Contributing

Vault-LD is an open standard and contributions are welcome: see CONTRIBUTING.md. Spec changes start life as an issue using the Spec change proposal template; fixes to the reference tools, example vault, and docs can go straight to a pull request.

License

Apache License 2.0.

If Vault-LD is useful to you, a ⭐ on GitHub helps others find it.

</div>

docs/versioned_docs/version-0.5.0/HISTORY.md

description: "The narrative history of Vault-LD, release by release — what changed, why it changed, and what it means for implementers."

History

The narrative record of Vault-LD, release by release — what changed, why it changed, and what it means for implementers. This file is written by humans and their agents (see AGENTS.md): substantive work adds its rationale under ## [Unreleased], and the section is titled with the version when that release ships.

The mechanical per-PR ledger — every merged change, one line each, with links — is generated by release-please: attached to each GitHub release and accumulated in .github/CHANGELOG.md. Neither is ever written by hand.

0.5.0 — 2026-07-14

Process

The docs site is now search-discoverable — moving to vault-ld.org reset the site's search presence, and an audit found it actively hurting itself: the same documents served at three URL sets (the release at the root, /next, superseded versions), splitting ranking signal three ways, and the homepage <title> fell back to the doc id ("index") because Docusaurus cannot extract a title from the README's raw-HTML header. Superseded and unreleased versions are now noindex (cutting a release adds the version it demotes to the list in docusaurus.config.js), so search engines see only the released docs at the site root, and robots.txt points at the sitemap. Page titles and meta descriptions are injected at assembly time (assemble.mjs) rather than written into the root markdown — frontmatter would render as a metadata table atop the README on GitHub — the same source-stays-clean principle as the existing image-path rewriting. A schema.org WebSite record with alternateName spellings ("Vault LD", "vld") joins the existing JSON-LD so unhyphenated queries connect to the brand, and a social card (og:image) gives shared links a preview image.

0.4.0 — 2026-07-14

Process

Pinned docs links resolve from day one — the latest released snapshot serves at the docs-site root, so its version-prefixed URLs (…/vault-ld/0.3.0/SPEC) would 404 until the next release demotes it to an archived path — yet release notes and HISTORY headings want to link the pinned form immediately. The site's 404 page (which GitHub Pages serves for any unknown path) now forwards a latest-version-prefixed path to the same page at the root (docs/src/theme/NotFound/Content/); every other missing path still gets a real 404. The alternative — giving every released version an explicit path and redirecting the root to the newest — was rejected as more moving parts: it takes the docs off the site root and adds a config touch to every release.
Self-updating README badges — the docs badge reads the released version live from docs/versions.json on main, so it tracks each snapshot with no README edit per release; the release badge went live after v0.3.0.

0.3.0 — 2026-07-07

Process

Automated releases via release-please — PR titles are now conventional commits (enforced in CI, squash-merged so the title is the commit), and release-please maintains a running release PR whose merge cuts the version, tag, and GitHub release. A bespoke label-driven release workflow was prototyped first and discarded before ever landing: it kept accumulating edge cases the standard tool already handles (concurrent-merge races, batch bumping, re-run idempotency), and owning release machinery isn't this project's job. The record split three ways: the machine ledger in .github/CHANGELOG.md and on each GitHub release, the human/agent narrative here in HISTORY.md, and AGENTS.md instructing agents to record their rationale so the repo's reasoning stays queryable.
Versioned documentation site — the spec is now published as a static site (docs/, Docusaurus, deployed to GitHub Pages on every push to main). Docusaurus was chosen over MkDocs/Jekyll because the site is meant to grow beyond rendered markdown — media, custom theming, React components — and it is the established standard for that shape of OSS docs. The pages are assembled from the repo's root markdown at build time (docs/assemble.mjs) so there is no second copy to drift, and every page embeds schema.org JSON-LD — a linked-data spec should itself be discoverable as linked data. Released versions are frozen snapshots cut at release time (npm run snapshot -- X.Y.Z, part of the release ritual in CONTRIBUTING.md): the site root always serves the latest release, older releases stay in the version dropdown, and main's live state is browsable as "Next (unreleased)" — so a reader can never mistake unreleased spec text for a published version.

Security

The reference tools now treat both a cloned vault and ingested RDF as untrusted input throughout (see the new SECURITY.md for the trust model). All findings from the branch security review are fixed and exploit-tested; the roundtrip is unchanged.

Contained file writes — a vld:path placement hint from ingested RDF can no longer escape the vault. Absolute paths, .. traversal, and hints that resolve outside the vault root (including via symlinks) are refused, closing an arbitrary-file-write path in rdf_to_vault.py.
No network I/O on ingest — rdf_to_vault.py disables outbound requests before parsing, so hostile RDF can no longer make rdflib fetch remote @context/imported documents (SSRF). A remote reference now fails with a clear error instead.
Contained context reads — a @context string reference in a vault can no longer read files outside the vault; absolute and .. references are refused (vault_to_rdf.py), matching the existing remote-refusal.
Bounded frontmatter parsing — YAML frontmatter is parsed with an alias-refusing SafeLoader variant (blocks "billion laughs" alias-expansion DoS) and capped at 1 MiB per note; malformed frontmatter skips the note rather than aborting the whole sweep.
Symlink-safe vault sweeps — both tools skip .md files that are symlinks or resolve outside the vault root.
Pinned dependencies — scripts/requirements.txt pins exact versions (rdflib==7.6.0, PyYAML==6.0.3); the ingest tool's safety depends on the resolved parser behaviour.
Repository hygiene — added .github/workflows/ci.yml that rejects tracked bytecode/venv and runs the roundtrip as a regression guard; removed stray local __pycache__ artifacts.
Security regression suite — new scripts/test_security.py (make test, stdlib unittest only) pins every SECURITY.md guarantee as an executable test and runs in CI, so a pull request cannot silently weaken a patch.
No frontmatter injection — coined YAML keys (and all generated scalars) are quoted unless they provably re-parse to themselves, so a hostile predicate IRI whose localname contains a newline or : can no longer inject frontmatter lines into generated notes (rdf_to_vault.py).
Hardened file-stem sanitisation — stems derived from IRI localnames now also strip control characters and leading dots (./../hidden files), and every note write is re-checked against the vault root as a final containment guard (rdf_to_vault.py).
Bounded context and note reads — context.jsonld documents are capped at 4 MiB and parse failures warn instead of crashing (a malformed root context is a hard error and is never overwritten); the 1 MiB frontmatter cap now binds before a note is read into memory, and undecodable notes are skipped instead of aborting the sweep (vault_to_rdf.py, rdf_to_vault.py).
Network kill-switch everywhere — disable_network moved to vault_to_rdf.py and is now also applied by compare_builds.py, so no reference tool can be made to fetch a remote document while parsing RDF.
CI hardening — the workflow token is read-only (permissions: contents: read), checkout no longer persists credentials, and actions are pinned to full commit SHAs (actions/checkout v4.3.1, actions/setup-python v5.6.0) instead of mutable tags.
--unsafe-allow-network ingest flag — an explicit opt-out of the network kill-switch for RDF whose remote references (public @context documents, e.g. schema.org's) you have verified as trustworthy. It prints a warning banner and pauses five seconds before proceeding so the run can be cancelled with Ctrl-C; cancellation exits before anything is fetched or written. The fail-closed default is unchanged.

[0.2.0] — 2026-07-03

Specification

Identity (§4.5)

One minting rule for every note: the IRI is the governing @base + the file name alone, without .md. Folders never enter any IRI — Recipes/ is shelving, not naming. Moving a note never re-mints its identity; only renaming does.
The governing context is formalised as the nearest context.jsonld at or above the note (the scoped-base assembly rule, §4.2). Only the vault-root context is mandatory; per-ontology, per-vocabulary, and nested data-folder contexts are optional refinements that govern their subtrees.
An explicit id MUST be a full absolute IRI (http(s)://…), used verbatim. It disambiguates same-named files (which otherwise mint the same IRI and merge), pins an identity across renames, and can place a subject in any namespace. Relative values are non-conforming.
Non-IRI-safe file names (spaces, most commonly) are percent-encoded on minting and percent-decoded on ingest, so Red Lentil Soup.md round-trips without a pin.

Placement roundtrip (§5.4, §5.5)

vld:path "<path>.md" — Vault-LD's own string-valued property, the sole term of the vld: namespace (https://github.com/The-Knowledge-Graph-Guys/vault-ld#) — carries the true, context-relative location of every note the graph could not otherwise reconstruct — every pinned note, every instance not sitting directly in its governing context's folder, and every schema note away from the flat placement of §5.1. The triple exists only on the RDF side: it materialises on export and dissolves back into file placement on ingest, never appearing in frontmatter. File placement is now part of roundtrip fidelity (§5.6).
Emitting the placement triples is required only of a roundtrip-face export; an export produced purely for querying — a read-only artifact — omits them (§5.4 step 7).

Hierarchy (§5.2)

Frontmatter is the only carrier of subClassOf / subPropertyOf / broader / topConceptOf; a tool MUST NOT derive hierarchy from folder nesting. Nesting a schema file is purely organisational.
skos:topConceptOf joins the frontmatter contract, with the concept-vs-scheme distinction (broader links concept to concept, never to the scheme) made explicit.
Canonical, reconstructable schema placement is the flat form of §5.1 (classes in Classes/, properties in Properties/, concepts at the vocabulary top level). Folder organisation beyond that is deliberately a tool's own affair, outside the spec — any layout round-trips via vld:path.

Keywords and the context (§4.2, §4.3)

Keyword aliasing: the context maps type → @type and id → @id (JSON-LD 1.1 §4.1.7), so notes read as plain YAML keys with no quoting. Conforming tools honour declared aliases on input and write them on output.
Host-tool keys (tags, aliases, cssclasses) are recognised as editor affordances: never emitted as triples, never warned about, and promotable to first-class terms by mapping them in the context.
Context composition hardened: array contexts compose by reference with left-to-right override; a conflicting redefinition of an established term or prefix warns (an identical re-declaration stays silent); the scoped-@base rule is framed as Vault-LD's assembly rule above stock JSON-LD, with the interoperability consequence (generic processors ignore @base in referenced contexts) stated plainly.

Wiki links (§4.4.1)

Full link grammar: paths disambiguate, aliases are display-only, fragments resolve to the note. Generation and resolution are two halves of one contract — ambiguous names are emitted path-qualified and resolved right-aligned on segment boundaries; dangling links are flagged, never dropped.

Conformance and compatibility (§6, Appendix B)

Conformance checklist expanded to cover name-only minting, absolute id, frontmatter-only hierarchy, vld:path handling, and body preservation; conforming exporter / ingester / roundtrip tool defined.
Appendix B: lifting an OKF bundle — a compatibility profile turning a Google Open Knowledge Format bundle into a conforming vault without modifying a single bundle file.

Tools

scripts/rdf_to_vault.py — the reference ingester (RDF → vault), closing the roundtrip: one note per subject, in-place updates that preserve bodies and host keys, context synthesis and term coining for unmapped predicates, vld:path consumption, and percent-decoded file names. Opt-in --nest lays fresh schema files out along the declared hierarchy — a folder-management convenience, not spec behaviour.
Tooling moved to scripts/, with a root Makefile (make roundtrip = export → rehydrate → rebuild → graph-isomorphic compare), scripts/compare_builds.py, and scripts/requirements.txt.
--data-ns on both tools is now an explicit override for the vault-root base only; nested data-folder contexts still govern their subtrees.
Exporter defaults to query-only output — a lean, read-only .ttl with no placement bookkeeping; --source opts into the roundtrip face that emits the vld:path triples (used by make roundtrip).
Makefile venv paths work on Windows (venv/Scripts/) as well as Unix (venv/bin/); added a make query target for the default lean export.
Exporter warnings sharpened: same-IRI merges suggest an explicit absolute id, ambiguous names suggest path-qualified links, non-absolute id values are flagged and recovered.

Project

Open-source governance: Apache-2.0 LICENSE, CONTRIBUTING.md (spec changes start as issues), CODE_OF_CONDUCT.md, issue and PR templates.

[0.1.0] — 2026-07-02

Initial release of the specification and the reference exporter.

The format: a directory of Markdown notes read as an RDF graph — each note's YAML frontmatter, interpreted as YAML-LD through one shared, external context.jsonld, is that note's triples; the body is documentation for human and machine readers, deliberately not converted to RDF.
Resource-per-file: one note = one subject. Two layers, one mechanism: the schema layer under Ontologies/ and Vocabularies/ (classes, properties, concept schemes, concepts) and the instance layer everywhere else, linked by @type wiki links.
Wiki links are the edges: [[Target]] is simultaneously the object IRI and a clickable, bidirectional link in the host tool (Obsidian, Notion, any editor).
Identity: a note's IRI is its file name resolved against its namespace's @base — ontologies and vocabularies declare scoped bases in their own composed contexts, instances resolve against the data base; an explicit @id pins a stable IRI.
Hierarchy lives in the frontmatter: subClassOf / subPropertyOf / broader as wiki links; the Classes/ folder is flat and folders carry no formal meaning.
The roundtrip: export (vault → Turtle, split into schema.ttl and data.ttl) and ingest (RDF → vault) specified as inverses, with neither serialization privileged as source of truth; unmapped constructs are flagged, never silently dropped.
Reference exporter vault_to_rdf.py and the bundled example vault (Culinary ontology, DifficultyLevels vocabulary, recipe instances).
Conformance criteria (§6), relationship to existing work (§7), and a minimal copyable example bundle (Appendix A).

85 files · 5 not served here

Advantages

Enables lossless roundtripping between Markdown notes and RDF graphs. evidence: README.md
Allows business semantics to be edited as human-friendly Markdown rather than raw Turtle. evidence: README.md
Supports composition of multiple contexts for complex knowledge modeling. evidence: README.md

Limitations

Requires adherence to specific YAML-LD frontmatter and context structures. evidence: README.md
Certain RDF features like blank nodes or language tags may require specific handling or flagging. evidence: README.md

Security

Risk level: Low risk

This brain runs

Agent instructions that run or fetch something (3)

AGENTS.md: This is the repo's institutional memory, and it is load-bearing: contributors are expected to point their own agent at HISTORY.md (plus `git log`) to produce commits and PRs that match this repo's standards.
AGENTS.md: **Security guarantees are pinned as tests** (`scripts/test_security.py`, run via `make test` in CI).
AGENTS.md: **`make test` and `make roundtrip` must be green** before proposing a change.