AutoGineer
Generated banner for MirrorArc
Generated logo for MirrorArc

MirrorArc

LLM wiki kit

First committed by cz1993

Published in cz1993/MirrorArc. Publisher page

97 GitHub stars

What it is

Local-first, governed documentation for humans and AI agents—mirror Office, PDF, and GitHub sources into linked Markdown knowledge workspaces.
— the description by cz1993

Summary from the brain’s metadata

Local-first, governed documentation for humans and AI agents—mirror Office, PDF, and GitHub sources into linked Markdown knowledge workspaces. Preserve sources. Minimize derived documents. Generate knowledge views when they are needed.

How it works

Memory model: LLM wiki

AutoGineer overview, from the brain’s own files

Memory model: LLM wiki. Folders: .github (agent config, 3 files); docs (docs, 33 files); examples (examples, 139 files); scripts (scripts, 11 files); src (source code, 126 files); template (templates, 54 files); tests (tests, 57 files). The agent reads README.md, docs/PRODUCT.md. The agent writes CHANGELOG.md.

Folders
FolderRoleFiles
.githubagent-config3
docsdocs33
examplesexamples139
scriptsscripts11
srcsource-code126
templatetemplates54
teststests57
.githooksfolder1
clifolder1

The agent reads

The agent writes

Requirements

Apps: Obsidian

Runtimes: Python >=3.11

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Google Antigravity, Kiro, OpenCode, Amp, Roo Code, Cline, JetBrains Junie, Devin (ex-Windsurf)

Set it up

  1. This brain runs code automatically: 2 scripts. Review .githooks/pre-commit, scripts/init.sh before opening the folder in Claude Code. No step below turns a hook on for you.

    • .githooks/pre-commit: runs on git pre-commit
    • scripts/init.sh: sh scripts/init.sh

  2. Clone the repository at the commit AutoGineer scanned:

    git clone https://github.com/cz1993/MirrorArc.git && cd MirrorArc && git checkout 1df13ebf6efb33018e29a1ea365cde0cf53d41c7

  3. Point Claude Code at AGENTS.md: it does not read that file on its own.

  4. cd MirrorArc && claude

  5. AutoGineer scanned these files at commit 1df13eb.

  6. Not scanned by AutoGineerThe author’s README also describes installing it from a clone of its repository. Read its install section: AutoGineer did not scan that path. https://github.com/cz1993/MirrorArc/blob/1df13ebf6efb33018e29a1ea365cde0cf53d41c7/README.md#quick-start

  7. First committed by cz1993. Published in cz1993/MirrorArc. Licence: GNU Affero General Public License v3.0 only.

Usage

From the brain’s own files

  • Create a vault without cloning MirrorArc itself.
  • The installable command requires Python 3.11+; uvx and pipx run will build an isolated environment when they can find a compatible Python.
  • Before the package is published to PyPI, use the Git URL:
  • Once MirrorArc is published to PyPI, the equivalent short forms are uvx mirrorarc init --profile data-product ~/my-data-product and pipx run mirrorarc init --profile data-product ~/my-data-product.
  • For repeated local use, install the console command once:
  • Then open the vault in Obsidian if you want a human UI, point your agent at it (CLAUDE.md routes all agents to _meta/agent-rules.md), and run the installed command from any folder:
  • Run sandbox from a duplicated pilot vault, not the original document folder.
  • It is read-only and checks copy-boundary, mirror isolation, manifest/recovery readiness, and basic backup posture without printing source paths or document text.

Documentation by cz1993

MirrorArc

Image: Python 3.11+ Image: License: AGPL-3.0 Image: Status: technical alpha

Preserve sources. Minimize derived documents. Generate knowledge views when they are needed.

Open the live Ontario Grid demo portal →

MirrorArc is an open-source, local-first Python toolkit that turns changing Office files, PDFs, GitHub repositories, datasets, and Markdown into a governed knowledge-projection system. It keeps original records authoritative, creates deterministic Markdown projections for opaque sources, maintains source-backed relationships, and renders human or agent context without adding a vector database or an AI-generated wiki.

Image: MirrorArc beginner landing page showing the Ontario Electricity evidence tutorial

Why MirrorArc exists

Generative AI made documentation fast and cheap to create. It did not make a growing collection of summaries, entity pages, hubs, and agent notes cheap to verify or maintain. MirrorArc treats every durable derivative as a liability that must justify its existence; the information and knowledge carried by the sources remain the asset.

The product principle is:

Preserve authoritative records. Treat every durable derivative as a maintenance liability. Generate views instead of accumulating documents.

Common failureMirrorArc's response
Office files, PDFs, spreadsheets, repos, and Markdown stay isolated.L1 specular projections: opaque sources receive stable, source-addressed Markdown read models while originals remain authoritative.
AI documentation creates more files than anyone can govern.Anti-proliferation by architecture: relationships are data and synthesis is a view, not another note by default.
Cross-document meaning is hidden or asserted without evidence.Relationship and provenance ledger: typed connections carry evidence anchors, hashes, generation method, confidence, and review state.
Human readers need interpretation rather than raw extraction.L2 knowledge views: many sources produce a few purpose- and audience-specific reading surfaces that are ephemeral unless pinned or reviewed.
Source changes leave summaries and agent context stale.Dependency-aware refresh: changed sources invalidate only affected projections, relationships, views, reviews, and context.
Sensitive or stale material enters an opaque index.Visible governance: authority, licensing, PII boundaries, lifecycle, freshness, and publication gates remain explicit.

See the connected evidence

The self-contained Catalog Explorer provides a relationship map, a provenance and lifecycle inspector, and a rendered document view. When a governed repository is selected, an optional Code view adds revision-bound source evidence, call paths, impact, and affected-test candidates. The catalog panel is resizable, long filenames wrap instead of disappearing, and INDEX.md is pinned as the beginner guide.

Image: MirrorArc relationship map connecting an original Word document and its generated Markdown mirror

Image: MirrorArc metadata view showing provenance, authority, lifecycle state, and source-to-mirror relationships

Flagship demo: Ontario electricity evidence workspace

examples/ontario-electricity-evidence-vault/ is the flagship clean-room data-product example, built from OGL Ontario data, metadata-only public references, independently authored Word/Excel artifacts, and a synthetic repository fixture. The current snapshot demonstrates the implemented L1, lifecycle, catalog, and portal foundation. Its large hand-authored Markdown layer predates the knowledge-projection architecture and is now a migration fixture rather than the target output.

The local implementation supports source-addressed L1, evidence relationships, reviewed L2 views, dynamic/frozen context, and dependency invalidation. The checked-in example remains deliberately small. Historical stress tests are recorded separately; current batch proof is documented in the Trustworthy Context handoff when available.

The committed snapshot is an independently assembled educational corpus, not live grid data, forecasting, alerts, or an affiliated Ontario energy product.

Launch the hosted demo and start with the pinned INDEX.md five-minute tour. GitHub Pages rebuilds this content-enabled portal only from the provenance-documented public example and blocks deployment unless both the repository-wide provenance/no-data gate and the generated-artifact scan pass alongside vault lint.

The hosted project is progressively discoverable: the project overview identifies MirrorArc's purpose, differentiators, installation path, licence, and canonical GitHub source in semantic HTML. The demo's initial HTML contains the rendered beginner tour, while documents/ provides one semantic, no-JavaScript page per public record. Search and agent discovery surfaces include sitemap.xml, robots.txt, llms.txt, raw public Markdown, and a metadata-only catalog.json.

Try the demo locally

git clone https://github.com/cz1993/MirrorArc.git
cd MirrorArc
python3.11 -m venv .venv
. .venv/bin/activate
python -m pip install -e .

mirrorarc --root examples/ontario-electricity-evidence-vault sync
mirrorarc --root examples/ontario-electricity-evidence-vault code doctor
# If Ready, optionally analyze the synthetic repository fixture:
mirrorarc --root examples/ontario-electricity-evidence-vault code analyze --repo repo_252e1077b0082592ba3d --symbol demand_change
mirrorarc --root examples/ontario-electricity-evidence-vault catalog --html --include-content
python -m http.server 8000 --directory examples/ontario-electricity-evidence-vault

Open http://127.0.0.1:8000/CATALOG.html. Start with the pinned INDEX.md, then follow the five-minute tour. The --include-content output embeds bounded Markdown bodies and must remain local for private or proprietary vaults. The hosted MirrorArc demo is a narrow exception: it is built only from the public example corpus and scanned again before every deployment.

Choose a task and export its evidence

From a synced vault, choose a task, inspect what was selected and omitted, then freeze evidence:

mirrorarc --root <vault> context build --lens orientation --mode metadata --query "inspection interval" --json
mirrorarc --root <vault> context build --lens orientation --query "inspection interval" --name inspection-review
# Use the definition ID printed above:
mirrorarc --root <vault> context resolve <definition-id> --json
mirrorarc --root <vault> context freeze --definition <definition-id> --task "Explain the inspection interval with citations."
mirrorarc --root <vault> catalog --html

Query retrieval is opt-in: lexical ranking can miss relevant evidence. Inspect source hashes, exact spans, selected/excluded reasons, warnings and omissions before use. Metadata contains references only. Frozen packs include bounded evidence and report exact serialized bytes plus an estimated token count, not a model-independent token guarantee. A tiny budget can fail because instructions and citations alone do not fit. offline_complete is false if selected evidence is missing or truncated; it is never a guarantee that the sources answer your question.

The portable Catalog's selection download stays metadata-only. Its governed-context panel shows saved tasks, copyable commands and frozen status. HTML does not execute those commands. Use --include-content only when intentionally embedding local evidence; protect the resulting file according to its sensitivity. Source changes require refresh, context resolution, and Catalog regeneration. Frozen bytes remain unchanged and status reports staleness.

Review PDF page evidence

The optional PageIndex integration adds physical-page evidence and unreviewed answer candidates to the same governed workflow. Install its separate pinned runtime using the PageIndex setup and review guide, then start with model-free indexing:

mirrorarc --root /absolute/path/to/copied-vault document doctor
# Use a registered PDF source ID from mirrorarc status --json:
mirrorarc --root /absolute/path/to/copied-vault document index --source SOURCE_ID --context frozen --page 2
mirrorarc --root /absolute/path/to/copied-vault catalog --html --include-content

In the Catalog, select the PDF, then Document metadata → PDF page evidence. Optional questions require an approved model/endpoint and explicit per-command consent; indexing alone does not create an answer. Answer review shows frozen cited excerpts, current/stale state and uncertainty. A valid page address does not establish that a claim is supported. The local results separate tested behavior from real-model quality, owner acceptance and release readiness.

How it works

  1. Register authoritative sources without changing them.

  2. Project each opaque source into one stable, machine-owned L1 Markdown read model.

  3. Connect evidence through typed relationships with provenance and review state rather than creating more notes.

  4. Interpret many sources through a small number of purpose-specific L2 knowledge views.

  5. Assemble dynamic or frozen context for a human review or agent task.

  6. Refresh only affected projections and dependants when sources change, with reconciliation and full sync retained for recovery and verification.

The default data-product profile covers sources, contracts, pipelines, analysis, models, outputs, governance, and operations. Additional packaged profiles support business operations, research/learning, software-project documentation, and minimal blank starts.

Inspired by Andrej Karpathy's "LLM wiki" pattern. See docs/positioning.md for the product landscape and boundaries.

Quick start

Create a vault without cloning MirrorArc itself. The installable command requires Python 3.11+; uvx and pipx run will build an isolated environment when they can find a compatible Python. Before the package is published to PyPI, use the Git URL:

uvx --from git+https://github.com/cz1993/MirrorArc.git mirrorarc init --profile data-product ~/my-data-product

# or, with pipx:
pipx run --spec git+https://github.com/cz1993/MirrorArc.git mirrorarc init --profile data-product ~/my-data-product

Once MirrorArc is published to PyPI, the equivalent short forms are uvx mirrorarc init --profile data-product ~/my-data-product and pipx run mirrorarc init --profile data-product ~/my-data-product.

For repeated local use, install the console command once:

uv tool install git+https://github.com/cz1993/MirrorArc.git
mirrorarc --version

# or, with pipx:
pipx install git+https://github.com/cz1993/MirrorArc.git
mirrorarc --version

Then open the vault in Obsidian if you want a human UI, point your agent at it (CLAUDE.md routes all agents to _meta/agent-rules.md), and run the installed command from any folder:

mirrorarc --root ~/my-data-product doctor          # check dependencies and vault structure
mirrorarc --root ~/my-data-product plan            # inspect proposed mirror actions first
mirrorarc --root ~/my-data-product sync            # mirror Office files and configured repos
mirrorarc --root ~/my-data-product status          # review manifest-backed lifecycle state
mirrorarc --root ~/my-data-product sync --json     # machine-readable sync evidence for agents/pilots
mirrorarc --root ~/my-data-product status --json   # machine-readable lifecycle status
mirrorarc --root ~/my-data-product doctor --json   # machine-readable preflight report
mirrorarc --root ~/my-data-product conversion --guide   # read-only conversion spot-check + guide
mirrorarc --root ~/my-data-product conversion --init-results # private quality review scaffold
mirrorarc --root ~/my-data-product conversion --results _meta/conversion-quality-results.yml --require-reviewed # after filling scaffold
mirrorarc --root ~/my-data-product migration       # dry-run report for legacy/unknown folders
mirrorarc --root ~/my-data-product migration --runbook  # legacy folder move protocol
mirrorarc --root ~/my-data-product migration --normalize-frontmatter-domains --worksheet # review domain alias cleanup
mirrorarc --root ~/my-data-product recovery --worksheet # review manifest recovery actions
mirrorarc --root ~/my-data-product sandbox --source-root /path/to/original-documents
mirrorarc --root ~/my-data-product catalog         # generate CATALOG.md inventory gateway
mirrorarc --root ~/my-data-product catalog --html  # generate the interactive CATALOG.html explorer
mirrorarc --root ~/my-data-product catalog --html --include-content # local content-review portal
mirrorarc --root ~/my-data-product code doctor     # optional local repository-analysis readiness
mirrorarc --root ~/my-data-product code analyze --repo <repo-id> # bounded revision evidence
mirrorarc --root ~/my-data-product code status --json # current/stale/failed analysis state
mirrorarc --root ~/my-data-product m365            # Microsoft 365/Copilot handoff readiness
mirrorarc --root ~/my-data-product review --json   # summarize metadata-only human review decisions
mirrorarc --root ~/my-data-product overlap         # calibrate overlap thresholds without note bodies
mirrorarc --root ~/my-data-product pilot           # aggregate pilot evidence, no source content
mirrorarc --root ~/my-data-product pilot --worksheet    # redacted Markdown private-pilot summary
mirrorarc --root ~/my-data-product benchmark            # validate agent-readiness task pack, if present
mirrorarc --root ~/my-data-product benchmark --init-tasks    # create private task scaffold
mirrorarc --root ~/my-data-product benchmark --worksheet     # print private benchmark run sheet
mirrorarc --root ~/my-data-product benchmark --init-results  # create private result scaffold
mirrorarc --root ~/my-data-product benchmark --results _meta/agent-readiness-results.yml --require-prompt-safety # after scoring
# edit tools/repos.yml, then:
mirrorarc --root ~/my-data-product sync           # mirror configured GitHub repos too
mirrorarc --root ~/my-data-product lint           # health check

Run sandbox from a duplicated pilot vault, not the original document folder. It is read-only and checks copy-boundary, mirror isolation, manifest/recovery readiness, and basic backup posture without printing source paths or document text.

Use review after spot-checking mirrors, catalogs, or handoff reports. It appends metadata-only decisions to _meta/review-ledger.jsonl with artifact hashes, so later changes are reported as stale reviews instead of silently preserving old approvals.

The generated CATALOG.html is a self-contained Catalog Explorer. It opens on INDEX.md, offers separate relationship-map, document-metadata, rendered-document, and contextual Code views, preserves explicit original-to-mirror lineage, and keeps local context-pack exports metadata-only. Repository analysis runs before generation; the passive HTML never starts a process or fetches data. The safe default does not embed document bodies. Use catalog --html --include-content only for a local review copy; that opt-in mode embeds bounded Markdown and generated-mirror bodies, so the resulting HTML must be protected like the vault itself. The public GitHub Pages demo is built from the explicitly public, provenance-documented example and passes the no-data gate before deployment. Neither mode requires an application server or creates an evidence index.

Source checkout fallback:

git clone https://github.com/cz1993/MirrorArc.git mirrorarc && cd mirrorarc
python3.11 -m pip install -e .
mirrorarc --version
mirrorarc profile list
mirrorarc init --profile data-product ~/my-data-product
mirrorarc init --profile business-operations ~/my-business-vault
mirrorarc init --profile research-learning ~/my-research-vault
mirrorarc init --profile software-project ~/my-software-vault
mirrorarc init --profile blank ~/my-blank-vault
mirrorarc --root ~/my-data-product profile validate
mirrorarc --root ~/my-data-product profile diff 0.2.0
mirrorarc --root ~/my-data-product profile migrate --plan
mirrorarc --root ~/my-data-product profile migrate --write
mirrorarc --root ~/my-data-product profile views --check
mirrorarc --root ~/my-data-product migrate annotations --plan
mirrorarc --root ~/my-data-product plan

The packaged v1 profiles are data-product (the default), business-operations, research-learning, software-project, and blank. Each initializes from the installable package with profile-owned folders, _meta/profile.yml, generated scaffold docs, and only the templates declared by its contract.

Profile contract details: docs/PROFILE_SCHEMA.md.

Step-by-step: docs/quickstart.md.

Documentation

The whitepaper defines architectural principles; the current Trustworthy Context plan governs this local batch. Earlier implementation plans record historical evidence, not current acceptance.

  • Product contract — audience, workflow, outcomes, and non-goals.

  • Quickstart — demo-first setup and daily workflow.

  • Sync specification — authority, mirroring, manifests, and refresh behavior.

  • Security model — trust boundaries, prompt safety, and local-review rules.

  • Profile schema — configurable domains, note types, folders, and policies.

  • Methodology — L0/L1, relationships, L2, context, review, and promotion discipline.

  • Positioning — where MirrorArc fits relative to LLM wikis, RAG, PKM, and document-management tools.

Status

v0 - technical alpha. Source-preserving L1 mirrors, manifests, lifecycle state, audit logs, journaled changed-file materialization, reconciliation, full recovery sync, profiles, lint/safety guards, the portable catalog, stable projection identities, and the versioned evidence-backed relationship ledger work today.

Dependency invalidation, L2 knowledge views, dynamic/frozen context assembly, and optional revision-bound Repository Intelligence are implemented locally. The current Ontario example is retained as migration and regression evidence; do not interpret its curated Markdown count as the target MirrorArc model. This remains a technical alpha, and local proof is not hosted or production proof.

License

MirrorArc is licensed under the GNU Affero General Public License v3.0 or later for the open core, with a separate commercial license planned for enterprise/closed use and consulting. LICENSE contains the full AGPL-3.0 text; commercial terms and contribution policy still require owner/counsel finalization before accepting outside contributions. See LICENSING.md. "MirrorArc" is a trademark — see TRADEMARK.md.

Credits

The "LLM wiki" pattern (Andrej Karpathy), markitdown (Microsoft), and Obsidian. Interoperates with — rather than competes with — tools like basic-memory and Copilot for Obsidian.

docs/SYNC_SPEC.md

Sync Specification

Objective

MirrorArc sync must prove the core product promise: source files remain untouched while generated knowledge artifacts can be regenerated, audited, and reviewed.

Full sync is the implemented baseline and remains the recovery/verification path. Stage 1B adds journaled changed-file materialization as the normal steady-state path: event-identified candidate sources are fingerprinted, hashed only when needed, materialized through the same package-owned mirror logic, and reconciled against authoritative sources because watcher events are not authoritative.

Sync is also the dependency boundary for the knowledge-projection architecture. A successful source/L1 change must update stable artifact identity, refresh deterministic relationships, and record bounded invalidation for semantic relations, reviews, L2 views, and frozen-context freshness. Dynamic context definitions remain intact and resolve current evidence when next used. The portal consumes this shared state; it does not infer a parallel lifecycle.

The current runtime has the local journal/state foundation plus deterministic feed filtering, event coalescing, and cheap metadata fingerprint primitives, plus lease-protected event claims and interrupted-worker recovery. It also has a source-addressable Office materialization primitive that processes one vault-relative source through the existing Office mirror engine instead of creating a second mirror writer, plus deterministic file-stability settling before conversion and a lease-protected worker path for current-path Office events, plus idempotent journal replay that recovers interrupted processing events and optionally retries failed events under the same worker lease, plus explicit reconciliation that queues missed source/manifest events with metadata-first comparison and candidate-only hashing for safe move detection. mirrorarc sync --changed composes reconciliation and replay, while mirrorarc sync and mirrorarc sync --full preserve the full sync recovery path. mirrorarc watch --once runs the deterministic watch-start cycle: startup reconciliation, feed-event queueing, and journal replay. Synthetic journal benchmarks are reproducible with scripts/benchmark_journaled_materialization.py; benchmark output is a local validation artifact, not committed product documentation. mirrorarc watch --native provides optional watchdog-backed native event capture through the same feed/replay boundary.

Source Identity

Path-based mirror names are user-friendly but insufficient as the durable identity model. Office mirror sync maintains _meta/source-manifest.json with stable source IDs. Repo mirror sync maintains _meta/repo-manifest.json with stable repo IDs derived from configured repo/note identity.

The future change journal may store paths, fingerprints, hashes, event states, and checkpoints, but it does not replace manifest-owned source identity or lifecycle state.

Each source record should include:

  • stable source ID;

  • stable projection ID when a separate L1 projection is required;

  • current source path;

  • previous source paths;

  • stable L1 projection identity;

  • mirror path;

  • source file size;

  • source hash;

  • normalized extracted-content hash;

  • converter name and version;

  • sync configuration version;

  • lifecycle state;

  • lifecycle contract path and schema version;

  • last successful sync timestamp;

  • warnings, omissions, and errors.

One active opaque source identity normally has exactly one active L1 projection identity. Native Markdown/plain text is registered as directly readable unless profile policy requires isolation or an immutable projection. Moves preserve identity when unambiguous. Sibling/orphan projections and manual generated-body drift are blocking integrity failures.

An applied journal event refreshes the affected source/projection relationship subgraph and walks reverse dependency edges with explicit depth/node bounds and cycle protection. Hash changes or deletes invalidate only semantic edges and downstream artifacts that depend on the prior hash. Move-only events keep semantic identity when the evidence hash still resolves. Reviewed views are preserved and marked stale, generated views are queued for replacement, frozen contexts become stale, and dynamic definitions remain intact. Each invalidation records the journal sequence, dependency identity, prior/current hashes, reason, and timestamp. Reconciliation compares manifest hashes with ledger state to repair a missed invalidation.

Current implementation status:

  • implemented for Office mirrors: stable source and L1 projection IDs, current/previous source paths, mirror path, source hash/size, converter/config version, lifecycle state, warnings/errors, source-missing marking, non-mutating plan/status reports, sensitive-name warnings, duplicate-byte warnings, and format-specific conversion-quality warnings, plus contract-backed lifecycle next-action guidance in plan/status output, and post-conversion source hash checks that abort mirror writes if the source changes while conversion is running, plus mirror-root-change conflict detection when the old generated mirror still exists, plus moved-source review blocking while the previous generated mirror still exists, plus update-path annotation-migration blocking tests, plus converter-failure and mirror-write-failure recovery tests that preserve the previous mirror and allow later clean regeneration, plus a read-only conversion spot-check report that turns manifest warnings, errors, lifecycle states, formats, and source/mirror existence into an operator review list, plus blocking lint checks for generated Office mirrors whose source bytes changed or whose manifest lifecycle state is no longer current, plus Office sync/status detection and repair for managed source frontmatter metadata drift, plus conservative .xlsx mirror cleanup for obvious NaN placeholders and empty Unnamed:* table columns, plus ambiguous same-hash move detection that blocks sync as conflict when multiple missing manifest records could match one new source, plus profile-first source-domain routing with _meta/domain-map.yml retained as a legacy alias layer;

  • implemented for Stage 1B Office materialization: mirrorarc.changes.materialize can process one vault-relative Office source through the same Office mirror planning/sync path, preserving source bytes, profile-defined mirror roots, manifest updates, audit events, and the existing source-change-during-conversion safeguard;

  • implemented for Stage 1B candidate stability: mirrorarc.changes.stability waits for the cheap metadata fingerprint to remain unchanged for a configurable settle interval, supports a bounded timeout, and can be injected into source-addressable materialization so unstable sources skip conversion and writes before worker integration;

  • implemented for Stage 1B current-path Office worker processing: mirrorarc.changes.worker acquires the workspace lease, claims queued/ready journal events, materializes supported current-path Office sources through mirrorarc.changes.materialize, records source ID/hash on finished journal rows, and finishes unsupported/no-current-path events as review-required or materialization errors as failed;

  • implemented for Stage 1B deleted-source lifecycle replay: manifest-backed deleted journal events with a previous source path are applied through mirrorarc.changes.materialize.materialize_office_delete, marking the Office source manifest record source_missing, retaining the generated mirror for review, appending audit evidence, and finishing the journal event as applied; deleted events without matching manifest evidence still require review;

  • implemented for Stage 1B move/recreate replay: sync --changed blocks unambiguous moved-source materialization while the previous generated mirror still exists, then reconciliation requeues the resolved source_moved record after that previous mirror is removed so replay can create the new mirror with the same source ID; delete/recreate of the same source path queues a metadata update and returns the manifest record to clean;

  • implemented for Stage 1B journal replay: mirrorarc.changes.replay and mirrorarc journal replay recover interrupted processing events, optionally requeue failed events only when --retry-failed is requested, process claimable work through the existing worker/materialization path under one lease, and expose bounded/JSON replay results;

  • implemented for Stage 1B explicit reconciliation: mirrorarc.changes.reconcile and mirrorarc reconcile discover current Office/PDF candidates through the existing mirror scanner, compare source metadata against _meta/source-manifest.json, queue missed creates, metadata-changed updates, missing-source deletes, and same-hash moves into the local journal, requeue resolved source_moved records after the previous generated mirror has been removed, avoid duplicate unresolved events, update last_reconciliation_at, and full-hash only suspicious new paths that can match missing manifest records;

  • implemented for Stage 1B changed sync: mirrorarc.changes.changed_sync and mirrorarc sync --changed run explicit reconciliation and then replay claimable journal work through the existing worker/materialization path; plain mirrorarc sync remains the existing full Office/repo sync path, and mirrorarc sync --full names that recovery path explicitly;

  • implemented for Stage 1B watch startup orchestration: mirrorarc.changes.watch and mirrorarc watch --once run startup reconciliation, queue normalized/coalesced feed events through the existing change-feed interface, and replay claimable journal work under the existing worker lease; plain mirrorarc watch exits with mode guidance;

  • implemented for Stage 1B optional native watch capture: mirrorarc.changes.native_watch maps watchdog events to advisory ObservedChange records, watches only existing profile/legacy content roots, ignores directories and outside paths, buffers events under a thread-safe handler, and mirrorarc watch --native flushes captured events through the same reconciliation, feed, coalescing, lease, replay, and source-addressable materialization path; the optional mirrorarc[watch] extra supplies the watchdog dependency without changing default installs;

  • implemented for Stage 1B benchmark tooling: scripts/benchmark_journaled_materialization.py builds a temporary synthetic vault with 1,000 source records, replays a known-path event batch with one save storm, one move, and one deletion, and records paths enumerated, source bodies read, bytes hashed, converter invocations, event counts, elapsed time, and peak memory in a caller-selected local report;

  • implemented for repo mirrors: stable repo IDs, configured/resolved repo, note path, local-tree or remote HEAD hash, lifecycle state, warnings/errors, non-mutating plan/status reports, and generated-region manual-edit detection, plus contract-backed lifecycle next-action guidance in plan/status output, plus repo stub-to-populated tests that preserve configured generated frontmatter without carrying legacy in-mirror notes forward, plus repo-note write-failure recovery tests that preserve the previous note and allow later clean regeneration, plus config validation and blocking lint checks for duplicate configured repo note targets, plus blocking lint checks for configured repo entries whose expected generated note is missing or unmanaged, and for generated repo mirrors whose manifest lifecycle state is no longer current, whose frontmatter repo or commit drifts from the manifest, or whose local source tree changed, plus repo sync/status detection and repair for managed repo frontmatter identity drift, plus repo_unconfigured lifecycle reporting and lint blocking when a previously synced repo mirror is retained after its tools/repos.yml entry is removed;

  • implemented for mirror-annotation migration: mirrorarc migrate annotations --plan reports above-sentinel source/repo mirror annotations without printing their text, and --write stores them under _meta/mirror-annotations/source/<source_id>.md or _meta/mirror-annotations/repo/<repo_id>.md while leaving original sources and generated mirrors untouched; fresh Office and repo mirrors use machine-owned headers without a curated ## Notes region; sync blocks unmigrated above-sentinel annotations as force-blocking review work and, after a matching sidecar exists, resets regenerated mirrors to a machine-owned header instead of preserving the migrated human annotation region in the mirror; lint blocks generated source/repo mirrors with unmigrated above-sentinel annotations so release gates force sidecar migration;

  • implemented for lifecycle operator semantics: the template now includes _meta/lifecycle-states.yml, a machine-readable Office/repo state contract with entry conditions, user-visible explanations, permitted next actions, exit conditions, and manifest-state markers; sync plan/status guidance reads this contract, source/repo manifest records and audit events identify the contract path/schema version that governed the lifecycle state, mirrorarc doctor validates the contract during preflight, and tests ensure sync/report states remain covered;

  • partially implemented: full move/rename UX beyond unique hash matching and ambiguous-move conflict detection;

  • not complete: full rename/move UX, rollback automation, automated conversion-quality scoring beyond private operator-entered result packs, and exhaustive conflict-resolution flows.

Human review decisions are recorded outside generated artifacts in _meta/review-ledger.jsonl. The ledger stores artifact paths, hashes, reviewer/status fields, and short metadata notes. It does not change source files, mirror bodies, manifests, or lifecycle states, but it lets operators detect when a prior approval is stale because the reviewed artifact hash changed.

Lifecycle States

Minimum states:

  • planned

  • clean

  • source_changed

  • stale

  • regenerated

  • reviewed

  • unsupported

  • source_missing

  • source_moved

  • conflict

  • manual_modification

  • converter_changed

  • error

Every release-ready state must have:

  • entry condition;

  • user-visible explanation;

  • permitted next actions;

  • exit condition.

The authoritative machine-readable contract lives in _meta/lifecycle-states.yml in every initialized vault. Sync plan/status guidance reads this contract and falls back to local concise guidance only when a nonstandard or damaged vault is missing the contract. New lifecycle states must be added to the contract and covered by tests before release.

Rename, Move, and Delete Handling

  • Rename/move should be detected by stable source ID and source hash where possible, not treated only as delete plus create.

  • If a new source's bytes match multiple missing manifest records, sync must not choose a source history automatically. It should report conflict with the candidate paths so an operator can choose the correct history or treat the file as a deliberate duplicate/new source.

  • If multiple non-synthetic manifest records claim the same current source path, sync must report conflict with the duplicate source IDs rather than choosing one manifest record silently.

  • When a source move changes the generated mirror path, sync should not create the new mirror while the previous generated mirror still exists. The operator must preserve, move, archive, or remove the old mirror first so governed dependencies are not stranded and duplicate generated mirrors are not created.

  • Source deletion should not delete mirrors automatically. It should mark the manifest record as source_missing and surface a review action.

  • Repo config removal should not delete repo mirrors automatically. It should mark the manifest record as repo_unconfigured and surface a review action so the operator can restore the config entry, archive the mirror, or deliberately remove retired manifest state.

  • Mirror path changes should be planned before writes and reported after writes.

Conflict Rules

Sync must flag conflict when:

  • the generated region was manually edited;

  • a mirror exists at the target path but does not belong to the source ID;

  • the configured mirror root changes and an old mirror still exists;

  • a source's bytes match multiple missing manifest records and the correct move history is ambiguous;

  • multiple non-synthetic manifest records claim the same current source path;

  • source and reviewed derived-output changes require human reconciliation;

  • a previous sync failed after partial output.

Transactional Writes

Mirror writes are atomic in the current Office and repo sync tools:

  • write to a temporary file in the target directory;

  • fsync or equivalent where practical;

  • replace the prior mirror only after successful conversion and frontmatter generation;

  • preserve previous mirror when conversion fails.

  • preserve previous mirror when the mirror write fails before replacement.

  • preserve previous mirror when the source bytes change during conversion, because the extracted text can no longer be tied safely to the planned source hash.

Sync should never leave half-written mirrors as successful output. If an OS/process interruption leaves a hidden atomic temp file behind, mirrorarc recovery must surface it as temp:interrupted_write so the operator can rerun status/sync and remove the temp file after backup review.

Idempotency

A second sync with unchanged source bytes, unchanged converter/config version, and unchanged curated region must produce no content diff. The append-only _meta/sync-audit.jsonl log may receive new events; idempotency assertions should compare stable generated mirrors, repo notes, and manifests separately from the audit log.

For Stage 1B, event replay must also be idempotent: duplicate or interrupted journal events must not produce duplicate successful materialization or corrupt manifests/mirrors.

Current idempotency regression coverage includes:

  • Office mirrors;

  • the Office source manifest;

  • repo mirrors;

  • empty repo config;

  • unsupported source;

  • source lock file skip;

  • alias-to-canonical mirror routing.

Rollback and Audit

Every generated change should be explainable from:

  • manifest before/after state;

  • source hash;

  • converter/config version;

  • sync log entry;

  • machine-readable _meta/sync-audit.jsonl event;

  • output path.

  • optional _meta/review-ledger.jsonl decision tied to the generated artifact hash.

  • relationship, dependency, view, and context invalidation tied to journal sequence and source hash.

Audit events must include the generated artifact path, lifecycle state, lifecycle contract path/schema version when available, status, and structured warnings / errors copied from the manifest record after the sync attempt. They must not include raw source content, extracted document text, repo document bodies, secrets, or tokens.

Rollback guidance lives in docs/RECOVERY.md and must be tested before a public release.

Derived-State Rebuild

.mirrorarc/state.sqlite stores the journal plus versioned relationship, dependency, review, L2, and context metadata. It stores no authoritative source bodies. A full derived-state rebuild must be deterministic when semantic proposals are disabled: recreate state from current sources, manifests, profile rules, governed view definitions, and accepted semantic records; then compare current hashes and counts. Reviewed output files and frozen packs are preserved as governed artifacts and reattached by identity/hash rather than silently replaced.

Original Source Integrity

Tests must prove that sync does not mutate original source bytes.

The release gate is byte-for-byte equality of source files before and after sync for representative supported formats and skipped/unsupported files. Example-vault regeneration tests should snapshot representative source payloads before plan, dry-run, sync, status, and lint operations, then assert those payloads are unchanged. They should also run a second sync and assert stable generated mirrors, repo notes, and manifests remain byte-for-byte unchanged when inputs did not change.

439 files · 404 not served here

Advantages

Limitations

  • No ZIP download: install it from its source (a rule-derived fact from the brain’s files)

Security

Risk level: Low risk

This brain runs

Git hooks (1)

  • .githooks/pre-commit (pre-commit): runs on git pre-commit

Agent instructions that run or fetch something (2)

  • AGENTS.md: Operator direction (2026-09-18): project development and validation run locally.
  • AGENTS.md: Every behavior change: run the package-owned vault lint through the example's tool shim, run the test suite, and update `CHANGELOG.md` and relevant docs.

Install scripts (1)

  • scripts/init.sh: sh scripts/init.sh
  • Low risk: scripts-present (.githooks/pre-commit)
  • Low risk: scripts-present (scripts/init.sh)

Attribution and licence

First committed by
GitHub avatar of cz1993cz1993
Declared authors
  • cz1993 (declared in pyproject.toml)
Publisher
Ci Zhu (Community)
Source
cz1993/MirrorArc · commit 1df13eb
Licence
GNU Affero General Public License v3.0 only (files served here)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
cz1993

AutoGineer is not affiliated with or endorsed by cz1993.

Install from source

404 of its files are not served here, so there is no ZIP. cz1993/MirrorArc at 1df13eb