Engrava

Configuration

engrava supports YAML-based configuration for production deployments. This document covers all configuration options.

Quick-start profiles

The base install (pip install engrava) is light: five direct dependencies, no torch. Whether a machine-learning model ever loads into your process — and how much it costs to get there — is decided entirely by which embeddings extra you install and which provider you point engrava.yaml at. Nothing below changes an engine default; each profile is an ordinary engrava.yaml, shipped as a real file under examples/ so you can copy it as-is.

Pick one before your first pip install, not after:

lexical — no embedding provider

database:
  path: "./engrava.db"
  wal_mode: true

No embeddings: section at all — that is the whole profile. FTS5/BM25 keyword search, the edge graph, and MindQL all work exactly as usual, and the journal is available (off by default, same as every profile). Semantic search is inert, not degraded: with no provider configured, search_hybrid() / recall() never run a vector arm — verified against a live store: HybridSearchResult.backends_used comes back as {'fts5', 'priority'}, never containing "vector", and a query that only a vector arm could answer returns nothing, silently, rather than raising. No embeddings dependency is installed and no model is ever downloaded — the store-open cost is exactly the base install's, not claimed to be zero or "instant".

network — Ollama, model out of process

database:
  path: "./engrava.db"
  wal_mode: true

embeddings:
  provider: ollama
  model: nomic-embed-text
  base_url: "http://localhost:11434"
  auto_embed: true

extensions:
  vector:
    backend: numpy
    dimension: 768   # nomic-embed-text's embedding dimension

Needs three things, none of which is a Python dependency of engrava beyond one small package:

1. Ollama installed and running, reachable at base_url.

2. The model pulled into Ollama once: ollama pull nomic-embed-text.

3. The engrava[embeddings-ollama] extra — it declares one direct dependency, httpx (a ~70 KB wheel on PyPI), which brings in its own dependencies in turn (for example httpcore, h11); that is the sole cost this profile adds to the Python environment. No model file ever touches this process.

Verified against a live Ollama server: store.recall() against this exact profile returns backends_used = {'fts5', 'priority', 'vector'} — the vector arm ran, driven by a real HTTP call to Ollama, not a local model load.

An OpenAI-compatible endpoint (provider: openai-compatible, its own base_url, and a real api_key) is a variant of this same profile, not a separate one — the model still runs outside this process, on the same "one HTTP dependency" cost — it is documented here as a variant rather than as its own profile because a profile with two backends is not one reproducible thing.

What you give up: this profile needs Ollama reachable at query time. There is no offline fallback — if Ollama is down, embedding calls fail; the local profile trades that for a large upfront download instead.

local — sentence-transformers, offline after warm-up

database:
  path: "./engrava.db"
  wal_mode: true

embeddings:
  provider: sentence-transformer
  model: all-MiniLM-L6-v2
  auto_embed: true

extensions:
  vector:
    backend: numpy
    dimension: 384

Needs the engrava[embeddings-local] extra, which pulls sentence-transformers and torch. Real, measured numbers, not general knowledge about these libraries:

• torch 2.14.1's current PyPI Linux x86_64 wheel for Python 3.11 is 554.6 MB, measured from the PyPI package index on 2026-10-16. On Linux, that wheel's own dependency metadata adds the NVIDIA CUDA libraries too: 19 more distributions, 2,445.4 MB together. Installing the extra therefore pulls about 3,000 MB in all before sentence-transformers's own dependencies, most of it usable only with an NVIDIA GPU. The macOS arm64 wheel (127.3 MB) and the Windows amd64 wheel (124.1 MB) carry none of that. Without an NVIDIA GPU, the CPU-only Linux build is 196.2 MB and pulls no CUDA dependency at all; see "No NVIDIA GPU: install the CPU-only build" below.

• sentence-transformers itself is a sub-1 MB wheel, but it brings in transformers, tokenizers, and huggingface_hub on top, so the extra's total download exceeds torch alone.

• The model this profile names, all-MiniLM-L6-v2, is a further, separate download — measured 88 MB on disk under ~/.cache/huggingface/hub after the first call that uses it.

What you give up, and what "offline" actually requires. After the dependency install and one warm model load, later embed() calls run in-process with no network call — but only once two environment variables are set: HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. This was verified directly, not assumed: against this exact profile, with the model already cached and the network made unreachable, loading the model still raised with those two variables unset — the underlying transformers library issues a HEAD request checking for a PEFT adapter config on every load, cache or not, and does not fall back to the cache silently when that request fails. With both variables set, the identical load and query succeeded with no network reachable at all. Set them before your process starts if you rely on this profile being offline.

What you give up versus network: a large one-time download (and the two offline environment variables above) in exchange for never needing a reachable service again — the inverse trade from network, which needs no local download but needs Ollama reachable at every query.

No NVIDIA GPU: install the CPU-only build

Install the CPU-only build of torch first, in the same environment the next command installs into. This only works in a fresh virtual environment: pip treats an existing torch, even the default CUDA build, as already satisfying the extra's requirement and leaves it as is, and uninstalling torch alone leaves its CUDA libraries behind.

# Run in a fresh virtual environment, before either package is installed.
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install 'engrava[embeddings-local]'

The second command keeps the CPU-only torch the first one just installed: it already satisfies the extra's own requirement (torch>=2.0.0; 2.14.1+cpu satisfies it), so nothing pulls the default PyPI build or its CUDA dependencies in behind it.

With an NVIDIA GPU, skip the first command: keep the default PyPI build, or pick the index that matches your GPU and driver from pytorch.org's install selector.

The same holds for engrava-mcp[local], whose own metadata declares local = ["engrava[embeddings-local]>=0.7,<0.8"] (engrava-mcp's pyproject.toml, read 2026-10-16): install the CPU-only torch first, then pip install 'engrava-mcp[local]' in the same environment.

Configuration File

Create a engrava.yaml file:

database:
  path: "./engrava.db"
  wal_mode: true

search:
  default_fts_weight: 0.30
  default_vector_weight: 0.55
  default_recency_weight: 0.10
  default_priority_weight: 0.05
  default_graph_weight: 0.00       # opt-in graph signal (0.0 = OFF by default)
  recency_half_life: 50
  recency_now_half_life_seconds: 604800  # transaction-time axis: 7 days
  priority_boost_p1: 1.0
  priority_boost_p2: 0.6
  priority_boost_p3: 0.3
  priority_boost_p4: 0.0
  graph_edge_decay: 0.5            # 1-hop distance penalty
  max_neighbors_per_candidate: 5   # safety cap
  reflection_boost: 1.0            # REFLECTION score multiplier (1.0 = neutral)
  reflection_topk_cap: 0.3         # max fraction of top-K that may be REFLECTIONs
  collapse_pool_factor: 4          # arm-budget widening when collapse_key is set
  vec0_overfetch_factor: 4         # sqlite-vec over-fetch before the live-row trim

extensions:
  vector:
    backend: numpy
    dimension: 384

  dreaming:
    enabled: true
    schedule_every_n_cycles: 100
    promote_threshold: 0.7
    candidates_limit: 200
    gates:
      min_confirmations: 2
      min_age_cycles: 1
      max_promoted_per_run: 20
      allow_zero_confirmation: true

derive:
  enabled: false
  on_error: log
  max_derived_per_source: 32

Loading Configuration

from engrava import load_config, SqliteEngravaCore

config = load_config("engrava.yaml")

async with await SqliteEngravaCore.from_config("engrava.yaml") as store:
    thought = await store.get_thought("abc")

Unknown keys fail fast. The YAML loader raises ConfigError for unrecognised keys at the top level and inside every statically shaped section, so a misspelling such as promtoe_threshold cannot silently retain a default. Dynamic mapping keys remain supported where the contract permits them, such as service names and custom Dreaming signal names.

Full Factory Method

from engrava.config import load_config, resolve_embedding_provider

config = load_config("engrava.yaml")
# resolve_embedding_provider takes the EmbeddingConfig, i.e. config.embeddings
provider = resolve_embedding_provider(config.embeddings)

Configuration Reference

database

search

Controls hybrid search behavior (FTS5 + vector + recency + priority).

All 22 SearchConfig fields are settable here; every one has a default, so the whole section is optional.

Signal weights and per-priority boosts:

Reflection handling:

Graph-expansion (candidate-pool widening via CONSOLIDATED_FROM edges):

Bounded pool multipliers:

Weights are redistributed proportionally when a signal is unavailable (e.g. no current_cycle → recency skipped). Set any weight to 0.0 to disable that signal entirely.

The graph ranking signal is off by default. default_graph_weight is 0.0, so a default store runs no graph ranking queries. This is separate from graph_expansion_enabled (default true), which controls candidate-pool widening over CONSOLIDATED_FROM edges — the ranking graph signal stays off until you give default_graph_weight (or a per-call graph_weight) a non-zero value.

See search.md for the full 5-signal ranking model.

Silent-behaviour footguns

A few defaults keep a signal or a code path quiet unless you opt in, while conflicting explicit recency arguments fail loudly. The quiet default paths do less than you might expect, so they are worth knowing before you rely on the behaviour.

• Recency needs exactly one resolved reference. search_hybrid() / recall() resolve the two recency axes in this order:

  1. Supplying both an explicit current_cycle and recency_now raises RecencyModeConflictError; the two clocks are never combined.

  2. An explicit recency_now with no explicit current_cycle selects transaction-time recency and suppresses the passive cycle_provider.

  3. Otherwise an explicit current_cycle (including 0) selects cognitive-cycle recency; if it is absent, a configured cycle_provider supplies the cycle.

  4. With neither explicit reference and no provider, recency is inactive and its weight is redistributed to the other signals.

  The final no-reference/no-provider case does not raise. On a sufficiently large store, recall() emits one DEBUG breadcrumb per store instance.

• default_graph_weight=0.0 ⇒ the graph signal is off. As noted above, a default store runs no graph ranking queries. Give the weight (or a per-call graph_weight=) a non-zero value to opt in.

• Manual SqliteEngravaCore(...) construction requires ensure_schema(). The from_config() / EngravaManager factories create tables, indexes, the FTS5 virtual table and enable foreign keys for you. If you instead construct SqliteEngravaCore(conn, ...) directly against a raw connection, you must await store.ensure_schema() once before use — otherwise the tables (and FK enforcement) are absent and the first query fails. ensure_schema() is idempotent, so calling it on an already-migrated database is a no-op.

• sqlite-vec falls back to numpy silently. With extensions.vector.backend: sqlite-vec, the vec0 backend is used only if the optional sqlite-vec package is importable. If it is absent (or fails to load), the store logs a WARNING and transparently falls back to the brute-force numpy cosine arm — results are identical, but you do not get the vec0 index. Searches keep working, so a missing extension is easy to miss; install the sqlite-vec extra (and check for the load warning) if you intend to run on the native index.

  engrava gc is the one path that does not fall back — it refuses. If the database already carries an embedding_vec table from an earlier run with the extra, a gc pass that is about to physically delete stops before deleting anything and exits 1 (Install 'engrava[vec]' and retry), because removing the rows without removing their vectors would strand those vectors in the index. --dry-run is never refused, and neither is a run with nothing to delete — but note that gc --expired under the default ttl.strategy: archive stops after archiving only when it actually archived something, and with no expired rows falls through to the archived-collection pass, which is refused when there are archived rows to collect. See CLI reference → gc.

embeddings

Embedding provider configuration. (The YAML key is embeddings, plural.) The vector dimension lives under extensions.vector.dimension, not here.

Asymmetric prefixes are opt-in and for instruction-tuned models only. Models like E5, BGE, GTE, and Ollama's nomic-embed-text are trained with mandatory role instructions ("query: " on the query, "passage: " on the document) and retrieve worse when run without them. Leave both prefixes empty (the default) for OpenAI and other symmetric models — the empty path is byte-identical to prior behaviour, and no existing store needs migrating. See the embeddings guide for the full re-embed policy.

dreaming

Memory consolidation configuration. Setting enabled: true wires store.consolidate() when the store is created with from_config(); it does not start a background task. DreamingExtension.is_due() and run_if_due() apply schedule_every_n_cycles for callers that drive a cycle loop. run_consolidation() and store.consolidate() remain explicit, unconditional operations.

Default signals: recency 0.25, staleness 0.20, confirmation 0.20, confidence 0.15, frequency 0.20, action_outcome 0.15. These are relative weights and intentionally sum to 1.15. A configured signal name must be a built-in signal or be supplied through custom_signals when constructing DreamingExtension; otherwise extension construction raises ValueError.

dreaming.gates

Promotion, clustering, and cluster-quality gates.

With quality gating enabled, duplicate-content clusters and clusters flagged by the lightweight English token-pair contradiction heuristic are also rejected; those checks have no numeric YAML knob. A cluster must survive duplicate, persona, contradiction, cohesion, external-source, named-entity, and meaningful-keyphrase checks before a REFLECTION is written.

dreaming.edges

Edge creation from dreaming. Promoted thoughts create ASSOCIATED edges to their nearest neighbours.

The REFLECTION payload is structural JSON schema v2 containing legacy member_ids, keywords, and cluster_hash fields plus type, version, member_count, cluster_algorithm, created_at, top_keyphrases, member_excerpts, temporal_span, and named_entities. No LLM is called. See Dreaming for execution order and interpretation.

services

Multi-service isolation (one database file per named service, stored under a shared data_dir as <name>.db).

Each service entry under configs supports a single optional override (there is no per-service db_path — the file is derived as <data_dir>/<name>.db):

The restore CLI passes the top-level embeddings section as default_embeddings only when --re-embed is set; a plain restore (no re-embed) constructs EngravaManager with no default_embeddings fallback. A per-service override at services.configs.<name>.embeddings always takes precedence when present, and EngravaManager falls back to default_embeddings for a service that does not declare its own provider. See Asymmetric prefixes for instruction-tuned models for the --re-embed path itself.

journal

The hash-chain audit trail. Off by default. See Audit Trail.

journal:
  enabled: true
  verify_on_open: true

ttl

Time-to-live / auto-expiry of thoughts. See the data-lifecycle recipes.

ttl:
  strategy: archive          # or "delete"
  check_every_n_operations: 100
  default_ttl_seconds: 2592000   # 30 days

hygiene_policy

The rule-based Memory Hygiene forgetting loop — built-in scoring makes no LLM calls; a configured custom on_retrieve or decay_function hook can — archives cold, low-value thoughts and, separately opt-in, garbage-collects them after a restore window. Absent or enabled: false (the default) ⇒ the loop never runs and no read/write path changes. Distinct from ttl: TTL expires by wall-clock expires_at; hygiene forgets by a signal-derived keep-score.

Default signal_weights: recency 0.30, frequency 0.25, confirmation 0.20, confidence 0.15, staleness 0.10. (confidence contributes to the keep-score but is not protection.)

hygiene_policy:
  enabled: false                 # OFF by default — the whole loop is opt-in
  eviction_threshold: 0.20
  protected_priorities: ["P1"]
  signal_weights:
    recency: 0.30
    frequency: 0.25
    confirmation: 0.20
    confidence: 0.15
    staleness: 0.10
  check_every_n_cycles: 1
  max_evictions_per_run: 100
  auto_gc_enabled: false             # Stage 2 physical delete is separately opt-in
  gc_min_archive_age_cycles: 10      # cycle restore window before a GC is eligible
  gc_restore_window_seconds: 2592000 # AND a 30-day wall-clock window; 0 disables it
  min_inactivity_age_seconds: 604800 # cold-start guard: 7 days untouched; 0 disables
  dry_run: false                     # set true to preview without mutating

Garbage collection here is cognitive hygiene, not compliance deletion — it is best-effort and offers no deletion guarantee. See Memory Hygiene and Data lifecycle for the full mechanics.

Cold-start archive gates. A below-threshold score is not sufficient for archival. The thought must also be inactive for at least min_inactivity_age_seconds, measured from COALESCE(last_accessed_at, updated_at, created_at). A row with no known contact time fails closed and is kept. At the run level, archival is also suppressed unless the candidate pool has at least one active usage-history signal: frequency (with access tracking enabled), confirmation, or action outcome. These two gates prevent cycle/ingest order alone from classifying fresh imports as disposable.

Memory Hygiene's built-in scoring does not call an LLM (a configured on_retrieve or decay_function hook can). The fixed store, full configuration, current_cycle, and now (and deterministic custom hooks) fix the inactivity gate and the wall-clock GC window for the run, and the selection is reproducible on those boundaries alone. Calling run_hygiene() without now= reads the current UTC wall time once for those boundaries. Candidate collection is separate: it pages through list_thoughts(), whose own default expiry filter reads real wall-clock time on every call regardless of now, so a thought's expiry crossing during a long pass can still change which candidates are collected. Pass a fixed timezone-aware datetime as now= when replaying or benchmarking a selection.

ingest

Ingest-layer behaviour (content-hash deduplication).

Note: this flag advises ingest-layer callers; the persistence-layer create_thought still defaults to deduplicate=False — see Recipes → Deduplicate repeated facts.

derive

Controls the optional derived-records extension seam. The section only enables and bounds automatic production; a hooks object that implements DerivedRecordProducerProtocol must also be configured. With no producer, the seam is inert even when enabled. See Extension hooks.

derive:
  enabled: true
  on_error: log               # or "raise"
  max_derived_per_source: 32

hooks

Wire a custom EngravaHooksProtocol implementation by dotted path. See Extensions.

hooks:
  class: "my_package.hooks.MyHooks"

The path is split on the final dot (module.path + ClassName) — this is a plain dotted path, not the module.path:ATTRIBUTE colon form used by manifests.paths below.

manifests

Load extension manifests (their hooks + schema migrations). Accepts a plain list of dotted paths, or a mapping with discover / paths. See Extensions.

# list form
manifests:
  - "my_plugin.manifest:MANIFEST"

# or mapping form
manifests:
  discover: true
  paths:
    - "my_plugin.manifest:MANIFEST"

The metrics: section (latency window size, enable/disable) is documented in Observability.

Environment Variables

Both are read by the engrava CLI only (library callers pass paths explicitly to load_config / SqliteEngravaCore).

Multi-Service Usage

from engrava import EngravaManager, load_config

config = load_config("engrava.yaml")

async with await EngravaManager.from_config(config.services) as mgr:
    store = await mgr.get_store("main")
    # Use store normally...

Passing config.services (a bare ServicesConfig) wires only the per-service embeddings override, the PRAGMAs, and the vector backend — every store this builds behaves exactly like one from an older engrava release. To apply the rest of engrava.yaml to every service as well — the journal, hygiene policy, TTL, hooks, dreaming, and require_embedding — pass the whole config instead: EngravaManager.from_config(config). Either form requires config.services to be set; passing config additionally carries everything else config holds through to each service's store, the same way a single-database SqliteEngravaCore.from_config(...) open already does.

See the CLI --service flag for command-line multi-service access.