QMD - Query Markup Documents

Changelog

Unreleased

Fixed

• qmd doctor selects vector sample identities before loading document bodies, avoiding excessive SQLite memory use on large indexes with duplicate paths. #978 (thanks @naveenspark)

• Vector diagnostics match passages by their saved character position, avoiding false mismatches when earlier chunks change the sequence numbering. #978 (thanks @naveenspark)

Added

• Added Oxlint lint fence.

• Document metadata and metadata filtering. Markdown documents can opt into typed metadata through a namespaced frontmatter block (qmd.metadata with strings, numbers, booleans, or flat homogeneous arrays), and every search surface — CLI search/vsearch/query via --filter <json>, the SDK's filter option on search()/searchLex()/searchVector(), the MCP query tool, and HTTP POST /query and /search — accepts one shared recursive filter AST discriminated by operator: and/or/not logical groups, eq/ne/gt/gte/lt/lte comparisons, in/nin/all membership, contains/prefix/suffix text matching, type for the stored type of a key's values, and exists presence, with an optional caseInsensitive flag on any condition whose value is a string (ASCII folding). Every returned result satisfies the filter (applied before RRF fusion and reranking); like collection filtering, highly selective filters remain best-effort for top-K completeness. Frontmatter stays ordinary searchable content — no chunking, embedding, snippet, or line-number changes — and documents without qmd.metadata behave exactly as before. JSON/SDK/MCP/HTTP results now include each document's indexed metadata, and qmd status reports how many documents still need metadata extraction (a normal qmd update backfills existing indexes).

• Metadata discovery. Filtering is only useful when the caller knows what to filter on, so every surface now reports the metadata keys, types, and value counts already in the index — read from the existing metadata tables, with no re-indexing. The CLI adds qmd collection metadata [name...] with --filter <json> to count only documents matching a filter (same AST as search) and --match <json> to select which metadata entries are reported (the same AST evaluated per entry, where a condition's field is the entry's key or value, so one grammar covers a single key, a family of keys, reverse lookup of a value, typed thresholds, and any and/or/not composition), plus --key-limit/--key-offset/--all-keys and --value-limit/--value-offset/--all-values to page both windows, and --sort count|value and --min-count <n> to order and trim values. Every windowed list reports its exact remainder, counts are documents rather than values, numbers report min/median/max, keys whose documents disagree on type are split per type with their own counts, and with --filter every coverage count is measured against the documents the filter admits. qmd collection list names each collection's top keys, qmd collection show details them with a value preview, and qmd status summarizes coverage. The SDK adds listMetadata(options) (the same options, spelled keyLimit, valueLimit, and so on, with MetadataOptionError for a value outside its domain and MetadataBindingBudgetError for a filter and match that together bind too many SQL parameters), the MCP server adds a metadata tool and lists each collection's most covered keys in status, and the HTTP server adds POST /metadata. MetadataFilter and MetadataMatch are both MetadataPredicate<Condition>, one recursive grammar over the conditions a document or a metadata entry admits.

Fixed

• Filtered vector search binds candidate IDs as one JSON list, so a parser-valid metadata filter cannot exhaust Node's SQL variable limit during document lookup. Applies to both exact scans and the capped global fallback.

• Collection- and metadata-scoped BM25 search (qmd search -c, the lex leg of qmd query, MCP, SDK searchLex) no longer returns false-empty or incomplete results when stronger matches outside the scope fill the old limit * 10 candidate window (#922). The scope is now applied to the full FTS5 match set, materialized once, so search -c <collection> is exact for common terms; unscoped search keeps its early-terminating plan. A search over several collections, such as the default ones, runs one keyword query instead of one per collection, which made keyword search about 12x faster over 13 collections (thanks @brettdavies). Builds on the approach in #918 (thanks @fxstein). #953 (thanks @Mr-Beasley)

• qmd update no longer deactivates every document in a collection whose root folder is missing, for example an unmounted drive or an offline network share. The folder globbed to an empty list, which read as "every file was deleted", so search went empty and a qmd cleanup before the drive came back deleted the rows for good. The collection is now reported as not found and its index is left unchanged. Permission and I/O errors still fail with their original cause (#989). #990 (thanks @ParkerRex)

• Embedding generation and legacy fingerprint adoption now tokenize documents with the store-selected embedding model instead of the global default. This keeps chunk boundaries aligned with the model that creates and verifies the stored vectors without initializing an unrelated provider.

• qmd doctor no longer stalls or fills the temp directory on large indexes when checking legacy (empty-fingerprint) embeddings. The adoption sample query joined content and grouped by the document body, so SQLite materialized the full body once per legacy chunk and per active path before LIMIT 1 discarded it, the same pattern as the doctor vector-sample check (#978). It now picks the sample row through indexes and loads only that row's body; the sampled chunk is unchanged (#994). #995 (thanks @mjaverto)

Changed

• The MCP server caches its instructions per store for up to 60 seconds instead of rebuilding them, with a full index-status scan, for every HTTP request. Concurrent requests share one build and a failed build is not cached; index changes from other processes show up in the instructions within a minute. #815 (thanks @fxstein)

• Avoid a redundant runtime process on packaged CLI calls where the launcher is already running under its selected Node or Bun runtime. #923 (thanks @ilepn)

• qmd update no longer reads files whose modification time and size match the last pass (tracked in a new file_sync_state table), so re-indexing an unchanged collection takes seconds. A cached entry counts only while its document is still active at that path with that content, so a collection removed and added back is re-indexed, while a renamed one keeps its entries; files with missing or outdated metadata are still re-extracted. qmd update and qmd collection add skip files over 10 MB with FILE_TOO_LARGE, and qmd update deactivates a previously indexed file that becomes empty or is over 10 MB, including one indexed by an earlier release. #962 (thanks @rikvanriel)

• Search results carry at most the first 262,144 characters of each document body (qmd search --full, MCP results, keyword and vector hits), which bounds memory on indexes with very large documents. A match past that point is still found, but its snippet, and the passage the reranker scores, come from the start of the document. #962 (thanks @rikvanriel)

• qmd update and qmd collection add write a collection scan in short transactions instead of committing every row on its own, so a first index of 20,000 files takes about 5 s instead of about 3 minutes. #1021 (thanks @brettdavies)

• qmd query and qmd vsearch now drop repeated query expansions before searching. The expansion model can repeat a line, and the cache kept every copy: one reported query ran 23 expansions where 9 were distinct, and each copy ran its own search. Cached expansions are deduplicated when read, so existing caches need no rebuild. Repeated lex lines used to count more than once in the rank fusion, so result order can shift slightly. (#921) #1000 (thanks @ParkerRex)

• Structured searches over several collections (qmd query with lex:/vec:/hyde: lines, the MCP query tool, SDK queries) run one search per line over the whole collection list, a keyword search for a lex: line and a vector search for a vec: or hyde: line, and fuse one ranked list per search. Rankings no longer depend on the order the collections are named. #946 (thanks @shalom-t), #1009 (thanks @xidus90)

Changed

• The vector index is partitioned by collection. Each embedded chunk is stored once per collection that holds it, under an integer collection id, so a collection-scoped qmd query or qmd vsearch scans only that collection's vectors instead of pre-filtering the whole index, and a small collection is never crowded out of the results by a large one. The first command after upgrading converts the index in place and prints its progress; the conversion resumes from where it stopped if interrupted, two commands started at once share it, and a VACUUM at the end reclaims the space of the old table. qmd collection rename no longer touches vectors, qmd update removes, as it finishes, the vectors of content that no indexed document holds any more (content that comes back later is embedded again), and qmd update and qmd embed copy an already-embedded document into a collection that gains it instead of embedding it again. A metadata-filtered vector search returns the nearest documents the filter admits however many chunks it admits, and a vector search fills its limit even when one long document holds all of the nearest chunks. Downgrading to an older qmd afterwards needs qmd embed -f, and a qmd mcp server started before the upgrade must be restarted. #983 (thanks @brettdavies)

Changed

• qmd cleanup repacks the vector table when fewer than 90% of its chunk reads hold live rows. vec0 fills the first free slot of its newest chunk and reclaims a chunk only once it is empty, so re-embeds and orphan removal leave holes in older chunks that every brute-force scan still reads; a 794k-vector index at 36% occupancy scanned twice as slowly as a packed copy of the same rows. The repack moves the live rows of each mostly-empty chunk to the tail, one chunk per short transaction, so it never holds the write lock for long and an interrupted run leaves a consistent table. The cleanup output and qmd cleanup --dry-run report the chunk counts. #937 (thanks @brettdavies)

[2.8.3] - 2026-08-16

Security

• qmd update no longer runs a project-local .qmd/index.yml's update: commands without approval (#886). That file arrives with a git clone and is adopted automatically for any command run inside the tree, so cloning a repository and running qmd update executed shell commands chosen by whoever wrote it. On a terminal QMD now lists the commands and asks; with nobody to ask it skips them and keeps indexing. Approvals are recorded per config file and per command set in <config dir>/trusted.json, so editing a command — or a git pull that rewrites one — asks again. New qmd trust, qmd trust list and qmd trust revoke manage approvals, and QMD_TRUST_UPDATE_HOOKS=1 opts unattended runs back in. Commands in your own ~/.config/qmd/*.yml, including anything qmd collection update-cmd writes, are unaffected.

• The same project-local trust gate now covers collection path values that resolve outside the project and non-default models.embed / models.rerank / models.generate URIs (#889). In-project paths still index unattended; out-of-project directories are skipped until qmd trust, and custom model URIs are not loaded or downloaded. QMD_TRUST_LOCAL_CONFIG=1 opts unattended runs back in (and QMD_TRUST_UPDATE_HOOKS=1 still does). qmd collection add records trust as it writes, the same way update-cmd does.

• Indexing no longer follows file symlinks or glob ../ / absolute patterns out of the collection directory. fast-glob already skipped symlinked directories, but a file symlink (or a mask like ../**/*.md) still resolved via realpath and ingested the target. qmd:// filesystem resolution uses the same containment check, so qmd://collection/../../../etc/passwd no longer produces a path outside the collection.

• qmd mcp --http now validates the Origin and Host headers on every request and answers 403 when they name anything but a loopback address (#881). Binding to localhost is no defence against the user's own browser: a page can re-point its hostname at 127.0.0.1 (DNS rebinding) and read the indexed corpus through POST /query or POST /mcp. Requests with no Origin (curl, MCP clients, editors) are unaffected. Extend the allowlists with QMD_ALLOWED_ORIGINS / QMD_ALLOWED_HOSTS, or set QMD_ALLOWED_ORIGINS=* behind your own authenticating proxy. A wildcard bind (--host 0.0.0.0) skips the host check and warns at startup.

Changed

• Dependencies: node-llama-cpp 3.18.1 → 3.20.0 (llama.cpp b8390 → b10361, 2026-08-11). Also safe patch/minors: picomatch 4.0.4 → 4.0.5, web-tree-sitter 0.26.8 → 0.26.12, tsx 4.21.0 → 4.23.12, vitest 3.2.4 → 3.2.7. No zod/vitest major; @modelcontextprotocol/server stays 2.0.0 (no 2.x patch). flake.nix FOD hashes are not updated here.

• generate and query expansion now await LlamaContextSequence.dispose() before disposing the parent context. node-llama-cpp 3.20 made sequence dispose async; the library's context-onDispose path does not wait.

• MCP server now speaks protocol revision 2026-07-28 via the official TypeScript SDK 2.x (@modelcontextprotocol/server). HTTP is sessionless (no Mcp-Session-Id, no initialize handshake, no idle-session TTL / #816 reaper). Clients send version and capabilities in _meta; server/discover is implemented; Streamable HTTP POST requires Mcp-Method / Mcp-Name (mismatch → -32020); tools/list is deterministic and carries ttlMs / cacheScope. 2025-era stdio clients still work (serveStdio dual-speak); 2025-era HTTP initialize is answered per-request without minting a session. Existing tools (query / get / multi_get / status), stdio EOF shutdown, and named-index daemon PIDs are unchanged. No release.

• qmd pull (and implicit model downloads in embed/query) no longer print node-llama-cpp's download progress bar. The bar redraws every few kilobytes and flooded agent transcripts with thousands of tokens (#776). Pass qmd pull --progress to show it on an interactive terminal.

• --full-path no longer degrades silently when a result cannot be resolved on disk (#785). A fallback there means the file moved or was deleted since the last index, so search, query, get and multi-get now print a notice to stderr naming how many results fell back and suggesting qmd update; stdout stays machine-readable.

• search/query now decide per result whether to show the docid under --full-path, matching multi-get and get: a result that resolved shows its on-disk path and no docid, one that did not keeps its qmd:// URI and its docid, so it is still addressable. Previously the docid was dropped for every row whenever the flag was set, leaving unresolved rows with neither a usable path nor an identifier.

• search --format csv always emits the docid column, empty for rows that resolved to an on-disk path. Under --full-path the header previously dropped the column entirely — which also disagreed with the empty-result header, always printed with docid. Column positions are now stable across runs and formats.

Fixed

• Concurrent first-open of a cold index no longer fails with table documents_fts already exists on Bun/macOS. FTS5 CREATE VIRTUAL TABLE IF NOT EXISTS is not atomic across WAL connections: two processes can both see a missing table on their schema snapshot and the loser throws. Table create and legacy-schema repair now use the same BEGIN IMMEDIATE + double-check as the FTS sync triggers, and treat a concurrent "already exists" as success when the table is present.

• Nix flake qmd-node-modules FOD hashes updated for x86_64-linux and aarch64-darwin after the MCP SDK 2.0 bump. nix build / Nix GHA was failing with a fixed-output hash mismatch.

• CJK FTS rebuild no longer skips leftover fts5(name, body, content='documents') tables when fts_cjk_normalized_version is already stamped, and schema repair now checks live FTS columns (PRAGMA table_info) as well as sqlite_master.sql. The MCP HTTP test helper still seeds that legacy table; startMcpHttpServer / createStore on it must not throw no such column: T.name (#792 regression).

• qmd collection add --glob is no longer silently ignored. parseArgs ran with strict: false, so OpenClaw's --glob memory.md (and any other --glob) fell through, the default **/*.md was used, and a second collection on the same path collided as a duplicate instead of indexing the requested mask (#536). --glob is now an alias for --mask.

• The bin/qmd trampoline now execs process.execPath instead of re-resolving node from PATH. Native addons (better-sqlite3) are compiled for the Node that installed qmd; a version manager (nvm, fnm, mise) selecting a different major in the working directory used to spawn that other binary and fail with NODE_MODULE_VERSION / ERR_DLOPEN_FAILED (#577 leftover; #319). bun bin/qmd still resolves node from PATH so Node-ABI addons are not loaded into bun.

• qmd update / qmd collection add no longer swallow unreadable files silently (#460). readFileSync failures (ETIMEDOUT on APFS compressed files, EAGAIN, EACCES, …) still skip the file so the rest of the collection indexes, but the CLI now warns with the path and error code and reports the skip count. The SDK update() result includes skipped.

• The architecture diagram no longer draws Vec expansions into BM25 search (#680). lex expansions are FTS-only; vec and hyde expansions are vector-only. The original query still goes to both backends.

• qmd bench no longer runs to a wall of 0.00 when the fixture collection is missing or empty (#716). It errors up front with the same "Collection not found" / index hint as qmd search -c, and if every backend still scores zero it warns on stderr to check qmd ls.

• qmd cleanup now reclaims the content and FTS space left behind after a wrong-directory qmd update. Deactivating files (the next update in the right directory) only tombstoned the documents rows; cleanup deleted those rows and vacuumed, but never dropped the unreferenced content hashes and never ran FTS5 optimize, so documents_fts_data kept the old bodies (#550). Cleanup now deletes inactive docs, then orphaned content, then compact FTS, then vacuum. --dry-run reports the content hashes too.

• Concurrent query calls with rerank: true on a cold MCP server no longer race ensureRerankContexts(). Embed already serialized context creation; rerank did not, so two overlapping first queries both saw an empty pool, both created ranking contexts, and the inactivity timer disposed the loser (Object is disposed, #682). Callers now await the in-flight create.

• Embedding-context pool size no longer assumes every GGUF costs 150 MB of VRAM (the nomic-embed figure). Larger models such as Qwen3-Embedding-0.6B are ~1190 MB per 2048-token context; opening 8 of those exhausted an 8 GB card so qmd query failed with Failed to create any rerank context even though the reranker itself was fine. The pool is now sized from the weight file, and 1 GB is reserved for the reranker (#799). Default embeddinggemma/nomic throughput is unchanged. QMD_EMBED_PARALLELISM still overrides.

• Multi-collection -c A -c B (and SDK/MCP collections: [A, B]) no longer searches globally then post-filters. A large unrelated collection could fill the FTS/ANN top-k so the requested collections vanished, yielding false-empty results even though each collection matched on its own. searchFTS / searchVec now search each requested collection, then merge by score (#775). Single-collection exact-scan (#791, #803) is unchanged.

• Query expansion no longer consumes caller intent, and a cached expansion whose sub-queries all miss is dropped instead of replaying forever (#818). Intent still steers reranking and snippet/chunk selection; it just no longer enters the expansion prompt or cache key, where the model copied meta-language ("so I can compare spend settings") into lex/vec terms that matched nothing.

• Files whose names differ only in the characters the legacy slug collapsed to - (spaces, underscores) no longer evict each other from the index (#717). The handalized-path migration now skips any row whose path is still owned by a file in the current scan, so it only adopts genuinely stale pre-2.6 rows. qmd get and qmd ls <prefix> also match _ and % in paths literally instead of as SQL LIKE wildcards, so qmd get 2026_06_16.md no longer returns a sibling 2026-06-16.md.

• CLI multi-get and SDK/MCP multi_get now share one comma-list resolver. Collection-prefixed paths (qmd/docs/SYNTAX.md) work in both transports, unanchored LIKE '%name' no longer silently fetches a different document for a filename fragment (NTAX.md ≠ SYNTAX.md), and ambiguous names across collections error with the candidate list instead of LIMIT 1 (#759).

• The Nix flake wrapper now seeds the same pre-import env as bin/qmd. Nix installs exec bun src/cli/qmd.ts directly, so they previously skipped the launcher: qmd mcp could leak llama/ggml native logs onto JSON-RPC stdio, and Darwin CLI exits dumped a ggml Metal residency-set stack trace after an otherwise successful query. The wrapper now quiets those logs for mcp and sets GGML_METAL_NO_RESIDENCY=1 on Darwin unless QMD_METAL_KEEP_RESIDENCY=1 (#723).

• Rerank context creation no longer swallows the real failure. A VRAM OOM or corrupt model previously produced only Reranker unavailable — skipping reranking (and a dead identical retry whose comment claimed it disabled flash attention, which ranking contexts never supported). The warning now includes the underlying message so the two cases are distinguishable (#782).

• The Nix flake package now ships skills/ next to src/ in $out/lib/qmd/. findPackageRoot() walks up from the wrapped src/cli/qmd.ts looking for a sibling skills/ directory; without it, qmd skill show and qmd skills list always failed with "QMD skill not found" on Nix-installed binaries (#722).

• /release step 1 no longer points at a missing script. skills/release/scripts/release-context.sh now exists: it silently installs git hooks and prints version info, working-tree status, commits and files since the last tag, [Unreleased], and the previous changelog entry. The skill's process list also drops the duplicate step 7 and checks dependency updates before cutting the release (#796).

• store.searchVec() (and SDK searchVector()) now embed the query with the store's pinned embed model instead of the global QMD_EMBED_MODEL. A store created with a non-default models.embed previously failed with Dimension mismatch ... Expected N ... received M and loaded the wrong (often much larger) model at query time. Hybrid/precomputed/session search paths were unaffected and stay unchanged (#690).

• qmd collection add --mask "a.md,*.txt" now indexes the union of each pattern. The comma-separated form was documented and commonly guessed, but the joined string was passed to fast-glob as one literal glob, so it matched zero files with no error. Brace form {a.md,*.txt} is unchanged. The same split applies on qmd update for stored comma-list masks (#557).

• NixOS / immutable-root installs no longer crash qmd embed with EACCES when node-llama-cpp tries to compile llama.cpp into a read-only node_modules. The flake wrapper puts Nix's glibc and libstdc++ on LD_LIBRARY_PATH so prebuilt binaries can dlopen them, and getLlama() uses build: "never" when the llama directory is not writable (#574).

• CJK FTS rebuild no longer raises "database is busy" when flushing insert batches. The streaming .iterate() cursor stayed open across BEGIN on the same connection; the scan now uses keyset-paginated LIMIT batches so each SELECT finalizes before the insert transaction starts (#797).

• Quoted FTS phrases containing dotted tokens (e.g. "1.0.21") now match the indexed document. The porter unicode61 tokenizer stores dotted strings as adjacent parts, but phrase sanitization stripped the dots into a single token (1021) that could never hit. Dotted tokens inside quotes are split into adjacent phrase terms, matching the bare-term rewrite from #563 (#757).

• scripts/build.mjs no longer passes shell: true to spawnSync on Windows. With the default Node install path (C:\Program Files\nodejs\node.exe), cmd.exe split the unquoted process.execPath at the space, the tsc spawn failed, and prepare could still report success with no dist/ — leaving bin/qmd at "not built". The helper always receives a real binary path plus an args array, so no shell is needed. A spawn error now prints the missing binary path instead of failing silently. (#681)

• Case-sensitive collections no longer collapse distinct document identities that differ only by path casing. The implicit COLLATE NOCASE legacy migration was unsafe for filesystems that contain both README.md and readme.md; case-only legacy migrations must now be explicit and operator-reviewed (#801).

• cleanupOrphanedVectors now runs its orphan count and both DELETEs in a single immediate transaction. An interruption between the two DELETEs (crash, SQLITE_BUSY) could desync vectors_vec from content_vectors, leaving stale metadata rows that make a later reactivation of the same content hash look already-embedded — so qmd embed skips it and the document becomes silently unsearchable by vector, with no orphan left to clean up (#766).

• qmd embed no longer splits a UTF-16 surrogate pair (emoji, etc.) across a chunk boundary. A chunk ending or starting mid-pair produced an unpaired surrogate in the chunk text, which some remote embedding APIs reject as invalid JSON — permanently failing that chunk on every retry, since the boundary calculation is deterministic. This covers both the character-based chunker and chunkDocumentByTokens's recursive re-splitting for astral-plane-dense content, which could previously drive the char budget low enough to reproduce the same split (#777).

• insertContext looks up store_collections by name. #754 retargeted the query from the dropped collections table but left WHERE id = ?, and store_collections has name TEXT PRIMARY KEY with no id column — the call threw no such column: id. Matches deleteContext / updateStoreContext (#853).

• qmd collection add with no path argument now errors with usage instead of silently indexing the current working directory (#684). Pass . to index CWD, matching the documented examples.

• qmd status reports orphaned embedding chunks, qmd update hints when they exceed 10% of vectors, and qmd cleanup --dry-run previews what would be removed. Incremental update still does not auto-prune vectors (a transient empty mount would otherwise force a full re-embed) (#768).

• qmd --index <name> mcp --http --daemon now scopes PID/log files per index (mcp-<name>.pid) and passes the resolved database path to the child, so a named-index daemon no longer collides with the default mcp.pid or opens the default store (#772).

• Opening a store no longer throws SQLiteError: no such column: T.name when documents_fts is still the legacy fts5(name, body, content='documents') schema. CREATE VIRTUAL TABLE IF NOT EXISTS left that table in place, and the CJK FTS rebuild's DELETE FROM documents_fts compiled against documents.name, which does not exist (#792).

• Rerank cache keys now include the resolved models.rerank URI, so swapping the configured reranker no longer serves the previous model's cached scores (#764).

• vsearch -c <collection> no longer returns empty results for small collections crowded out of the global ANN candidate pool. searchVec now exact-scans the collection's vectors with vec_distance_cosine when the set is within 20k rows (ANN + post-filter cannot see collections that never enter global top-k, and sqlite-vec caps k at 4096 so a larger multiplier alone is not enough). Larger collections still use capped ANN over-fetch (#791, #803).

• multi-get --format files now emits the docid as its own CSV field (#docid,path,...) instead of prepending it into the path field with a space (#docid path,...), matching search --format files and keeping naive comma-splitting usable (#760).

• qmd embed now takes an exclusive process lock (.qmd-embed.lock next to the index DB) so concurrent invocations no longer race on vectors_vec and fail with UNIQUE constraint failed: vectors_vec.hash_seq. A second embed exits early with Another embed process is already running. Skipping. Stale locks from crashed processes are recovered via PID identity checks (#825).

• windows prepare fix #778 keeps dist build #824

• qmd mcp (stdio) now shuts down gracefully when stdin reaches EOF instead of orphaning to PID 1 when the parent MCP client dies (#751): the server closes its transport, gives in-flight request handlers a bounded window to settle, closes the store (which disposes its llama.cpp instance), and lets the process drain via process.exitCode (no forced process.exit(), which has caused exit-time native crashes before).

• qmd --version no longer reports an unrelated repository's commit (#787). The commit was discovered at runtime with git -C <installDir> rev-parse, and git -C walks up, so any install nested inside another checkout reported that checkout's HEAD — a global npm install under a git-managed prefix such as Homebrew's /opt/homebrew claimed Homebrew's commit as qmd's. Identical tarballs reported different "commits" depending only on where they were installed, which is why one issue can collect three distinct hashes for the same published build. scripts/build.mjs now stamps the commit it built from into dist/cli/build-info.json (suffixed -dirty when built from a modified tree), and the runtime prefers that. Source checkouts still resolve their own HEAD, but only after confirming the enclosing repository is qmd's; anything else reports no commit rather than a misleading one. The lookup also no longer interpolates the install path into a shell string, so a path containing a space stops silently dropping the commit, and --version from a build runs no subprocess at all.

• qmd doctor no longer false-positives .etag HTTP sidecars (written by qmd pull next to each download) as invalid GGUF models. The model-cache check now only inspects real .gguf files, so a sidecar that happens to sort before the blob no longer poisons the report. #812

• qmd mcp stop and qmd mcp --http --daemon now verify that a pidfile PID still belongs to a qmd process before signalling it or refusing to start. Recycled PIDs (common after reboot) are treated as stale: the pidfile is unlinked instead of SIGTERM'ing an unrelated process or blocking daemon start with a false "Already running" error (#806).

• qmd collection add now rejects missing paths and regular files before creating collection configuration or index state. The error reports both the received and resolved path so malformed shell arguments can be corrected.

• Claude Code plugin: scope the plugin source to ./skills so installs copy just the skills (~50 KB) instead of the entire repository. Previously a canonical install materialized ~230 MB / 9,000+ items into ~/.claude/plugins/cache/ — including a full npm dependency install triggered by the repo-root package.json. Both skills (qmd and release) still ship, unchanged. (#790)

• Claude Code plugin: releases now bump the plugin version in .claude-plugin/marketplace.json in lockstep with package.json. The plugin cache is keyed on this version, and it had been stuck at 0.1.0 since February — so installed plugins never received skill updates (users who installed in February are still being served that snapshot today, despite the qmd skill nearly tripling in size since). Also bumps the plugin to 2.6.3 as a one-time catch-up so existing installs pick up the current skill on their next claude plugin update. (#789)

Changed

• --full-path no longer degrades silently when a result cannot be resolved on disk (#785). A fallback there means the file moved or was deleted since the last index, so search, query, get and multi-get now print a notice to stderr naming how many results fell back and suggesting qmd update; stdout stays machine-readable.

• search/query now decide per result whether to show the docid under --full-path, matching multi-get and get: a result that resolved shows its on-disk path and no docid, one that did not keeps its qmd:// URI and its docid, so it is still addressable. Previously the docid was dropped for every row whenever the flag was set, leaving unresolved rows with neither a usable path nor an identifier.

• search --format csv always emits the docid column, empty for rows that resolved to an on-disk path. Under --full-path the header previously dropped the column entirely — which also disagreed with the empty-result header, always printed with docid. Column positions are now stable across runs and formats.

[2.6.3] - 2026-06-24

Added

• qmd embed --timeout <minutes> overrides the embed session's max duration (previously hardcoded to 30 minutes). Use a larger value to let a big index finish in one run, or --timeout 0 to remove the cap entirely. When the cap is reached, remaining document batches are skipped as before, so re-running qmd embed continues where it left off.

Documentation

• README: added a "Configuring index.yml" section documenting the full config schema — global_context, editor_uri, the models.embed/rerank/generate overrides, and per-collection path/pattern/ignore/update/ includeByDefault/context — with file-location rules (XDG_CONFIG_HOME, QMD_CONFIG_DIR, named {name}.yml, project-local .qmd/index.yml). Every key is verified against src/collections.ts and its consumers. Documents behavior that previously existed only in code, where the absence of docs led contributors to repeatedly re-submit already-shipped model-resolution fixes (#502, #559, #564) and to request config that already works (#645, #678). Added XDG_CONFIG_HOME/QMD_CONFIG_DIR to the environment-variable table and noted the index.yml models: / QMD_EMBED_MODEL override path in the Model Configuration section.

• README: expanded the per-collection update field into an "Automatic update commands" subsection — the feature the maintainer publicly called under-documented — covering execution via bash -c in the collection's directory, the run-then- reindex order, the non-zero-exit abort behavior, and the qmd collection update-cmd set/clear shortcut. The ignore key now states it is YAML-only (no CLI command) and additive with the un-overridable built-in exclusions.

• example-index.yml: overhauled from three near-identical collections into a fully-commented starter template where each collection demonstrates a distinct feature (hierarchical context, auto-update, ignore patterns, non-markdown globs, includeByDefault: false, and an all-fields example), plus commented editor_uri/models stubs. README now links to it. Model URIs are intentionally left as placeholders so the template can't drift from the defaults.

• README: documented collection filtering (-c semantics), the collection show/include/exclude/update-cmd subcommands, the --intent/--no-rerank/ -C/--full-path search flags, the --format <kind> output selector (with the legacy --json/--csv/--md/--xml/--files booleans noted as aliases), vector-search/deep-search aliases, embed memory flags (--max-docs-per-batch/--max-batch-mb), a sample --explain score trace, the qmd doctor/qmd init commands, the get :from:count suffix and --no-line-numbers, an MCP tool parameter reference, and a Benchmarking section for qmd bench.

• docs/SYNTAX.md: removed the non-existent q MCP parameter example (the query tool and REST endpoint accept only the searches array) and added a Scoping section.

• README: removed the misleading qmd update --pull example. The --pull flag is parsed but never consumed (updateCollections() ignores it); the real mechanism for running git pull before re-indexing is a per-collection update command, set via qmd collection update-cmd.

Fixed

• MCP server instructions now tell agents to scope with the plural collections parameter (matching the schema). The previous singular collection hint led agents to pass a parameter that Zod silently strips, producing unscoped results. The get instruction line also now documents the full file.md:from:count range suffix instead of only the single-line file.md:100 offset.

• Filesystem paths with special characters (#, &, spaces, [], (), etc.) now round-trip correctly through index → search → get. Previously reindexCollection called handelize() on relative paths before storing them, turning # Meeting - 234232 3432 __ 5.md into Meeting-234232-3432-5.md and making qmd get <actual-path>, qmd get --full-path, and qmd ls return dead or garbled paths. Paths are now stored verbatim. Existing indexes auto-migrate on the next qmd update.

• FTS5 search now correctly matches dotted version strings like 2026.4.10. The porter unicode61 tokenizer splits on dots (storing 2026, 4, 10 as separate tokens), but the query sanitizer was stripping dots and producing 2026410 which never matched. Dotted terms are now split and ANDed together so version-string searches work as expected (#563).

• HTTP REST endpoints /query and /search now return qmd://collection/path URIs in the file field, matching the output format used by the CLI and MCP resource URIs. Previously the raw displayPath (collection/path) was returned without the scheme prefix (#576).

• The embed session maxDuration is now env-configurable via QMD_EMBED_MAX_DURATION_MS (default: 30 min). This prevents large-corpus embeddings from being aborted by the hardcoded 30-minute ceiling (#673).

• qmd query, qmd update, and other commands no longer fail with SQLiteError: database is locked when multiple processes run against the same index in parallel (e.g. an update racing a long embed, the first-open schema migration racing a routine command, or an agent fanning out searches). openDatabase now sets PRAGMA busy_timeout = 120000 on every connection, so a writer that loses the race queues at batch boundaries instead of throwing immediately. WAL handles read/write concurrency but does not serialise concurrent writers, and bun:sqlite and better-sqlite3 both default the timeout to 0, so the loser previously failed on the first DDL statement in initializeDatabase. Override the default with QMD_SQLITE_BUSY_TIMEOUT (milliseconds; 0 restores fail-fast). Two more crashes on the same concurrent-open path are fixed: trigger documents_ai already exists (the FTS sync triggers were dropped and recreated as separate statements on every open, so two processes interleaved between the DROP and the CREATE; busy_timeout serialises individual statements but not the pair) and database is locked while migrating a cold database to WAL (the journal_mode switch needs a brief exclusive lock and does not invoke the busy handler). FTS trigger setup is now gated behind PRAGMA user_version inside one IMMEDIATE transaction, and the WAL migration retries within the busy-timeout budget.

[2.5.3] - 2026-05-28

Features

• qmd get now accepts a :from:count suffix on a path or docid (e.g. qmd get "#abc123:120:40" reads 40 lines starting at line 120). Explicit --from/-l flags still override the suffix. The MCP get tool accepts the same suffix.

• qmd get and qmd multi-get are now line-numbered by default and print the document's #docid and qmd:// path in the output header. Disable line numbers with --no-line-numbers. The MCP get/multi_get tools default lineNumbers to true to match.

• qmd multi-get now includes the #docid in every output format (--md, --json, --csv, --xml, --files, and the default CLI view), consistent with qmd search.

• qmd get and qmd multi-get accept --full-path, which replaces the qmd:// path + #docid with the document's on-disk filesystem path (handy for piping into Read/Edit/an editor). Falls back to the canonical qmd:// + docid header when the file no longer exists on disk.

• qmd search / qmd query now show a clearer hit identifier: the default CLI view (and the new **file:** line in --md output) always prints the full qmd://collection/path URI so you can pipe it straight back into qmd get.

• qmd search / qmd query accept --full-path with the same semantics as qmd get: the result label becomes the file's on-disk path — ./-prefixed relative path when the file lives in a subfolder of $PWD, absolute realpath otherwise — and the per-result #docid is dropped because the path is the identifier. The leading ./ is intentional so the output is unambiguously a filesystem path. Applies to all output formats.

• qmd get and qmd multi-get now also use the ./-prefixed convention when --full-path renders a path under $PWD, matching search/query.

• New --format <kind> flag selects the output format (cli | json | csv | md | xml | files) for search, query, and multi-get. The legacy boolean aliases (--json/--csv/--md/--xml/--files) still work but are no longer in --help; prefer --format.

Fixes

• Launcher: source-mode runner selection now prefers Node + tsx over Bun when both package-lock.json and bun.lock are present in the package root, mirroring the dist-mode "npm priority" rule. Fixes pnpm-global installs that copy the entire working tree (including .git and bun.lock) into the install dir and previously routed through Bun, causing ABI mismatches with the Node-built better-sqlite3 / sqlite-vec native modules.

• Darwin Metal: llama-using commands (query, vsearch, embed) no longer dump a multi-kB GGML/Metal backtrace at process exit even when output succeeded. The libggml-metal static ggml_metal_device destructor asserts [rsets->data count] == 0 during __cxa_finalize_ranges, but the buffer-free path never calls the symmetric ggml_metal_device_rsets_rm to remove released rsets from the device collection (upstream ggml-org/llama.cpp#22593, one-line fix open as PR #22595). The assertion only fires when process.exit() skips Node's beforeExit hook, which is what node-llama-cpp uses to auto-dispose Metal contexts. Primary fix: finishSuccessfulCliCommand now sets process.exitCode = 0 and returns instead of calling process.exit(0), so beforeExit fires and the native binding cleans up before libc's static destructor runs. Defense-in-depth: the launcher (bin/qmd) and the npm test driver (scripts/test-all.mjs

  • the test:bun / test:unit package.json scripts) also set GGML_METAL_NO_RESIDENCY=1 on darwin before spawning node/bun, covering error paths and tests that still terminate via process.exit(). The env var must be set before node/bun start — libggml-metal reads it via libc getenv at module-load time, and Bun does not propagate process.env mutations to libc setenv — so it lives in the launcher rather than in test-preload. Residency sets give no measurable speedup for QMD's short-lived CLI workflow (benchmarked on M3 Pro). Opt back in with QMD_METAL_KEEP_RESIDENCY=1 for long-lived qmd processes (e.g. the MCP daemon may benefit on hot reload) or to triage the upstream fix. qmd doctor reports the mitigation state. Minimal reproduction: scripts/repro-metal-rsets-crash.mjs.

Docs

• qmd skill: emphasize reading line ranges with get's built-in :from:count suffix / --from/-l flags instead of piping through sed/head/tail; cite the docid and line numbers now present in retrieval output; and author structured intent:/lex:/vec:/hyde: queries yourself rather than relying on built-in query expansion.

[2.5.2] - 2026-05-22

Fixes

• Launcher: Rewrite bin/qmd as a Node-based shebang polyglot to fix global npm installation execution failures on Windows (#668 / #452), while supporting seamless fallback to Bun in Node-less environments.

[2.5.1] - 2026-05-20

Changes

• Release: publish from GitHub Actions via npm Trusted Publishing/OIDC instead of a long-lived NPM_TOKEN secret.

[2.5.0] - 2026-05-19

Changes

• Dependencies: update core SQLite/config/chunking packages (better-sqlite3, yaml, web-tree-sitter, tree-sitter-go, and tree-sitter-python) while keeping incompatible zod, tsx, and vitest majors pinned.

• Agent skills: add qmd skills list|get|path to serve version-matched runtime skill instructions from the installed CLI, and make qmd skill install write a stable discovery stub so installed agent skills do not go stale after QMD upgrades.

• CLI: add qmd doctor for index/runtime diagnostics, including SQLite/sqlite-vec versions, embedding fingerprint freshness, mixed-fingerprint detection, safe legacy fingerprint adoption, and content-hash sampling.

Fixes

• Launcher: prefer runnable TypeScript source in git checkouts even when ignored dist/ artifacts exist, while packaged installs continue to run dist/.

• GPU: keep node-llama-cpp's documented gpu: "auto" initialization as the primary path, then perform no-build packaged CUDA/Vulkan/Metal probes only if auto falls back to CPU.

• CLI: move GPU/CPU runtime diagnostics out of qmd status; use qmd doctor for device probing and related environment guidance.

• CLI: point unexpected command/setup failures toward qmd doctor so diagnostics are the default next step when QMD behaves incorrectly.

• Doctor: explicitly warn when content_vectors contains multiple non-empty embedding fingerprint names, with the per-fingerprint document/chunk breakdown.

• Embed: make the TTY progress line label byte-based input progress explicitly, show embedded chunks as a count, and shorten the displayed model name.

• Embed: retain per-chunk failure details, retry failed chunks after later successful embeds and again when no other chunks remain, clear recovered errors, and cap retries to avoid endless loops.

• Tests: expand the container smoke harness to cover npm-global, npx-style, and Bun-global install scenarios, always checking auto and QMD_FORCE_CPU=1 doctor modes, with opt-in tiny qmd embed and GPU probe runs for supported container runtimes.

• Embedding: fingerprint vector metadata using the active embedding model and formatting/chunking parameters so stale vectors are treated as pending after search semantics change. Legacy content_vectors columns are migrated lazily on first vector-health/write use to preserve fast QMD startup.

• Skill: expand the packaged QMD skill with retrieval-first workflows, structured query examples, wiki/source collection guidance, and safe fallbacks when model-backed search is unavailable.

• Tests: make bun run test execute the local unit suite under both Node/Vitest and Bun (test:node + test:bun) so runtime-specific regressions are caught before CI.

• Model config: centralize embedding/rerank/generation model resolution so qmd embed, status, query, vsearch, pull, SDK vector search, and bench use the same active .qmd/index.yaml model hints and environment fallbacks.

• GPU/status: qmd status now uses the same embedding model identity as qmd embed when computing pending embeddings, so URI-backed embeddings are not incorrectly reported as pending under the legacy embeddinggemma alias.

• GPU status: qmd status now always shows GPU mode/configuration without unsafe native probing, and CPU-fallback warnings point to QMD_STATUS_DEVICE_PROBE=1 qmd status for an actual backend probe. The no-GPU warning is emitted once per process instead of once per LLM instance during benchmarks.

• GPU: add QMD_FORCE_CPU=1 / --no-gpu to bypass CUDA/Vulkan/Metal probing entirely, and route native llama.cpp stdout noise to stderr so JSON output stays parseable during search/query commands.

• Snippet line numbers: qmd_query (MCP), HTTP /query, and qmd query (CLI JSON output and snippet headers) now return absolute source-file line numbers instead of chunk-local ones, so the line field can be passed back to qmd_get as fromLine without a separate lookup. Snippet selection remains scoped to the best matching chunk (preserves #149).

• CLI: qmd query --full now emits the full document body in all output formats (json, csv, md, xml), restoring the documented behavior of the flag. Previously it returned only the best matching chunk (~3.6KB max per result). Output payload for --full queries is now proportional to total document size.

• macOS Metal: qmd query --json now flushes successful JSON output and uses a safe immediate-exit path on Darwin to avoid ggml Metal finalizer aborts; other commands still dispose LLM contexts/models before the llama runtime. #368

• Embedding: require complete chunk coverage before treating a document as embedded, remove partial vectors when chunk/session failures leave a document incomplete, and keep qmd status pending counts honest after interrupted long embed runs. #637 #378

• Embedding: qmd embed -c <collection> now scopes pending-doc selection to the requested collection instead of embedding global pending work. Scoped --force clears only collection-owned vectors, preserves shared hashes referenced by sibling collections, and drops vectors_vec only when the scoped clear empties all vectors.

• Hybrid search: weight RRF lists by query type so original FTS and original vector evidence get the intended 2x boost, instead of accidentally boosting the first lexical expansion. #591

• MCP: seed llama.cpp/GGML quiet env vars before launching qmd mcp so native logs cannot pollute stdio JSON-RPC framing. #593

• CLI: remove CommonJS require() calls from ESM index path normalization so qmd --index <path> no longer crashes with ERR_AMBIGUOUS_MODULE_SYNTAX on Node 22+. #634

• Windows CUDA: serialize llama.cpp embedding/reranking contexts by default to avoid intermittent ggml-cuda.cu:98 crashes in qmd query; set QMD_EMBED_PARALLELISM to opt back into parallel contexts if your driver is stable. #519

• MCP: make qmd mcp --index <name> use the selected index for both foreground and daemon HTTP servers instead of falling back to the default store. #343

• Embedding: respect QMD_EMBED_MODEL consistently for vector indexing and vector-backed search, with default-model fallback when unset.

• Config: use one home-directory resolver for YAML config and the default SQLite cache path, avoiding Windows CLI/MCP split-brain when HOME is unset.

• GPU: respect explicit QMD_LLAMA_GPU=metal|vulkan|cuda backend overrides instead of always using auto GPU selection. #529

• Fix: preserve original filename case in handelize(). The previous .toLowerCase() call made indexed paths unreachable on case-sensitive filesystems (Linux). qmd update automatically migrates legacy lowercase paths without re-embedding.

• CLI: make qmd status skip native node-llama-cpp device probing by default so status stays safe on machines with broken or unsupported GPU drivers. Set QMD_STATUS_DEVICE_PROBE=1 to opt in.

• CLI: lazy-load node-llama-cpp so lightweight commands such as qmd status do not import native ML dependencies or trigger llama.cpp builds on ARM/no-GPU machines. #491

• Store: keep content rows referenced by inactive documents during orphan cleanup so qmd update preserves soft-deleted tombstones for removed files. #585

• Packaging: install AST grammar WASM packages as required dependencies so Bun global installs include TypeScript/TSX/JavaScript grammars, and add a smoke:package-grammars verification command. #595

• Launcher: add wrapper smoke coverage for scoped package, npm/npx, Homebrew/Linuxbrew, Bun global symlink layouts, and $BUN_INSTALL false-positive runtime selection regressions. #351 #353 #354 #356 #358 #359

[2.1.0] - 2026-04-05

Code files now chunk at function and class boundaries via tree-sitter, clickable editor links land you at the right line from search results, and per-collection model configuration means you can point different collections at different embedding models. 25+ community PRs fix embedding stability, BM25 accuracy, and cross-platform launcher issues.

Changes

• AST-aware chunking for code files via web-tree-sitter. Supported languages: TypeScript/JavaScript, Python, Go, and Rust. Code files are chunked at function, class, and import boundaries instead of arbitrary text positions. Markdown and unknown file types are unchanged. --chunk-strategy <auto|regex> flag on qmd embed and qmd query (default regex). SDK: chunkStrategy option on embed() and search(). qmd status shows grammar availability.

• qmd bench <fixture.json> command for search quality benchmarks. Measures precision@k, recall, MRR, and F1 across BM25, vector, hybrid, and full pipeline backends. Ships with an example fixture against the eval-docs test collection. #470 (thanks @jmilinovich)

• models: section in index.yml lets you configure embed, rerank, and generate model URIs per collection. Resolution order is config > env var (QMD_EMBED_MODEL, QMD_RERANK_MODEL, QMD_GENERATE_MODEL) > built-in default. #502 (thanks @JohnRichardEnders)

• CLI search output now emits clickable OSC 8 terminal hyperlinks when stdout is a TTY. Links resolve qmd:// paths to absolute filesystem paths and open in editors via URI templates (default: vscode://file/{path}:{line}:{col}). Configure with QMD_EDITOR_URI or editor_uri in the YAML config. #508 (thanks @danmackinlay)

• --no-rerank flag skips the reranking step in qmd query — useful when you want fast results or don't have a GPU. Also exposed as rerank: false on the MCP query tool. #370 (thanks @mvanhorn), #478 (thanks @zestyboy)

• ONNX conversion script for deploying embedding models via Transformers.js. #399 (thanks @shreyaskarnik)

• GitHub Actions workflow to build the Nix flake on Linux and macOS.

Fixes

• Embedding: prevent qmd embed from running indefinitely when the embedding loop stalls. #458 (thanks @ccc-fff)

• Embedding: truncate oversized text before embedding to prevent GGML crash, and bound memory usage during batch embedding. #393 (thanks @lskun), #395 (thanks @ProgramCaiCai)

• Embedding: set explicit embed context size (default 2048, configurable via QMD_EMBED_CONTEXT_SIZE) instead of using the model's full window. #500

• Embedding: error on dimension mismatch instead of silently rebuilding the vec0 table. #501

• Embedding: handle vec0 OR REPLACE limitation in insertEmbedding. #456 (thanks @antonio-mello-ai)

• Embedding: fix model selection when multiple models are configured. #494

• BM25: correct field weights to include all 3 FTS columns — title, body, and path were not weighted correctly. #462 (thanks @goldsr09)

• BM25: handle hyphenated tokens in FTS5 lex queries so terms like "real-time" match correctly. #463 (thanks @goldsr09)

• BM25: preserve underscores in search terms instead of stripping them. #404

• BM25: use CTE in searchFTS to prevent query planner regression with collection filter.

• Reranker: increase default context size 2048→4096 and make configurable via QMD_RERANK_CONTEXT_SIZE. Fix template overhead underestimate 200→512. #453 (thanks @builderjarvis)

• GPU: catch initialization failures and fall back to CPU instead of crashing.

• MCP: read version from package.json instead of hardcoding. #431

• MCP: include collection name in status output. #416

• Multi-get: support brace expansion patterns in glob matching. #424

• Launcher: prioritize package-lock.json to prevent Bun false positive. #385 (thanks @rymalia)

• Launcher: remove $BUN_INSTALL check that caused false Bun detection. #362 (thanks @syedair)

• Launcher: skip Git Bash path detection on WSL. #371 (thanks @oysteinkrog)

• Model cache: respect XDG_CACHE_HOME for model cache directory. #457 (thanks @antonio-mello-ai)

• SQLite: add macOS Homebrew SQLite support for Bun and restore actionable errors. #377 (thanks @serhii12)

• Pin zod to exact 4.2.1 to fix tsc build failure. #382 (thanks @rymalia)

• Preserve dots and original case in handelize() — filenames like MEMORY.md no longer become memory-md. #475 (thanks @alexei-led)

• Include line in --json search output so editor integrations can jump directly to file:line. #506 (thanks @danmackinlay)

• Nix: fix paths in flake and make Bun dependency a fixed-output derivation so sandboxed Linux builds work offline. #479 (thanks @surma-dump)

• Sync stale bun.lock (better-sqlite3 11.x → 12.x). CI and release script now use --frozen-lockfile to prevent recurrence. #386 (thanks @Mic92)

• Approve native build scripts in pnpm so better-sqlite3 and tree-sitter modules compile correctly. Update vitest ^3.0.0 → ^3.2.4.

[2.0.1] - 2026-03-10

Changes

• qmd skill install copies the packaged QMD skill into ~/.claude/commands/ for one-command setup. #355 (thanks @nibzard)

Fixes

• Fix Qwen3-Embedding GGUF filename case — HuggingFace filenames are case-sensitive, the lowercase variant returned 404. #349 (thanks @byheaven)

• Resolve symlinked global launcher path so qmd works correctly when installed via npm i -g. #352 (thanks @nibzard)

[2.0.0] - 2026-03-10

QMD 2.0 declares a stable library API. The SDK is now the primary interface — the MCP server is a clean consumer of it, and the source is organized into src/cli/ and src/mcp/. Also: Node 25 support and a runtime-aware bin wrapper for bun installs.

Changes

• Stable SDK API with QMDStore interface — search, retrieval, collection/context management, indexing, lifecycle

• Unified search(): pass query for auto-expansion or queries for pre-expanded lex/vec/hyde — replaces the old query/search/structuredSearch split

• New getDocumentBody(), getDefaultCollectionNames(), Maintenance class

• MCP server rewritten as a clean SDK consumer — zero internal store access

• CLI and MCP organized into src/cli/ and src/mcp/ subdirectories

• Runtime-aware bin/qmd wrapper detects bun vs node to avoid ABI mismatches. Closes #319

• better-sqlite3 bumped to ^12.4.5 for Node 25 support. Closes #257

• Utility exports: extractSnippet, addLineNumbers, DEFAULT_MULTI_GET_MAX_BYTES

Fixes

• Remove unused import { resolve } in store.ts that shadowed local export

[1.1.6] - 2026-03-09

QMD can now be used as a library. import { createStore } from '@tobilu/qmd' gives you the full search and indexing API — hybrid query, BM25, structured search, collection/context management — without shelling out to the CLI.

Changes

• SDK / library mode: createStore({ dbPath, config }) returns a QMDStore with query(), search(), structuredSearch(), get(), multiGet(), and collection/context management methods. Supports inline config (no files needed) or a YAML config path.

• Package exports: package.json now declares main, types, and exports so bundlers and TypeScript resolve @tobilu/qmd correctly.

[1.1.5] - 2026-03-07

Ambiguous queries like "performance" now produce dramatically better results when the caller knows what they mean. The new intent parameter steers all five pipeline stages — expansion, strong-signal bypass, chunk selection, reranking, and snippet extraction — without searching on its own. Design and original implementation by Ilya Grigorik (@vyalamar) in #180.

Changes

• Intent parameter: optional intent string disambiguates queries across the entire search pipeline. Available via CLI (--intent flag or intent: line in query documents), MCP (intent field on the query tool), and programmatic API. Adapted from PR #180 (thanks @vyalamar).

• Query expansion: when intent is provided, the expansion LLM prompt includes Query intent: {intent}, matching the finetune training data format for better-aligned expansions.

• Reranking: intent is prepended to the rerank query so Qwen3-Reranker scores with domain context.

• Chunk selection: intent terms scored at 0.5× weight alongside query terms (1.0×) when selecting the best chunk per document for reranking.

• Snippet extraction: intent terms scored at 0.3× weight to nudge snippets toward intent-relevant lines without overriding query anchoring.

• Strong-signal bypass disabled with intent: when intent is provided, the BM25 strong-signal shortcut is skipped — the obvious keyword match may not be what the caller wants.

• MCP instructions: callers are now guided to provide intent on every search call for disambiguation.

• Query document syntax: intent: recognized as a line type. At most one per document, cannot appear alone. Grammar updated in docs/SYNTAX.md.

[1.1.2] - 2026-03-07

13 community PRs merged. GPU initialization replaced with node-llama-cpp's built-in autoAttempt — deleting ~220 lines of manual fallback code and fixing GPU issues reported across 10+ PRs in one shot. Reranking is faster through chunk deduplication and a parallelism cap that prevents VRAM exhaustion.

Changes

• GPU init: use node-llama-cpp's build: "autoAttempt" instead of manual GPU backend detection. Automatically tries Metal/CUDA/Vulkan and falls back gracefully. #310 (thanks @giladgd — the node-llama-cpp author)

• Query --explain: qmd query --explain exposes retrieval score traces — backend scores, per-list RRF contributions, top-rank bonus, reranker score, and final blended score. Works in JSON and CLI output. #242 (thanks @vyalamar)

• Collection ignore patterns: ignore: ["Sessions/**", "*.tmp"] in collection config to exclude files from indexing. #304 (thanks @sebkouba)

• Multilingual embeddings: QMD_EMBED_MODEL env var lets you swap in models like Qwen3-Embedding for non-English collections. #273 (thanks @daocoding)

• Configurable expansion context: QMD_EXPAND_CONTEXT_SIZE env var (default 2048) — previously used the model's full 40960-token window, wasting VRAM. #313 (thanks @0xble)

• candidateLimit exposed: -C / --candidate-limit flag and MCP parameter to tune how many candidates reach the reranker. #255 (thanks @pandysp)

• MCP multi-session: HTTP transport now supports multiple concurrent client sessions, each with its own server instance. #286 (thanks @joelev)

Fixes

• Reranking performance: cap parallel rerank contexts at 4 to prevent VRAM exhaustion on high-core machines. Deduplicate identical chunk texts before reranking — same content from different files now shares a single reranker call. Cache scores by content hash instead of file path.

• Deactivate stale docs when all files are removed from a collection and qmd update is run. #312 (thanks @0xble)

• Handle emoji-only filenames (🐘.md → 1f418.md) instead of crashing. #308 (thanks @debugerman)

• Skip unreadable files during indexing (e.g. iCloud-evicted files returning EAGAIN) instead of crashing. #253 (thanks @jimmynail)

• Suppress progress bar escape sequences when stderr is not a TTY. #230 (thanks @dgilperez)

• Emit format-appropriate empty output ([] for JSON, CSV header for CSV, etc.) instead of plain text "No results." #228 (thanks @amsminn)

• Correct Windows sqlite-vec package name (sqlite-vec-windows-x64) and add sqlite-vec-linux-arm64. #225 (thanks @ilepn)

• Fix claude plugin setup CLI commands in README. #311 (thanks @gi11es)

[1.1.1] - 2026-03-06

Fixes

• Reranker: truncate documents exceeding the 2048-token context window instead of silently producing garbage scores. Long chunks (e.g. from PDF ingestion) now get a fair ranking.

• Nix: add python3 and cctools to build dependencies. #214 (thanks @pcasaretto)

[1.1.0] - 2026-02-20

QMD now speaks in query documents — structured multi-line queries where every line is typed (lex:, vec:, hyde:), combining keyword precision with semantic recall. A single plain query still works exactly as before (it's treated as an implicit expand: and auto-expanded by the LLM). Lex now supports quoted phrases and negation ("C++ performance" -sports -athlete), making intent-aware disambiguation practical. The formal query grammar is documented in docs/SYNTAX.md.

The npm package now uses the standard #!/usr/bin/env node bin convention, replacing the custom bash wrapper. This fixes native module ABI mismatches when installed via bun and works on any platform with node >= 22 on PATH.

Changes

• Query document format: multi-line queries with typed sub-queries (lex:, vec:, hyde:). Plain queries remain the default (expand: implicit, but not written inside the document). First sub-query gets 2× fusion weight — put your strongest signal first. Formal grammar in docs/SYNTAX.md.

• Lex syntax: full BM25 operator support. "exact phrase" for verbatim matching; -term and -"phrase" for exclusions. Essential for disambiguation when a term is overloaded across domains (e.g. performance -sports -athlete).

• expand: shortcut: send a single plain query (or start the document with expand: on its only line) to auto-expand via the local LLM. Query documents themselves are limited to lex, vec, and hyde lines.

• MCP query tool (renamed from structured_search): rewrote the tool description to fully teach AI agents the query document format, lex syntax, and combination strategy. Includes worked examples with intent-aware lex.

• HTTP /query endpoint (renamed from /search; /search kept as silent alias).

• collections array filter: filter by multiple collections in a single query (collections: ["notes", "brain"]). Removed the single collection string param — array only.

• Collection include/exclude: includeByDefault: false hides a collection from all queries unless explicitly named via collections. CLI: qmd collection exclude <name> / qmd collection include <name>.

• Collection update-cmd: attach a shell command that runs before every qmd update (e.g. git stash && git pull --rebase --ff-only && git stash pop). CLI: qmd collection update-cmd <name> '<cmd>'.

• qmd status tips: shows actionable tips when collections lack context descriptions or update commands.

• qmd collection subcommands: show, update-cmd, include, exclude. Bare qmd collection now prints help.

• Packaging: replaced custom bash wrapper with standard #!/usr/bin/env node shebang on dist/qmd.js. Fixes native module ABI mismatches when installed via bun, and works on any platform where node >= 22 is on PATH.

• Removed MCP tools search, vector_search, deep_search — all superseded by query.

• Removed qmd context check command.

• CLI timing: each LLM step (expand, embed, rerank) prints elapsed time inline (Expanding query... (4.2s)).

Fixes

• qmd collection list shows [excluded] tag for collections with includeByDefault: false.

• Default searches now respect includeByDefault — excluded collections are skipped unless explicitly named.

• Fix main module detection when installed globally via npm/bun (symlink resolution).

[1.0.7] - 2026-02-18

Changes

• LLM: add LiquidAI LFM2-1.2B as an alternative base model for query expansion fine-tuning. LFM2's hybrid architecture (convolutions + attention) is 2x faster at decode/prefill vs standard transformers — good fit for on-device inference.

• CLI: support multiple -c flags to search across several collections at once (e.g. qmd search -c notes -c journals "query"). #191 (thanks @openclaw)

Fixes

• Return empty JSON array [] instead of no output when --json search finds no results.

• Resolve relative paths passed to --index so they don't produce malformed config entries.

• Respect XDG_CONFIG_HOME for collection config path instead of always using ~/.config. #190 (thanks @openclaw)

• CLI: empty-collection hint now shows the correct collection add command. #200 (thanks @vincentkoc)

[1.0.6] - 2026-02-16

Changes

• CLI: qmd status now shows models with full HuggingFace links instead of static names in --help. Model info is derived from the actual configured URIs so it stays accurate if models change.

• Release tooling: pre-push hook handles non-interactive shells (CI, editors) gracefully — warnings auto-proceed instead of hanging on a tty prompt. Annotated tags now resolve correctly for CI checks.

[1.0.5] - 2026-02-16

The npm package now ships compiled JavaScript instead of raw TypeScript, removing the tsx runtime dependency. A new /release skill automates the full release workflow with changelog validation and git hook enforcement.

Changes

• Build: compile TypeScript to dist/ via tsc so the npm package no longer requires tsx at runtime. The qmd shell wrapper now runs dist/qmd.js directly.

• Release tooling: new /release skill that manages the full release lifecycle — validates changelog, installs git hooks, previews release notes, and cuts the release. Auto-populates [Unreleased] from git history when empty.

• Release tooling: scripts/extract-changelog.sh extracts cumulative notes for the full minor series (e.g. 1.0.0 through 1.0.5) for GitHub releases. Includes [Unreleased] content in previews.

• Release tooling: scripts/release.sh renames [Unreleased] to a versioned heading and inserts a fresh empty [Unreleased] section automatically.

• Release tooling: pre-push git hook blocks v* tag pushes unless package.json version matches the tag, a changelog entry exists, and CI passed on GitHub.

• Publish workflow: GitHub Actions now builds TypeScript, creates a GitHub release with cumulative notes extracted from the changelog, and publishes to npm with provenance.

1.0.0 - 2026-02-15

QMD now runs on both Node.js and Bun, with up to 2.7x faster reranking through parallel GPU contexts. GPU auto-detection replaces the unreliable gpu: "auto" with explicit CUDA/Metal/Vulkan probing.

Changes

• Runtime: support Node.js (>=22) alongside Bun via a cross-runtime SQLite abstraction layer (src/db.ts). bun:sqlite on Bun, better-sqlite3 on Node. The qmd wrapper auto-detects a suitable Node.js install via PATH, then falls back to mise, asdf, nvm, and Homebrew locations.

• Performance: parallel embedding & reranking via multiple LlamaContext instances — up to 2.7x faster on multi-core machines.

• Performance: flash attention for ~20% less VRAM per reranking context, enabling more parallel contexts on GPU.

• Performance: right-sized reranker context (40960 → 2048 tokens, 17x less memory) since chunks are capped at ~900 tokens.

• Performance: adaptive parallelism — context count computed from available VRAM (GPU) or CPU math cores rather than hardcoded.

• GPU: probe for CUDA, Metal, Vulkan explicitly at startup instead of relying on node-llama-cpp's gpu: "auto". qmd status shows device info.

• Tests: reorganized into flat test/ directory with vitest for Node.js and bun test for Bun. New eval-bm25 and store.helpers.unit suites.

Fixes

• Prevent VRAM waste from duplicate context creation during concurrent embedBatch calls — initialization lock now covers the full path.

• Collection-aware FTS filtering so scoped keyword search actually restricts results to the requested collection.

0.9.0 - 2026-02-15

First published release on npm as @tobilu/qmd. MCP HTTP transport with daemon mode cuts warm query latency from ~16s to ~10s by keeping models loaded between requests.

Changes

• MCP: HTTP transport with daemon lifecycle — qmd mcp --http --daemon starts a background server, qmd mcp stop shuts it down. Models stay warm in VRAM between queries. #149 (thanks @igrigorik)

• Search: type-routed query expansion preserves lex/vec/hyde type info and routes to the appropriate backend. Eliminates ~4 wasted backend calls per query (10.0 → 6.0 calls, 1278ms → 549ms). #149 (thanks @igrigorik)

• Search: unified pipeline — extracted hybridQuery() and vectorSearchQuery() to store.ts so CLI and MCP share identical logic. Fixes a class of bugs where results differed between the two. #149 (thanks @igrigorik)

• MCP: dynamic instructions generated at startup from actual index state — LLMs see collection names, doc counts, and content descriptions. #149 (thanks @igrigorik)

• MCP: tool renames (vsearch → vector_search, query → deep_search) with rewritten descriptions for better tool selection. #149 (thanks @igrigorik)

• Integration: Claude Code plugin with inline status checks and MCP integration. #99 (thanks @galligan)

Fixes

• BM25 score normalization — formula was inverted (1/(1+|x|) instead of |x|/(1+|x|)), so strong matches scored lowest. Broke --min-score filtering and made the "strong signal" short-circuit dead code. #76 (thanks @dgilperez)

• Normalize Unicode paths to NFC for macOS compatibility. #82 (thanks @c-stoeckl)

• Handle dense content (code) that tokenizes beyond expected chunk size.

• Proper cleanup of Metal GPU resources on process exit.

• SQLite-vec readiness verification after extension load.

• Reactivate deactivated documents on re-index instead of creating duplicates.

• Bun UTF-8 path corruption workaround for non-ASCII filenames.

• Disable following symlinks in glob.scan to avoid infinite loops.

[0.8.0] - 2026-01-28

Fine-tuned query expansion model trained with GRPO replaces the stock Qwen3 0.6B. The training pipeline scores expansions on named entity preservation, format compliance, and diversity — producing noticeably better lexical variations and HyDE documents.

Changes

• LLM: deploy GRPO-trained (Group Relative Policy Optimization) query expansion model, hosted on HuggingFace and auto-downloaded on first use. Better preservation of proper nouns and technical terms in expansions.

• LLM: /only:lex mode for single-type expansions — useful when you know which search backend will help.

• LLM: HyDE output moved to first position so vector search can start embedding while other expansions generate.

• LLM: session lifecycle management via withLLMSession() pattern — ensures cleanup even on failure, similar to database transactions.

• Integration: org-mode title extraction support. #50 (thanks @sh54)

• Integration: SQLite extension loading in Nix devshell. #48 (thanks @sh54)

• Integration: AI agent discovery via skills.sh. #64 (thanks @Algiras)

Fixes

• Use sequential embedding on CPU-only systems — parallel contexts caused a race condition where contexts competed for CPU cores, making things slower. #54 (thanks @freeman-jiang)

• Fix collectionName column in vector search SQL (was still using old collectionId from before YAML migration). #61 (thanks @jdvmi00)

• Fix Qwen3 sampling params to prevent repetition loops — stock temperature/top-p caused occasional infinite repeat patterns.

• Add --index option to CLI argument parser (was documented but not wired up). #84 (thanks @Tritlo)

• Fix DisposedError during slow batch embedding. #41 (thanks @wuhup)

[0.7.0] - 2026-01-09

First community contributions. The project gained external contributors, surfacing bugs that only appear in diverse environments — Homebrew sqlite-vec paths, case-sensitive model filenames, and sqlite-vec JOIN incompatibilities.

Changes

• Indexing: native realpathSync() replaces readlink -f subprocess spawn per file. On a 5000-file collection this eliminates 5000 shell spawns, ~15% faster. #8 (thanks @burke)

• Indexing: single-pass tokenization — chunking algorithm tokenized each document twice (count then split); now tokenizes once and reuses. #9 (thanks @burke)

Fixes

• Fix vsearch and query hanging — sqlite-vec's virtual table doesn't support the JOIN pattern used; rewrote to subquery. #23 (thanks @mbrendan)

• Fix MCP server exiting immediately after startup — process had no active handles keeping the event loop alive. #29 (thanks @mostlydev)

• Fix collection filter SQL to properly restrict vector search results.

• Support non-ASCII filenames in collection filter.

• Skip empty files during indexing instead of crashing on zero-length content.

• Fix case sensitivity in Qwen3 model filename resolution. #15 (thanks @gavrix)

• Fix sqlite-vec loading on macOS with Homebrew (BREW_PREFIX detection). #42 (thanks @komsit37)

• Fix Nix flake to use correct src/qmd.ts path. #7 (thanks @burke)

• Fix docid lookup with quotes support in get command. #36 (thanks @JoshuaLelon)

• Fix query expansion model size in documentation. #38 (thanks @odysseus0)

[0.6.0] - 2025-12-28

Replaced Ollama HTTP API with node-llama-cpp for all LLM operations. Ollama adds convenience but also a running server dependency. node-llama-cpp loads GGUF models directly in-process — zero external dependencies. Models auto-download from HuggingFace on first use.

Changes

• LLM: structured query expansion via JSON schema grammar constraints. Model produces typed expansions — lexical (BM25 keywords), vector (semantic rephrasings), HyDE (hypothetical document excerpts) — so each routes to the right backend instead of sending everything everywhere.

• LLM: lazy model loading with 2-minute inactivity auto-unload. Keeps memory low when idle while avoiding ~3s model load on every query.

• Search: conditional query expansion — when BM25 returns strong results, the expensive LLM expansion is skipped entirely.

• Search: multi-chunk reranking — documents with multiple relevant chunks scored by aggregating across all chunks rather than best single chunk.

• Search: cosine distance for vector search (was L2).

• Search: embeddinggemma nomic-style prompt formatting.

• Testing: evaluation harness with synthetic test documents and Hit@K metrics for BM25, vector, and hybrid RRF.

[0.5.0] - 2025-12-13

Collections and contexts moved from SQLite tables to YAML at ~/.config/qmd/index.yml. SQLite was overkill for config — you can't share it, and it's opaque. YAML is human-readable and version-controllable. The migration was extensive (35+ commits) because every part of the system that touched collections or contexts had to be updated.

Changes

• Config: YAML-based collections and contexts replace SQLite tables. collections and path_contexts tables dropped from schema. Collections support an optional update: command (e.g., git pull) before re-index.

• CLI: qmd collection add/list/remove/rename commands with --name and --mask glob pattern support.

• CLI: qmd ls virtual file tree — list collections, files in a collection, or files under a path prefix.

• CLI: qmd context add/list/check/rm with hierarchical context inheritance. A query to qmd://notes/2024/jan/ inherits context from notes/, notes/2024/, and notes/2024/jan/.

• CLI: qmd context add / "text" for global context across all collections.

• CLI: qmd context check audit command to find paths without context.

• Paths: qmd:// virtual URI scheme for portable document references. qmd://notes/ideas.md works regardless of where the collection lives on disk. Works in get, multi-get, ls, and context commands.

• CLI: document IDs (docid) — first 6 chars of content hash for stable references. Shown as #abc123 in search results, usable with get and multi-get.

• CLI: --line-numbers flag for get command output.

[0.4.0] - 2025-12-10

MCP server for AI agent integration. Without it, agents had to shell out to qmd search and parse CLI output. The monolithic qmd.ts (1840 lines) was split into focused modules with the project's first test suite (215 tests).

Changes

• MCP: stdio server with tools for search, vector search, hybrid query, document retrieval, and status. Runs over stdio transport for Claude Desktop and MCP clients.

• MCP: spec-compliant with June 2025 MCP specification — removed non-spec mimeType, added isError: true to errors, structuredContent for machine-readable results, proper URI encoding.

• MCP: simplified tool naming (qmd_search → search) since MCP already namespaces by server.

• Architecture: extract store.ts (1221 LOC), llm.ts (539 LOC), formatter.ts (359 LOC), mcp.ts (503 LOC) from monolithic qmd.ts.

• Testing: 215 tests (store: 96, llm: 60, mcp: 59) with mocked Ollama for fast, deterministic runs. Before this: zero tests.

[0.3.0] - 2025-12-08

Document chunking for vector search. A 5000-word document about many topics gets a single embedding that averages everything together, matching poorly for specific queries. Chunking produces one embedding per ~900-token section with focused semantic signal.

Changes

• Search: markdown-aware chunking — prefers heading boundaries, then paragraph breaks, then sentence boundaries. 15% overlap between chunks ensures cross-boundary queries still match.

• Search: multi-chunk scoring bonus (+0.02 per additional chunk, capped at +0.1 for 5+ chunks). Documents relevant in multiple sections rank higher.

• CLI: display paths show collection-relative paths and extracted titles (from H1 headings or YAML frontmatter) instead of raw filesystem paths.

• CLI: --all flag returns all matches (use with --min-score to filter).

• CLI: byte-based progress bar with ETA for embed command.

• CLI: human-readable time formatting ("15m 4s" instead of "904.2s").

• CLI: documents >64KB truncated with warning during embedding.

[0.2.0] - 2025-12-08

Changes

• CLI: --json, --csv, --files, --md, --xml output format flags. --json for programmatic access, --files for piping, --md/--xml for LLM consumption, --csv for spreadsheets.

• CLI: qmd status shows index health — document count, size, embedding coverage, time since last update.

• Search: weighted RRF — original query gets 2x weight relative to expanded queries since the user's actual words are a more reliable signal.

[0.1.0] - 2025-12-07

Initial implementation. Built in a single day for searching personal markdown notes, journals, and meeting transcripts.

Changes

• Search: SQLite FTS5 with BM25 ranking. Chose SQLite over Elasticsearch because QMD is a personal tool — single binary, no server dependencies.

• Search: sqlite-vec for vector similarity. Same rationale: in-process, no external vector database.

• Search: Reciprocal Rank Fusion to combine BM25 and vector results. RRF is parameter-free and handles missing signals gracefully.

• LLM: Ollama for embeddings, reranking, and query expansion. Later replaced with node-llama-cpp in 0.6.0.

• CLI: qmd add, qmd embed, qmd search, qmd vsearch, qmd query, qmd get. ~1800 lines of TypeScript in a single qmd.ts file.