Create Context Graph

CLAUDE.md — Create Context Graph

Project Overview

Interactive CLI scaffolding tool that generates domain-specific context graph applications. Like create-next-app but for AI agents with graph memory. Invoked via uvx create-context-graph or npx create-context-graph.

Given a domain (e.g., "healthcare", "wildlife-management") and an agent framework (e.g., PydanticAI, Claude Agent SDK), it generates a complete full-stack application: FastAPI backend, Next.js + Chakra UI v3 + NVL frontend, Neo4j schema, synthetic data, and a configured AI agent with domain-specific tools.

Status: v0.14.0. Community PR hardening release — integrates PRs #52/#56/#58/#59/#60, closes the gaps found reviewing them, and fixes five breaks found by live-testing the full NAMS flow against the production service with neo4j-agent-memory 0.5.0: (1) conversation memory was silently failing on every message — the NAMS service only accepts messages addressed to conversation ids IT minted, so generated memory.py now creates conversations per session and targets the server id (_resolve_nams_conversation()), and store_message() treats MemoryIntegration's swallowed {"error": ...} returns as failures; (2) every document/body ingest write failed (role="document" rejected — only user/assistant/system allowed — plus the conversation-404), fixed across all three NAMS ingest implementations (run_nams_ingest, scaffolded import_data.py, make seed's ingest_fixtures_nams) with role="user" + metadata.kind markers and server-created channels; (3) /api/documents and /api/schema/visualization returned nothing (the live service coerces OBJECT/EVENT entity types to custom and rejects empty search queries) — both adapters are now cypher-first via the _pole_type: OBJECT_ description marker / type-count aggregation with search fallbacks; (4) graph expand + entity connections were dead (long_term.get_entity(id) doesn't exist in 0.5.x) — expand_node_nams/get_entity_detail_nams resolve through the cypher API; (5) NAMS reset never deleted anything (delete_entity doesn't exist upstream; the old loop swallowed the AttributeError and printed "0 removed") — --reset-database and make reset now honestly report that no delete API exists and point at the NAMS dashboard. NAMS domain ontology activation (new): NAMS pre-registers a server-side ontology for every bundled domain but auto-binds workspaces to nams-default until one is activated — generated apps now bind on connect_memory() (activate catalog match by domain id, or create-from-backend/app/ontology_document.json + activate for custom domains, best-effort via _ensure_nams_ontology()), the CLI ingest and scaffolded import_data.py run the same stage-0 sequence (parity-pinned), and build_nams_ontology_document() in ontology.py produces the server document shape ({domain, entity_types, relationships}) used by the renderer and ingest; verified live — workspace flips nams-default→healthcare on first connect, custom test-domain creates+activates server-side; stored entity type still coerces to {Person,Organization,Location,custom} server-side regardless (activation governs version stamp + extraction vocabulary). Verified live: 19/19 API checks, clean document ingest, working chat-memory writes, cypher reads (writes rejected server-side), trace ingest OK (list_traces NotSupported → /traces degrades to empty), and add_relationship still NotSupported so ccg-edges remains the design. New --neo4j-database / NEO4J_DATABASE for the self-hosted backend, threaded end-to-end: CLI flag, wizard prompt, Aura .env import (explicit flag wins), generated .env + .env.example, Settings.neo4j_database, MemorySettings (key omitted when blank so the SDK default neo4j applies), execute_cypher() sessions, scaffold-time ingest.py (_ingest_with_memory_client/_ingest_with_driver/reset_neo4j), the scaffolded import_data.py bolt session, and validate_connection(database=). Live memory-write failures now surface: store_message() records into the classified error state and the bolt /health gains memory/memory_error/memory_error_detail (startup lifecycle checks get_client() instead of assuming success). execute_cypher() dispatches to NAMS client.query.cypher (with _coerce_nams_records() shape coercion) so agent tools and POST /cypher work on the hosted backend; _require_neo4j() 503s when the NAMS client is missing. --ontology-file is implemented (was documented-only, issue #50): scaffolds from a hand-written YAML, copies it to data/ontology.yaml, mutually exclusive with --custom-domain, works through the wizard path too. load_domain() resolves custom domains from ~/.create-context-graph/custom-domains/ with a declared-domain.id fallback (issue #30); bundled domains shadow same-id customs; the test suite is hermetic against that directory via an autouse conftest fixture. Playwright spec template survives missing demo_scenarios AND a scenario with empty prompts. Schema-DDL splitter bug fixed: the old split(";") + skip-// pattern silently dropped 5 real statements behind comment headers (person_name, document_title, document_domain, document_name_unique, local_file_fulltext were never created by seeding) and executed a comment tail as Cypher; new shared split_cypher_statements() in ontology.py is used by ingest.py, the generated generate_data.py, and test_integration.py. New test surfaces: tests/test_generated_client_runtime.py (26 tests executing the rendered client/memory modules against doubles), +6 app-level route tests, +12 CLI tests, +5 wizard tests, +17 template pins, +9 ontology tests, +3 integration tests, generated test_routes.py gains 2 backend-specific /health tests, test_generated_tests.py runs one bolt scaffold, and e2e_smoke_test.py asserts the /health contract shape. Carry-forward from v0.13.1 (feedback-report triage): create-context-graph --dry-run no longer demands a NAMS API key (the credential gate is now scoped to non-dry-run flows in cli.py:392). Dead template_id parameter dropped from list_documents_nams (NAMS branch already raises 501). Generated Pydantic app/models.py now emits Field(...) for required fields instead of bare = ... Ellipsis literal. New GET /schema/models endpoint introspects app.models at runtime and returns the JSON Schema for each entity model — makes the previously-unused models.py load-bearing. Last key={\${e.name}-${i}`}site (DocumentBrowser mentioned-entities) replaced withkey={`${selectedDoc.document.title}-${e.name}`}. New regression tests (TestCompositeKeyRegressionsintest_frontend_logic.pyplusTestV0131ModelsPolishandTestV0131TemplateIdRemovalintest_generated_project.py) plus 3 new Playwright tests pin the v0.12.0/v0.13.0 fixes. The v0.13.0 report's claims about connector code being removed from scaffolded projects, pyproject.toml bloat on NAMS, and missing domains (media, insurance, supply-chain) were verified-false against current code — see CHANGELOG.md"Not Changed" section. **Carry-forward from v0.13.0 (v0.12.0 feedback-report fixes):** Fixed the bolt async/sync mismatch in the scaffoldedimport_data.py (ingest_via_bolt()isasync defand both call sites wrap withasyncio.run); ChatInterface stale-closure bug fixed by moving streamingEntities/streamingPreferencesto refs;list_documents_namspushes the OBJECT filter server-side;externalInputuseEffect depends onloading. Removed the --framework maf alias entirely (Click rejects with "Invalid value"). Restored 4 domains as YAML definitions (legal, education, cybersecurity, government); domain count is now 27. New Docusaurus page docs/docs/explanation/ccg-edges.mddocuments the relationship-encoding strategy. **Carry-forward from v0.12.0 (NAMS-native connector ingest release):** Default memory backend is the hosted Neo4j Agent Memory Service (NAMS,MEMORY_API_KEYenv), with--self-hostedpreserving the bolt-Neo4j path. **LiteLLM provider injection** for memory layer viaMEMORY_LLM/MEMORY_EMBEDDINGenv vars (LiteLLM-style strings) routes through native adapters when available, LiteLLM otherwise. Default framework is **AWS Strands**. Wizard collapsed from 11 prompts to 6 with autocomplete domain picker and a single "Customize advanced settings" gate. NAMS ingest now uses a **hybrid write shape**: entities viaadd_entitywith non-name properties markdown-serialized into thedescriptionfield; **outbound relationships encoded into the source entity'sdescriptionas a fenced ```ccg-edges``` YAML block** (sorted bytypethentarget) since NAMS REST has no add_relationshipyet — the frontend parses these out for the graph view, and a future migration will replay them as native edges; **documents dual-tracked** aslong_term.add_entity(type=OBJECT)(queryable source of truth, matches bolt:Documentshape) ANDshort_term.add_message(role="document")(extraction fuel); **entity bodies routed throughadd_message** when the connector declares a BODY_FIELDS: dict[label, property]map (e.g.Comment.bodyfor Linear,Message.contentfor Claude AI/ChatGPT/Claude Code,DecisionThread.content/Reply.contentfor Google Workspace,Document.description/Section.descriptionfor local-file); decision traces viareasoning.start_trace/add_step/complete_trace. Preferences and facts still unsupported by NAMS REST (logged-skipped). Schema DDL skipped on NAMS (server-owned). The two ingest consumers (src/create_context_graph/ingest.py run_nams_ingest()and the scaffoldedtemplates/backend/connectors/import_data.py.j2) share the same NAMS write sequence pinned by tests/test_nams_ingest_parity.py(~526 LOC contract test). Generatedimport_data.pyis **idempotent**: per-connector watermarks in.context-graph/watermarks.json(re-runs fetch deltas only), failures appended to.context-graph/deadletter.jsonl, new --dry-run(fetch only) and--retry(drain deadletter) modes, surfaced viamake import(fetch + ingest),make import-dry-run, make import-retry(the legacymake import-and-seedtarget is gone —make importis the single entrypoint)./documents?template_id=...returns HTTP 501 on NAMS (template filtering needsMENTIONSedges that NAMS doesn't have); un-filtered/documentsworks on both backends. Bolt ingest hardened withrequire_safe_cypher_identifier()(relationship types and labels validated against[A-Za-z][A-Za-z0-9]*before string interpolation) and labeledMATCH (a:SourceLabel)/MATCH (b:TargetLabel)(no more label-less fallback that could mis-merge).ingest_data()library entry point now accepts either aProjectConfigor the legacy(neo4j_uri, neo4j_username, neo4j_password) triple (v0.11.0 had broken legacy callers). Frontend routes (/expand, /documents, /traces, /schema/visualization, /entities/{name}, /search) dispatch through memory_adapter.pyon NAMS. MCP profile coerced tocoreon NAMS (extended-profile tools rely on unsupported endpoints).auto_preferencesforcedFalseon NAMS. Generatedpyproject.tomlpinsneo4j-agent-memory[litellm,sentence-transformers]>=0.4.0,<0.6.0; self-hosted scaffolds also pull [extraction,fuzzy]. 27 domains, 8 agent frameworks (7 working out of box — openai-agents requires OPENAI_API_KEY, google-adk requires GOOGLE_API_KEY with clear warnings), **streaming chat via Server-Sent Events** (token-by-token text in 6 frameworks + real-time tool call visualization with Timeline/Spinner/Collapsible + entities_extractedandpreferences_detected SSE events displayed as badges), neo4j-agent-memory v0.1.0 MemoryIntegration for multi-turn conversations with automatic entity extraction and preference detection (local sentence-transformers embeddings by default — no OpenAI key required; auto-upgrades to OpenAI embeddings if OPENAI_API_KEY is set), MCP server generation for Claude Desktop (--with-mcpflag,make mcp-server target, dual-interface architecture), configurable session strategies (per_conversation/per_day/persistentviaSESSION_STRATEGYenv var), interactive NVL graph visualization (schema view, double-click expand, drag/zoom, property panel, agent-driven graph updates — now updated incrementally during streaming, node hover tooltips, "Ask about this" button), LLM-generated demo data (80-90 entities, 25+ documents, 8-12 decision traces per domain) with post-generation value clamping for 28 property types, markdown rendering in chat and document browser with ReactMarkdown, document browser with pagination, entity detail panel, decision trace viewer, 13 SaaS connectors (including Linear for real project data import, Google Workspace for decision trace extraction from comment threads, Claude Code for local session history import with decision/preference extraction, Claude AI + ChatGPT for importing conversation exports from AI chat platforms, and local-file for deterministic ingestion of local Markdown / PDF / HTML / AsciiDoc / Word documents into Document → Section hierarchies), custom domain generation, Neo4j Aura .env import + neo4j-local support, Docusaurus documentation site (24 pages including quick-start, domain catalog, framework comparison, Neo4j Aura/Local guides, Docker guide, architecture diagram, "Why Context Graphs?" explainer, Google Workspace tutorial, decision trace explainer, GWS schema reference, chat history import tutorial, chat import schema reference, andccg-edgesencoding explainer), graceful Neo4j degradation with /health endpoint (retry with backoff on initial load) and 503 guards on all endpoints, Cypher injection prevention, enum identifier sanitization, configurable CORS/model/timeouts, --dry-run/--verbose/--reset-database/--demo CLI flags, CLI auto-slug generation (PROJECT_NAME optional in non-interactive mode), CLI warnings for framework-specific API key requirements (openai-agents, google-adk), constants module, WCAG accessibility improvements, chat timeout/cancel with AbortController, mobile-responsive layout, .dockerignore for Docker builds,make test-connectiontarget, framework-specific README sections, troubleshooting guide, thread-safe async bridging for sync frameworks (CrewAI/Strands), bounded agentic loops (max 15 iterations), domain-specific static name pools (1000+ names across 118 entity labels — all domain YAML labels covered) with domain-aware base entities, tool-use emphasis in all agent system prompts, domain-scoped chat history localStorage keys, SSR hydration fix, retry button on chat errors, PydanticAI tool serialization fix (JSON string return types), Google ADK API key support (--google-api-key flag) with AttributeError guard for SDK cleanup, Strands robust text extraction, CrewAI explicit Anthropic LLM config withcrewai[anthropic] dependency, domain-scoped MERGE keys ({name, domain}) for cross-domain isolation when sharing a Neo4j instance, improved static data quality (26 domain-specific industry pools, POLE-type-aware entity descriptions with 5 label categories + 7 label-specific overrides, entity-derived document titles, realistic decision trace observations, 20+ domain-specific property pools, float value clamping, taxonomy class correction), agent thinking text collapsible filter with continuation pattern support, Cypher query validation tests across all domains, fixture cross-validation tests (schema alignment + data quality), proper Document/DecisionTrace node ingestion via --ingest, Chakra UI Pro-inspired chat input redesign, list/get-by-id agent tools for all 27 domains (7-8+ tools per domain), ON CREATE/MATCH SET for constraint-safe seeding, hardened Linear connector (named constants, structured logging, URLError/JSONDecodeError/429 handling with retry, pagination safety limits, null-safe field access, team key validation in authenticate(), incremental sync via updated_after, decision traces in generated template), optional credential prompts in interactive wizard, Google Workspace connector with decision trace extraction from 6 Google APIs (Drive Files, Comments, Revisions, Activity, Calendar, Gmail) with 10 decision-focused agent tools and cross-connector Linear linking, scaffolded Claude Code connector with full entity extraction (9 entity types, 14 relationship types, secret redaction, decision/preference extraction, language detection), connector-specific demo scenarios, 1,454 passing tests in the fast suite; the full CI suite (dev+connectors extras, --slow --functional`) runs 1,866 passing / 1,881 collected.

Quick Reference

# Setup
uv venv && uv pip install -e ".[dev]"

# Run tests
source .venv/bin/activate && pytest tests/ -v

# Default scaffold (NAMS-backed, empty graph — agent populates via chat)
create-context-graph my-app --domain healthcare --framework strands --nams-api-key sk-...

# Self-hosted scaffold (bolt Neo4j, full demo fixtures)
create-context-graph my-app --domain financial-services --framework pydanticai --self-hosted --demo-data

# List available domains
create-context-graph --list-domains

Architecture

src/create_context_graph/
├── cli.py              # Click CLI entry point (interactive + flag modes)
├── wizard.py           # 7-step Questionary interactive wizard
├── config.py           # ProjectConfig Pydantic model
├── ontology.py         # YAML domain ontology loader, validation, code generation
├── custom_domain.py    # LLM-powered custom domain YAML generation
├── renderer.py         # Jinja2 template engine (renders project scaffold)
├── generator.py        # LLM-powered synthetic data pipeline (4 stages)
├── name_pools.py       # Realistic name pools and value generators for static fallback
├── ingest.py           # Neo4j ingestion via neo4j-agent-memory or direct driver
├── neo4j_validator.py  # Neo4j connection testing
├── connectors/         # SaaS data connectors (10 services)
│   ├── __init__.py     # BaseConnector ABC, NormalizedData model, registry
│   ├── github_connector.py
│   ├── notion_connector.py
│   ├── jira_connector.py
│   ├── slack_connector.py
│   ├── gmail_connector.py   # Prefers gws CLI, falls back to Python OAuth2
│   ├── gcal_connector.py    # Prefers gws CLI, falls back to Python OAuth2
│   ├── salesforce_connector.py
│   └── oauth.py        # Shared OAuth2 flow + gws CLI helpers
├── domains/            # 23 YAML ontology definitions + _base.yaml
├── fixtures/           # 22 pre-generated JSON fixture files
└── templates/          # Jinja2 templates for generated projects
    ├── base/           # .env, .env.example, Makefile, docker-compose, README, .gitignore
    ├── backend/
    │   ├── shared/     # FastAPI main, config, neo4j client, GDS, vector, models, routes
    │   ├── agents/     # Per-framework agent.py (8 frameworks)
    │   └── connectors/ # Generated connector modules + import_data.py
    ├── frontend/       # Next.js + Chakra UI v3 + NVL components
    └── cypher/         # Schema constraints + GDS projections

Key Design Decisions

Templates are domain-agnostic, data-driven

No per-domain template directories. The ontology YAML drives all domain customization via Jinja2 context variables. Only backend/agents/{framework}/agent.py.j2 varies by framework — everything else is shared.

Jinja2 + JSX/Python escaping

Templates that contain JSX curly braces or Python dict literals must use {% raw %}...{% endraw %} blocks to avoid conflicts with Jinja2's {{ }} syntax. Break out of raw mode only for actual Jinja2 substitutions:

{% raw %}JSX code with {curly} braces{% endraw %}{{ jinja_var }}{% raw %}more JSX{% endraw %}

Two-layer ontology inheritance

_base.yaml defines shared POLE+O entity types (Person, Organization, Location, Event, Object). Domain YAMLs declare inherits: _base and add domain-specific entity types. The ontology.py loader merges base entities/relationships into each domain.

Rich fixture data pipeline

• Pre-generated fixtures (shipped): All 27 domains ship with high-quality LLM-generated fixture data (80-90 entities, 160-280 relationships, 25+ documents at 200-1600 words, 3-5 decision traces with multi-step reasoning). Generated via scripts/regenerate_fixtures.py using Claude API.

• Static fallback (no LLM key at runtime): Uses domain-specific name pools (name_pools.py) with 1000+ names across 118 entity labels (all domain YAML labels covered — no more fallback to generic "Label 1" names), label-aware ID prefixes (PAT- for Patient, ACT- for Account), contextual property generators (emails from names, realistic IDs, domain-appropriate ranges), 26 domain-specific industry pools (DOMAIN_INDUSTRY_POOL), POLE-type-aware entity descriptions (_generate_description with 5 label categories: _PERSON_LABELS, _ORGANIZATION_LABELS, _LOCATION_LABELS, _EVENT_LABELS, _OBJECT_LABELS + 7 label-specific override templates), entity-derived document titles (reference primary entities instead of sequential numbers), Markdown-formatted document content, 12+ domain-specific property pools (currency codes, ticker symbols, drug classes, medical specialties, severities, etc.), float clamping for confidence/rating/efficiency fields, and realistic decision trace observations (reference entity names).

• Post-generation validation (_validate_and_clamp() in generator.py): LLM-generated entities are post-processed to clamp unrealistic numeric values (28 property range rules) and fix taxonomy class mismatches (species → correct class mapping).

• LLM-powered (with --anthropic-api-key at runtime): Generates realistic entities, documents, and decision traces via Anthropic or OpenAI APIs.

• Data seeding (make seed): Loads all four data types into Neo4j — entities, relationships, documents (as :Document nodes with :MENTIONS links to entities), and decision traces (as :DecisionTrace → :HAS_STEP → :TraceStep chains).

Dual ingestion backends

ingest.py tries neo4j-agent-memory MemoryClient first, falls back to direct neo4j driver if the package isn't installed. Both paths create entities via MemoryClient or direct Cypher, plus create :Document and :DecisionTrace/:TraceStep nodes using direct Cypher (matching the generate_data.py.j2 schema the frontend queries expect). Both paths tag all entities with a domain property for cross-domain isolation when sharing a Neo4j instance.

Custom domain generation

custom_domain.py generates complete domain ontology YAMLs from natural language descriptions using LLM (Anthropic/OpenAI). Uses _base.yaml + 2 reference domain YAMLs as few-shot examples. Validates output against DomainOntology Pydantic model with retry loop (up to 3 attempts). Generated domains can be saved to ~/.create-context-graph/custom-domains/ for reuse.

Streaming chat via Server-Sent Events (SSE)

The chat uses a streaming architecture where the backend emits SSE events as the agent executes. The POST /chat/stream endpoint creates an asyncio.Queue on the CypherResultCollector and launches the agent in a background task. As tools execute, the collector emits tool_start and tool_end events (with graph data). For frameworks that support it (PydanticAI, Anthropic Tools, Claude Agent SDK, OpenAI Agents, LangGraph), text_delta events stream tokens as they arrive. Other frameworks (CrewAI, Strands, Google ADK) emit tool events in real-time with text arriving at the end. Two additional event types support memory feedback: entities_extracted (emitted by CypherResultCollector.emit_entities_extracted() when entities are extracted from stored messages) and preferences_detected (emitted by CypherResultCollector.emit_preferences_detected() when preferences are detected). These are displayed as badges in the frontend. The frontend parses SSE via fetch + ReadableStream (not EventSource, which only supports GET). Tool calls render as a Chakra UI Timeline with live Spinner indicators. Text deltas are batched (~50ms) before updating ReactMarkdown to avoid excessive re-renders. Graph visualization updates incrementally after each tool_end event. The original /chat endpoint is preserved for backward compatibility. The SSE endpoint has a 120s per-event idle timeout and a 300s overall timeout.

Thread-safe async bridging for sync frameworks

CrewAI and Strands are synchronous frameworks that run in worker threads via asyncio.to_thread(). Their tools need to call async execute_cypher(). The _run_sync() helper uses asyncio.run_coroutine_threadsafe() to schedule coroutines on the main event loop (captured via _capture_loop() before to_thread), with a 30s timeout. The CypherResultCollector._push_event() is also thread-safe — it detects worker threads and uses loop.call_soon_threadsafe() instead of direct put_nowait(). Google ADK uses nest_asyncio for same-thread reentrant calls with a cross-thread run_coroutine_threadsafe fallback.

Bounded agentic loops

Claude Agent SDK and Anthropic Tools use agentic while loops that process tool calls until the model stops. These loops are bounded to 15 iterations (for _iteration in range(max_iterations)) with a 60s timeout on each API call. If max iterations is exceeded, a fallback message is returned instead of hanging indefinitely.

Tool-use emphasis in system prompts

All 8 agent templates append a tool-use emphasis suffix to the domain system prompt: "IMPORTANT: You MUST use the available tools to query the knowledge graph before answering any question about the data." This ensures agents consistently invoke their tools rather than generating ungrounded answers.

Agent tool return type convention

All agent tools must return str (JSON-serialized via json.dumps(result, default=str)), not raw list[dict]. PydanticAI, CrewAI, Strands, Google ADK, and other frameworks need to serialize tool outputs to send them back to the LLM. Raw Neo4j Node/Relationship objects cause silent serialization failures. The default=str handler ensures Neo4j-specific types (datetime, spatial) serialize correctly.

Agent thinking text filter

The frontend ChatInterface includes a splitThinkingAndResponse() function that detects agent "thinking" patterns (lines starting with "Let me", "I'll", "First, I need to", etc.) and renders them in a collapsible "Show reasoning" section. CONTINUATION_PATTERNS catch multi-sentence thinking blocks (lines starting with "and", "also", "then", "so", "because", etc.) that follow an initial thinking line. Lines with markdown formatting (#, -, *, |) break out of thinking mode. This keeps the primary response focused while making the full reasoning chain available on demand.

Interactive graph visualization with agent integration

The frontend ContextGraphView starts in schema view (calls db.schema.visualization() via GET /schema/visualization) showing entity types as nodes and relationship types as edges. When the user interacts with the agent chat, tool call results flow to the graph automatically via the CypherResultCollector in context_graph_client.py. In streaming mode, graph data arrives incrementally with each tool_end SSE event — the ChatInterface calls onGraphUpdate for each tool completion, so the graph updates as each tool finishes rather than all at once. The page.tsx passes data to ContextGraphView as externalGraphData. Double-clicking a schema node loads instances of that label; double-clicking a data node calls POST /expand to fetch neighbors (deduplicated merge). NVL uses d3Force layout with drag/zoom/pan, click for property details, and canvas click to deselect. Nodes have hover tooltips showing full name, labels, and top 5 properties. Clicking a node reveals an "Ask about [entity]" button that sends a chat query via the onAskAbout → externalInput → ChatInterface callback chain.

neo4j-agent-memory integration

Generated projects use MemoryIntegration from neo4j-agent-memory v0.4+ for conversation memory with automatic entity extraction (and, on the bolt backend, preference detection). The memory.py.j2 template generates a shared backend/app/memory.py module with connect_memory(), close_memory(), store_message(), get_context(), and resolve_session_id() functions. context_graph_client.py.j2 delegates memory lifecycle to this module via from app.memory import connect_memory, close_memory. The store_message() return value includes extracted entities and detected preferences, which are emitted as SSE events (entities_extracted, preferences_detected) and displayed as badges in the frontend. Session identity is configurable via SESSION_STRATEGY env var: per_conversation (default), per_day, or persistent. The generated pyproject.toml pins neo4j-agent-memory>=0.4.0,<0.6.0 with extras conditional on the backend: NAMS scaffolds get [litellm,sentence-transformers] (extraction happens server-side); self-hosted bolt scaffolds also pull [extraction,fuzzy] for local spaCy/GLiNER + rapidfuzz. All 8 agent frameworks import from app.memory and call resolve_session_id(), store_message(), and get_context() (which returns messages + entities + preferences + traces). The frontend ChatInterface captures session_id from the first SSE response and sends it in all subsequent requests.

MCP server generation

Generated projects optionally include an MCP (Model Context Protocol) server configuration for Claude Desktop. When --with-mcp is passed (or selected in the wizard), the scaffold generates mcp/claude_desktop_config.json (pre-configured for the project's Neo4j instance), mcp/README.md (setup instructions), and a make mcp-server Makefile target. The MCP server runs neo4j-agent-memory's built-in MCP server with 16 tools (extended profile) or 6 tools (core profile, via --mcp-profile core). This creates a dual-interface architecture: the web app and Claude Desktop both query the same knowledge graph.

Neo4j driver serialization

context_graph_client.py.j2 uses a custom _serialize() function instead of the driver's .data() method. This preserves Neo4j Node metadata (elementId, labels), Relationship metadata (elementId, type, startNodeElementId, endNodeElementId), and Path expansion. Without this, the frontend graph visualization and agent tools receive flat property dicts with no type information.

SaaS connectors

connectors/ package with 13 service connectors (GitHub, Notion, Jira, Slack, Gmail, Google Calendar, Salesforce, Linear, Google Workspace, Claude Code, Claude AI, ChatGPT, local-file). Each connector implements BaseConnector ABC with authenticate(), fetch(), and get_credential_prompts() methods. Credential prompts support an optional flag — the wizard skips blank optional fields instead of aborting. Returns NormalizedData matching the fixture schema so ingest.py works unchanged. Gmail/Google Calendar prefer the Google Workspace CLI (gws) if available, with Python OAuth2 fallback. The Linear connector uses GraphQL with cursor-based pagination and rate limiting against https://api.linear.app/graphql, mapping 12 entity types (Issue, Project, Cycle, Team, Person, Label, WorkflowState, Comment, ProjectUpdate, ProjectMilestone, Initiative, Attachment) to the POLE+O entity model with 26 relationship types. Imports issue relations (BLOCKS, BLOCKED_BY, RELATED_TO, DUPLICATE_OF), threaded comments with resolution tracking (REPLY_TO, RESOLVED_BY), project updates with health status, milestones, initiatives, attachments, and Linear Docs. Issue history entries are transformed into decision traces (DecisionTrace → TraceStep chains) capturing state transitions, assignment changes, and priority changes with actor attribution. The Linear connector includes structured logging (logging.getLogger(__name__)), named constants for all page sizes and limits, error handling for URLError/JSONDecodeError/GraphQL errors, HTTP 429 retry with exponential backoff, pagination safety limits (MAX_PAGES), null-safe field access via _safe_nodes() helper, team key validation during authenticate() (lists available keys on mismatch), incremental sync via fetch(updated_after="ISO8601"), and truncation warnings when comments or history exceed page limits. The generated template (linear_connector.py.j2) has full parity with the source connector including decision trace generation. The Google Workspace connector imports from 6 Google APIs (Drive Files, Drive Comments, Drive Revisions, Drive Activity, Calendar, Gmail) with OAuth2 authentication and dynamic scope building. Its defining feature is decision trace extraction: resolved comment threads in Google Docs become DecisionThread graph nodes with question, deliberation, resolution, and participants — linked to documents, people, meetings, and email threads. Includes 10 decision-focused agent tools (find_decisions, decision_context, who_decided, document_timeline, open_questions, meeting_decisions, knowledge_contributors, trace_decision_to_source, stale_documents, cross_reference) injected via the renderer when the connector is active. Cross-connector linking detects Linear issue references (e.g., ENG-123) in comment bodies, doc names, email subjects, and meeting descriptions, creating RELATES_TO_ISSUE relationships. Rate limiting enforces 950 queries/100s with exponential backoff. 9 CLI flags (--gws-folder-id, --gws-include-comments, --gws-include-revisions, --gws-include-activity, --gws-include-calendar, --gws-include-gmail, --gws-since, --gws-mime-types, --gws-max-files). The generated template (google_workspace_connector.py.j2) has full parity with the source connector: all 7 stages (Files, Comments, Revisions, Activity, Calendar, Gmail, Cross-references) are included, using OAuth2 only (no gws CLI fallback). The Claude Code connector reads local session JSONL files from ~/.claude/projects/ — no authentication or API keys required. Parses user/assistant messages, tool_use/tool_result blocks, and progress entries. Extracts 7 entity types (Project, Session, Message, ToolCall, File, GitBranch, Error) with 10 relationship types (HAS_SESSION, HAS_MESSAGE, NEXT, USED_TOOL, MODIFIED_FILE, READ_FILE, PRECEDED_BY, ON_BRANCH, ENCOUNTERED_ERROR, RESOLVED_BY). Includes heuristic decision extraction (user corrections, deliberation markers, error-resolution cycles, dependency changes) producing Decision/Alternative entities and traces, and preference extraction (explicit statements, package frequency) producing Preference entities. Secret redaction (API keys, tokens, passwords, connection strings) is applied by default. 8 session intelligence agent tools (search_sessions, decision_history, file_timeline, error_patterns, tool_usage_stats, my_preferences, project_overview, reasoning_trace) injected via the renderer. 5 CLI flags (--claude-code-scope, --claude-code-project, --claude-code-since, --claude-code-max-sessions, --claude-code-content). Implementation split into connectors/claude_code_connector.py (main connector) and connectors/_claude_code/ subpackage (parser.py, redactor.py, decision_extractor.py, preference_extractor.py). The Claude AI connector imports conversations from the official Claude AI data export (Settings > Account > Export Data). Parses conversations.jsonl from .zip or raw .jsonl with streaming line-by-line processing for memory efficiency on large exports (1GB+). Extracts content from structured content blocks (text, tool_use, tool_result, thinking). Normalizes sender values (human → user). Creates Conversation, Message entities with HAS_MESSAGE/NEXT relationships. Generates Document per conversation for search/RAG. Deep mode extracts tool call sequences as DecisionTrace nodes. 7 CLI flags (--import-type, --import-file, --import-depth, --import-filter-after, --import-filter-before, --import-filter-title, --import-max-conversations). Implementation split into connectors/claude_ai_connector.py (main connector) and connectors/_claude_ai/ subpackage (parser.py). Shared data models and zip utilities in connectors/_chat_import/ subpackage (models.py, zip_reader.py). The ChatGPT connector imports conversations from the official ChatGPT data export (Settings > Data Controls > Export Data). Parses conversations.json from .zip or raw .json. Handles tree-structured mapping field with parent/children references — walks depth-first following last child at branch points. Filters out system messages and hidden messages (is_visually_hidden_from_conversation). Converts Unix float timestamps to ISO 8601. Handles content types: text, code, execution_output, multimodal_text. Tool-role messages captured with execution output as tool results. Same entity schema and CLI flags as Claude AI connector. Implementation in connectors/chatgpt_connector.py and connectors/_chatgpt/ subpackage (parser.py). The local-file connector ingests local Markdown, PDF, HTML, AsciiDoc, and Word documents into :Document → :Section hierarchies with LINKS_TO edges between sections and documents. Fully deterministic — no LLMs, no embeddings, no randomness. URI-keyed nodes integrate with the existing MERGE-on-(name, domain) pipeline in ingest.py without modification. Section URIs use GitHub/Pandoc slug rules (NFKD-normalize, ASCII-lower, [a-z0-9_] runs collapse to -); duplicate-heading collisions disambiguate per-parent (-1, -2, …); skipped heading levels (e.g. H1 → H3) preserve the original level on the child node. Per-format parser strategy: markdown-it-py with GFM extensions (Markdown); three-tier fallback pypdf outline → /StructTreeRoot → pdfplumber font-size heuristic (PDF); BeautifulSoup + lxml (HTML); pure-Python regex with block-delimiter state tracking (AsciiDoc); python-docx (Word). New optional dependencies in the connectors extra: markdown-it-py, mdit-py-plugins, pdf-oxide, pypdf, pdfplumber, beautifulsoup4, lxml, python-docx. Implementation in connectors/local_file_connector.py and connectors/_local_file/ subpackage (parser, mapper, slug, link-resolver). Connectors run at scaffold time AND are generated into the project with make import / make import-and-seed targets.

Domain Ontology YAML Schema

Each domain YAML file must contain:

• inherits: _base — merge base POLE+O types

• domain: — id, name, description, tagline, emoji

• entity_types: — label, pole_type (PERSON/ORGANIZATION/LOCATION/EVENT/OBJECT), subtype, color (hex), icon, properties (name, type, required, unique, enum)

• relationships: — type, source, target

• document_templates: — id, name, description, count, prompt_template, required_entities

• decision_traces: — id, task, steps (thought/action), outcome_template

• demo_scenarios: — name, prompts list

• agent_tools: — name, description, cypher query, parameters

• system_prompt: — multi-line agent system prompt

• visualization: — node_colors, node_sizes, default_cypher

Property types: string, integer, float, boolean, date, datetime, point YAML booleans in enum values must be quoted: enum: ["true", "false"] not enum: [true, false]

Generated Project Structure

When a user runs the CLI, the output is:

my-app/
├── backend/app/          # FastAPI + chosen agent framework
│   ├── main.py, config.py, memory.py, routes.py, models.py
│   ├── agent.py          # Framework-specific (8 frameworks available)
│   ├── context_graph_client.py, gds_client.py, vector_client.py
│   ├── ontology_document.json  # NAMS ontology doc (create-on-connect for custom domains)
│   ├── connectors/       # Only if SaaS connectors selected
│   │   ├── __init__.py
│   │   └── {service}_connector.py  # One per selected service
│   └── __init__.py
├── backend/tests/
│   ├── __init__.py
│   └── test_routes.py    # Generated test scaffold (health, scenarios)
├── backend/scripts/
│   ├── generate_data.py
│   └── import_data.py    # Only if SaaS connectors selected
├── backend/pyproject.toml
├── frontend/             # Next.js + Chakra UI v3 + NVL
│   ├── app/ (layout.tsx, page.tsx, globals.css)
│   ├── components/ (ChatInterface, ContextGraphView, DecisionTracePanel, DocumentBrowser, Provider)
│   ├── lib/config.ts, theme/index.ts
│   └── package.json, next.config.ts, tsconfig.json
├── cypher/ (schema.cypher, gds_projections.cypher)
├── data/ (ontology.yaml, _base.yaml, fixtures.json, documents/)
├── mcp/                 # MCP server config (only if --with-mcp)
├── .env, .env.example, Makefile, docker-compose.yml, README.md, .gitignore

Testing

Unit Tests

pytest tests/ -v                    # Fast suite: 1,454 passing (full CI suite with connectors extra: 1,866 passing / 1,881 collected)
pytest tests/test_config.py         # Config model + framework alias + google api key + crewai anthropic extra tests (26)
pytest tests/test_ontology.py       # Ontology loading + all 27 domains validate + enum sanitization + color collision checks + Cypher query validation
pytest tests/test_renderer.py       # Template rendering + all 8 frameworks + v0.3.0 features (64)
pytest tests/test_generator.py      # Data generation pipeline (14)
pytest tests/test_cli.py            # CLI integration + 8 domain/framework combos + neo4j types + validation + auto-slug + --neo4j-database + --ontology-file + Linear + Google Workspace + Claude Code connectors (84)
pytest tests/test_custom_domain.py  # Custom domain generation with mocked LLM (17)
pytest tests/test_connectors.py     # SaaS connectors with mocked APIs (125, includes 58 Linear + 28 Google Workspace + 38 Claude Code tests)
pytest tests/test_chat_import.py    # Chat history import: Claude AI + ChatGPT parsers, connectors, CLI flags (78)
pytest tests/test_generated_project.py # Deep validation: Python/TS/Cypher syntax, memory, neo4j types, streaming, QA fixes, async bridging, thread safety, tool prompts, embeddings config, NEO4J_DATABASE + NAMS cypher + scenario-fallback pins (269)
pytest tests/test_fixtures.py       # Cross-validation: schema alignment, agent tool property refs, data quality ranges (88)
pytest tests/test_security.py       # Cypher injection prevention: parameterization across 27 domains, generated code static analysis, run_cypher tool safety
pytest tests/test_doc_snippets.py   # Documentation validation: YAML examples parse, CLI flags exist, Cypher snippets valid, Make targets exist (8)
pytest tests/test_frontend_logic.py # Frontend logic: SSE event parsing, thinking/response split, backend/frontend event type contract (25)
pytest tests/test_routes_integration.py # Mounts the generated FastAPI app: NAMS/bolt route dispatch, /cypher on NAMS, 503 guards, /health memory surfacing (17)
pytest tests/test_generated_client_runtime.py # Executes rendered context_graph_client.py + memory.py against doubles: NAMS dispatch/coercion, database threading, store_message error state (26)
pytest tests/test_performance.py    # Timed generation tests (slow, 27 domains)
pytest tests/test_generated_tests.py # Scaffold + install + run generated project test suite for 4 frameworks (slow, 5)
pytest tests/test_integration.py --integration  # Neo4j integration: schema DDL, fixture ingestion, agent tool queries, domain scoping, explicit-database threading (10)

Unit tests do NOT require Neo4j or any API keys. All tests use tmp_path fixtures for output. Integration tests require --integration flag and a running Neo4j instance (NEO4J_URI, NEO4J_USERNAME, NEO4J_PASSWORD).

E2E Smoke Tests

Full-stack integration tests that scaffold a project, install dependencies, start the backend, and send chat prompts. Requires a running Neo4j instance and LLM API keys.

make smoke-test                     # Run 3 key framework tests (pydanticai, google-adk, strands)

# Or run directly with more control:
python scripts/e2e_smoke_test.py --domain healthcare --framework pydanticai --quick
python scripts/e2e_smoke_test.py --all-domains --framework claude-agent-sdk --quick
python scripts/e2e_smoke_test.py --domain gaming --framework openai-agents  # full mode (all prompts)

Required env vars: NEO4J_URI, NEO4J_USERNAME, NEO4J_PASSWORD, plus ANTHROPIC_API_KEY and/or OPENAI_API_KEY and/or GOOGLE_API_KEY depending on framework.

Makefile Targets

Adding a New Domain

1. Create src/create_context_graph/domains/{domain-id}.yaml following the schema above

2. Generate fixture data: run the CLI with --demo-data or use generator.py directly

3. Copy the fixture to src/create_context_graph/fixtures/{domain-id}.json

4. Verify: pytest tests/test_ontology.py::TestLoadAllDomains -v

Adding a New Agent Framework

1. Create src/create_context_graph/templates/backend/agents/{framework_key}/agent.py.j2 (use underscores for directory name; hyphens in config key are auto-converted via fw_key = framework.replace("-", "_"))

2. Add the framework key to SUPPORTED_FRAMEWORKS, FRAMEWORK_DISPLAY_NAMES, and FRAMEWORK_DEPENDENCIES in config.py

3. Template must export async def handle_message(message: str, session_id: str | None = None) -> dict returning {"response": str, "session_id": str, "graph_data": dict | None, "entities_extracted": list, "preferences_detected": list}

4. Import store_message, get_context, resolve_session_id from app.memory and call them in handle_message() for multi-turn conversation support with entity extraction

5. Pass tool_name= kwarg to execute_cypher() calls for tool call visualization

6. Use {% raw %}...{% endraw %} blocks for Python dict literals in the template

7. Use {% for tool in agent_tools %} to generate domain-specific tools from ontology

8. The template receives full ontology context: domain, agent_tools, system_prompt, framework_display_name, etc.

9. (Optional) Add handle_message_stream() for text streaming: Import get_collector from context_graph_client, use the framework's streaming API to iterate text chunks, call collector.emit_text_delta(chunk) for each, and collector.emit_done(response_text, session_id) at the end. Tool call events fire automatically via execute_cypher. If not provided, the /chat/stream route falls back to handle_message() with tool events still streaming in real-time.

10. Add tests to TestAllFrameworksRender in test_renderer.py, TestMultipleDomainScaffolds in test_cli.py, and TestStreamingAgentTemplates in test_generated_project.py

Current frameworks and their patterns

Adding a New SaaS Connector

1. Create src/create_context_graph/connectors/{service}_connector.py implementing BaseConnector

2. Use @register_connector("service-id") decorator to register it

3. Implement authenticate(credentials), fetch(**kwargs) -> NormalizedData, and get_credential_prompts()

4. Add the import to connectors/__init__.py

5. Create src/create_context_graph/templates/backend/connectors/{service}_connector.py.j2 (standalone version for generated projects)

6. Update import_data.py.j2 to handle the new connector

7. Add tests to test_connectors.py

Current connectors

Dependencies

Core: click, questionary, rich, jinja2, pyyaml, pydantic, neo4j Optional: anthropic, openai (for LLM data generation), neo4j-agent-memory (for memory-aware ingestion) Connectors (optional): PyGithub, notion-client, atlassian-python-api, slack-sdk, google-api-python-client, google-auth-oauthlib, simple-salesforce (Linear connector uses stdlib only) Dev: pytest, pytest-cov, pytest-asyncio Build: hatchling (src layout, bundles YAML/JSON/Jinja2 files automatically)

CI Pipeline

GitHub Actions (.github/workflows/ci.yml) runs on push to main and all PRs:

Every job installs from uv.lock via uv sync --locked --extra ... and runs commands with uv run --no-sync. Never mix uv pip install with a bare uv run in CI: uv run does an implicit lockfile sync that downgrades locked base deps underneath freshly-installed latest extras (this skew broke CI when a new anyio required a newer typing_extensions than the lock pinned). If --locked fails, run uv lock locally and commit the refreshed lockfile — dependency changes in pyproject.toml must ship with a matching uv.lock. The Makefile test/lint targets use uv run --extra dev ... so local runs resolve from the lock the same way.

The smoke-test job is gated behind vars.SMOKE_TESTS_ENABLED == 'true' (repository variable) and requires these repository secrets: NEO4J_URI, NEO4J_USERNAME, NEO4J_PASSWORD, ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY. It runs test_integration.py --integration before the E2E smoke tests. Uses fail-fast: false so one framework failure doesn't block others, and depends on the test job passing first.

A separate release workflow (release.yml) triggers on version tags (v*).

What's Not Yet Implemented

• TypeScript compilation validation in CI (requires Node.js in test environment)