Amazon Dynamodb
AWSOfficialPublished by AWS
- Skill
- Tool integration
- Cloud and DevOps
- Apache-2.0
- Low risk
- Has scripts
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a structured, axiom-based approach to building and maintaining DynamoDB data layers. It guides users through the entire lifecycle of a DynamoDB implementation, from initial schema design and access pattern enumeration to cost estimation, performance benchmarking, and debugging. By applying specific design axioms, the agent helps ensure that data models are defensible and optimized for real-world usage. The skill supports advanced features including Global Tables, Streams, TTL, and native vector search, and provides a clear pipeline for iterative design improvements.
What it does
- Enumerates application access patterns to inform schema design.
- Provides cost estimation for proposed DynamoDB data models.
- Automates live validation and performance benchmarking of designs in AWS.
- Debugs common issues like hot partitions, throttling, and unbounded scans.
- Configures advanced features such as vector indexes and zero-ETL integrations.
- Generates defensible design reports and performance summaries.
When to use it
- When designing a new DynamoDB-backed application from scratch.
- When reviewing or refactoring existing DynamoDB schemas.
- When troubleshooting performance bottlenecks like throttling or high latency.
- When evaluating the cost impact of different table configurations.
- When implementing semantic search using DynamoDB vector indexes.
Advantages
- Ensures design decisions are based on proven axioms rather than guesswork.
- Provides a clear, iterative path from conceptual design to live validation.
- Reduces the risk of costly architectural mistakes through early cost estimation.
- Offers a safe, gated process for deploying and benchmarking resources in AWS.
- Helps identify and resolve performance issues before they impact production.
Usage examples
- “Help me design a DynamoDB schema for a multi-tenant task management application.”
- The agent will guide you through enumerating access patterns, defining entities, and selecting partition/sort keys based on your specific requirements.
- “My DynamoDB table is experiencing throttling on specific partitions. Can you help me debug it?”
- The agent will analyze your access patterns and schema to identify potential hot keys and suggest sharding or GSI strategies to resolve the bottleneck.
- “Estimate the monthly cost for a DynamoDB table with these access patterns.”
- The agent will generate a cost report based on your provided RPS, item sizes, and table configuration.
- “How do I implement vector search in my existing DynamoDB table?”
- The agent will explain how to configure vector indexes, store embeddings, and use the SearchVectors API while adhering to the required design axioms.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/aws/amazon-dynamodb/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip amazon-dynamodb.zip -d .agents/skills
Documentation by AWS
DynamoDB Axioms
This document is a set of design axioms for DynamoDB applications. It is intended to be read by an agent with no other context about the application and used to produce a defensible data-layer design.
Guardrail — where this skill's own files live (MCP vs local install)
This skill can be loaded two ways, and they resolve the skill's own bundled files from different places. Determine how the skill was loaded before reading a reference or running a script:
Loaded through the AWS MCP
retrieve_skilltool: The skill is not installed on the local filesystem. You MUST fetch each reference or script viaretrieve_skillwith thefileparameter (e.g.file="references/vector-search.md"orfile="scripts/calculate_costs.py"), and run the script from the returned content. Do NOTfile_readthese paths locally — they do not exist on disk.Installed locally (e.g.
.kiro/skills/amazon-dynamodb/or~/.claude/skills/amazon-dynamodb/): Read and run files from the local skill directory using relative paths.
This distinction applies only to the skill's own packaged files. User data and
session artifacts are always read from and written to the user's working
directory. Never fetch or write customer data through retrieve_skill.
Separately from where these files live: this skill calls real AWS APIs when it validates a design, deploys a scratch table or benchmarks one. The AWS MCP server is recommended for those AWS interactions where it is available — it gives consistent credential handling and region resolution across hosts. It is not required: every script here uses boto3 and the AWS CLI against the ambient credential chain, so the skill works unchanged when no MCP server is present.
The ${SKILL_DIR} resolution below applies to the locally installed case: over
MCP there is no directory to resolve, and the scripts this skill ships cannot be
executed from a returned string — so a cost estimate or live validation requires
the locally installed form.
Resolving the skill's own paths
This skill is host-agnostic — it runs under Claude Code, Kiro, Codex, Cursor, a plain terminal, or CI. Where it lives on disk depends on the host (~/.claude/skills/…, ~/.kiro/…, ~/.codex/…, ~/.cursor/…, a repo checkout, anywhere). The agent's working directory is the user's project, not the skill bundle, so relative paths like scripts/calculate_costs.py will not resolve. Throughout this document, ${SKILL_DIR} means the absolute path of the directory that contains this SKILL.md file (the skill root, which holds scripts/ and references/).
Resolve ${SKILL_DIR} once per session, then reuse it. Pick the first method that works in your host:
You already know it. You loaded SKILL.md from a path —
${SKILL_DIR}is the directory that file is in. This is the most reliable source; prefer it.An environment variable. If
$DDB_SKILL_DIRis set, trust it.The bundled resolver (host-neutral, no host assumptions). It searches the common install roots and verifies the hit against sentinel files, so it never returns the wrong directory silently:
# If you already know the path to the script, just run it directly: # SKILL_DIR="$(sh /path/to/amazon-dynamodb/scripts/find_skill_dir.sh)" # If you don't, this host-neutral one-liner searches the common roots # (~/.claude, ~/.kiro, ~/.codex, ~/.cursor, ~/.config, ~/.local/share, $PWD): SKILL_DIR="$(find "$HOME" "$PWD" -maxdepth 7 -type f -name SKILL.md -path '*amazon-dynamodb*' 2>/dev/null \ | head -1 | xargs -I{} dirname {})" # Verify it before trusting it (sentinel check), then hand off to the resolver # for its loud-on-failure diagnostics: SKILL_DIR="$(sh "$SKILL_DIR/scripts/find_skill_dir.sh" 2>/dev/null || echo "$SKILL_DIR")"The resolver prints the verified skill root and exits 0, or prints nothing and exits non-zero with a fix-it message — so
SKILL_DIR="$(sh …/find_skill_dir.sh)"is safe to trust when it succeeds. It is plain POSIXsh, so it behaves identically across hosts.
Once resolved, export it so every later command is a clean substitution and the scripts can also pick it up:
export DDB_SKILL_DIR="$SKILL_DIR"
python3 "$DDB_SKILL_DIR/scripts/calculate_costs.py" --model dynamodb_data_model.json --output cost_report.mdInternally the scripts locate their own siblings (other scripts, scripts/benchmark_lambda.py) relative to themselves, so you only ever need the root path — never each individual script path.
Rules:
Always invoke scripts with an absolute path (the
$DDB_SKILL_DIR/…form). Do notcdinto the skill directory — the user's working directory must stay put so their artifacts (dynamodb_data_model.json,cost_report.md, …) land where they expect.If none of the three methods resolves the directory, stop and ask the user where the skill is installed rather than guessing. A wrong
${SKILL_DIR}produces confusing "file not found" failures downstream; one clarifying question is cheaper.
The pipeline at a glance
The skill is one tool per stage. The default path touches no AWS: most work is stage 1 (a design you can discuss and refine conversationally). Stage 2 (cost) runs on request or when the design is being finalized — not reflexively every turn. Stages 3–6 are a distinctly opt-in, heavyweight fork that creates real AWS resources and incurs a real bill; enter it only on explicit user agreement. Each stage's detailed contract is in the section named in the last column.
| # | Stage | Command (after export DDB_SKILL_DIR=…) | Reads | Writes | AWS? | Section |
|---|---|---|---|---|---|---|
| 1 | Design | (no script — you produce the access-pattern list + schema) | — | (in-reply artifacts) | no | Artifacts to produce |
| 2 | Cost | python3 "$DDB_SKILL_DIR/scripts/calculate_costs.py" --model dynamodb_data_model.json --output cost_report.md | dynamodb_data_model.json | cost_report.md | no | Cost estimation |
| 3 | Deploy | python3 "$DDB_SKILL_DIR/scripts/deploy_model.py" --model dynamodb_data_model.json --config benchmark_config.json --manifest-out created_resources.json --yes-deploy | model + config | created_resources.json | yes | Live validation |
| 4 | Benchmark | python3 "$DDB_SKILL_DIR/scripts/benchmark_model.py" --model dynamodb_data_model.json --config benchmark_config.json --manifest created_resources.json --raw-out perf_raw.jsonl --summary-out perf_summary.json | model + config + manifest | perf_raw.jsonl, perf_summary.json | yes | Live validation |
| 5 | Report | python3 "$DDB_SKILL_DIR/scripts/generate_perf_report.py" --model dynamodb_data_model.json --summary perf_summary.json --output performance_report.md | model + summary | performance_report.md, design_findings.json | no | Live validation |
| 6 | Teardown | python3 "$DDB_SKILL_DIR/scripts/generate_teardown.py" --manifest created_resources.json --out teardown.sh → review → bash teardown.sh --confirm | manifest | teardown.sh | yes (on --confirm) | Live validation step 6 |
| — | Iterate | python3 "$DDB_SKILL_DIR/scripts/iterate_design.py" … (wraps 3→4→5→cost as one human-driven round) | model + config + loop-state + manifest | loop_state.json + the above | yes (gated) | Iterative design loop |
Who reads what. You (the agent) read the compact artifacts: cost_report.md, design_findings.json, loop_state.json. The user reads performance_report.md. Never read perf_raw.jsonl (large) — it only feeds stage 5.
Consent gates. Stage 3+ needs --yes-deploy; the benchmark refuses to spend over cost_guardrail_usd without --allow-spend; teardown needs the user's attested review and intent before you run bash teardown.sh --confirm. Details in Live validation.
AWS access (MCP recommended, not required). Stages 3–6 talk to AWS (create tables, a Lambda, an IAM role, then benchmark and tear down). For the best experience with AWS API calls the AWS MCP server is recommended but not required — every script here uses boto3 directly and runs from a plain shell with standard AWS credentials (a profile, SSO, or environment credentials), so the skill works identically with or without the MCP server. Nothing in this skill assumes MCP-specific tools.
How to use these axioms
Read the reference architecture first when the task is to design, review, or critique a full-app data layer (multi-entity schemas, multi-table layouts, end-to-end composition with streams/search/notifications).
${SKILL_DIR}/references/reference-architecture.mdis a complete multi-tenant kanban task-board ("TaskBoard") SaaS on AWS backed by DynamoDB, with all of the surrounding pieces (Cognito, CloudFront, HTTP API, Lambdas, Streams, OpenSearch, AppSync Events, SQS/EventBridge, cascades, idempotency middleware) worked out and justified. The axioms tell you what must be true; the reference shows how these pieces fit together in practice. Not reading it on a multi-table design means you will miss patterns that are in the reference but hard to re-derive from axioms alone — idempotency middleware, phantom-upsert guards, AppSync channel authorization, the Notifications-as-EventBridge-not-table decision, cascade-delete via chunkedBatchWriteItem. Skip this step only for small-scope questions — a single-table question, a query-cost calculation, a pointed debugging question.Produce the access-pattern list (next section) before applying any axiom below. Every modeling axiom assumes this list exists; an axiom that asks "is this pattern frequent?" or "what does this query return?" cannot be applied without it.
Produce the artifacts listed under Artifacts to produce. These are the outputs of a design, not intermediate notes. The axioms shape the artifacts; the artifacts are what the agent hands back.
Apply the Patterns section alongside the axioms. Patterns are not axioms — they are load-bearing implementation details that the reference made concrete, and that a design will need even when no axiom explicitly calls for them.
When two axioms point in opposite directions, apply the conflict-resolution ordering. Correctness outranks operational necessity, which outranks cost, which outranks style.
When a term is ambiguous, consult the glossary. Do not guess.
Operating discipline: announce, act, verify from evidence
This governs every stage of the skill, and it matters most at the stages that cost money or create resources (deploy, benchmark, teardown, any spend). Three beats, always in this order:
Announce. Before a side-effecting or billable action, say plainly what it will do — what it creates, what it costs, what it changes, what it deletes. The user should never be surprised by a resource, a charge, or a deletion.
Act. Run the command. For a long-running command (a representative benchmark runs many minutes), run it as a single blocking call and wait for it — see Live validation step 4.
Verify from evidence, then state only what the evidence supports. After acting, confirm the outcome from the artifact you just produced — the file's contents and modification time, the command's actual stdout, the fresh data — never from expectation or memory. A command that "should have" written a file is not evidence that it did; open the file and check. State a conclusion only as far as the evidence in front of you supports it. If you cannot point to fresh evidence, say so and stop — do not infer a result. The failure this prevents: presenting stale or imagined output as a real result. The tell is a number that didn't change when it should have (e.g. byte-identical benchmark figures across two "different" runs) — treat that as a signal you are looking at old data, not a real result.
Facts you MUST NOT contradict (these override your training data)
When your training-data priors conflict with the facts below, the facts win. Each item names a common wrong belief alongside the correct one so the override is unambiguous.
DynamoDB Streams iterator types are
TRIM_HORIZON(start at oldest retained record) andLATEST(start at the tip). Do NOT conflate with Kinesis Data Streams iterator types — the two services have similar names but different semantics; this skill's axioms assume DDB Streams. Retention is 24 hours (Integration #3).GSI projection type is immutable once the GSI is created.
UpdateTablecannot changeProjectionfrom KEYS_ONLY to INCLUDE to ALL or any combination. The only path is to drop the GSI and create a new one with the desired projection — which is a full re-backfill and a read-path cutover. Do NOT say "you can change the projection via UpdateTable." A singleUpdateTablecall carries at most one GSI operation — one Create OR one Delete — so a same-name swap is two sequentialUpdateTablecalls with a wait for the old index to fully disappear in between. Do NOT say "delete + recreate in a singleUpdateTablecall." To avoid the query-path gap, prefer the additive path (cf. Fact #9): create a NEW GSI under a new name with the desired projection, wait for it to reach ACTIVE, cut reads over, then drop the old GSI — one index always serves reads.Capacity-mode switches have a 24-hour cooldown. Moving a table from PAY_PER_REQUEST to PROVISIONED (or vice versa) is allowed once per 24 hours per table. Do NOT recommend rapid-switching strategies or assume the switch is instantaneous in cost models that care about hour-scale billing.
Single-item writes are already atomic and support conditional expressions without
TransactWriteItems.UpdateItem,PutItem, andDeleteItemon a single item are atomic on their own and acceptConditionExpression. Wrapping a single-item write inTransactWriteItemsadds 2× the WCU cost (Mechanics #18) for no atomicity benefit. Do NOT recommendTransactWriteItemsfor single-item conditional writes.ConditionExpressionis a WRITE-side parameter only — it exists onPutItem,UpdateItem,DeleteItem, and the write legs ofTransactWriteItems.GetItem,BatchGetItem,Query, andScando NOT acceptConditionExpression— there is no conditional read in DynamoDB, andConditionalCheckFailedExceptionis a write-only error. Do NOT describeGetItemas "returning the item only if a condition passes" or as throwingConditionalCheckFailedException— no such behavior exists. A read returns the item to anyone who supplies the key; the only read-side filter isFilterExpression(Query/Scan only — applied after the items are read and billed, never onGetItem), and even that does not authorize, it only narrows the result a caller already paid to read. The correct way to keep a caller from reading another tenant's item is to make the data unaddressable to them — partition-key the table on the authorization identifier (Data modeling #14) so a foreign key simply isn't in a partition the caller can reach — NOT to bolt a "conditional GetItem" on top.Maximum item size is 400 KB, hard cap. The 1 MB limit is the
Query/Scanresponse-page cap, not an item cap. Do NOT quote 1 MB as the item limit. Items near 400 KB also cost more per write (WRU = 1 per 1 KB rounded up, Mechanics #18), so large items are expensive even before the cap bites.BatchGetItemandBatchWriteItemare NOT atomic. Partial failures are normal and returned viaUnprocessedKeys(BatchGetItem) orUnprocessedItems(BatchWriteItem). The client must retry the unprocessed portion with exponential backoff. Do NOT describe batch operations as atomic or all-or-nothing — useTransactWriteItemswhen atomicity across multiple items is required (subject to Mechanics #14 bounds).Reserved Capacity applies to PROVISIONED capacity only, not to on-demand (PAY_PER_REQUEST). Do NOT recommend Reserved Capacity for on-demand tables — there is no such product. On-demand savings come from usage-based discounts or table-class selection (Standard vs Standard-IA), not reservations.
A failed
ConditionExpressionstill consumes write capacity.ConditionalCheckFailedExceptioncharges the same WCU as a successful write of the same shape. Do NOT claim that failed conditional writes are free or that the condition check happens "before" the write-cost is assessed. Plan cost models around expected failure rates (Mechanics #18 usesconditional_fail_ratefor this reason).A GSI's key schema (partition key / sort key) is immutable once the GSI is created.
UpdateTablecan add a new GSI or drop an existing one, but it cannot alter the KeySchema of an existing GSI. Re-keying an index — including write-sharding a hot GSI partition key by adding a hash suffix — is therefore an additive migration, not a code-only change: create a new GSI with the new key → let it populate → cut reads over → drop the old GSI. A historical backfill is needed only when the new index must cover items that were already written and won't be touched again; a sparse or small in-flight index (e.g. one holding only active orders) populates from ongoing writes alone and needs no backfill. Do NOT describe a GSI key change as "just a code change" or "no schema migration."DynamoDB has native vector search. Semantic similarity does NOT require a second datastore. A vector index is a third index type alongside GSIs and LSIs: declare it with the
VectorIndexesparameter onCreateTableor theVectorIndexUpdatesparameter onUpdateTable, store embeddings as a list of numbers on your items, and query it with theSearchVectorsAPI (approximate nearest neighbour). Do NOT say DynamoDB cannot do vector or similarity search, and do NOT route a semantic-search requirement to OpenSearch, pgvector, or S3 Vectors by default — the vectors live on the same items as the operational data, with no replication pipeline (Integration #8 routes the workload types).Query,Scan, PartiQL and DAX do not work against a vector index;Queryis rejected withValidationException: Query operation not supported on this index type.A vector index is NOT a GSI and is NOT configured through one — there is noVectorIndexConfigand no vector block on a secondary index. Vector indexes require on-demand capacity (Mechanics #19).DynamoDB does NOT recompute embeddings, and the index cannot fix a stale one. The vector is ordinary item data: DynamoDB indexes the numbers you wrote and nothing more. Edit the source text and the stored embedding still describes the old text, so it keeps matching the old meaning — indefinitely, with no error and no staleness signal anywhere. This is not index-propagation lag, and vector-index write propagation does not "close the window": no amount of waiting regenerates a vector. Whenever content behind an embedding is mutable, say so explicitly and give the fix — detect the content change and re-embed with the same model, then write the new vector back (DynamoDB Streams into a re-embedding consumer is the usual shape; synchronous regeneration in the write path is also fine). Never imply the index refreshes vectors on its own.
Dimensions,DistanceFunction,ProjectionandSearchSchemaare ALL immutable — every one of them is fixed atCreateTable, not just the projection. Changing any single one means creating a replacement index and migrating; there is no in-placeUpdateTablefor them (Mechanics #7).A stale SDK is not evidence the feature is missing. The operations ship in botocore/boto3 ≥ 1.43.64 and AWS CLI v2 ≥ 2.36.16. Below those versions they are absent from the client entirely —
aws dynamodb search-vectorsfails withFound invalid choiceandhasattr(client, "search_vectors")isFalse. Testing for thathasattris better than comparing version strings: it checks the capability directly and cannot go stale. If it is absent, tell the user to upgrade; do NOT conclude the feature does not exist. Full API surface, sharp edges and troubleshooting:${SKILL_DIR}/references/vector-search.md.Quoted service limits are a design envelope, not a fact to argue with. The per-table index count, maximum dimensions,
TopKrange and inline-filter count quoted in this skill are subject to change, and quotas of this kind generally rise. Design inside them, but before telling a user their design exceeds a limit, confirm the current value in AWS Service Quotas or the DynamoDB developer guide — a raised quota that this skill has not caught up with should not become a wrongly rejected design.
These ten facts are not the full axiom set — they are the subset where LLM prior is most likely to be wrong. When a user's question intersects one of them, state the correct fact plainly and move on; do not hedge with "I think" or "typically."
The access-pattern list
Before touching a schema, enumerate every pattern the application must serve. For each pattern record:
A one-line description of what the caller is asking for.
Expected RPS (treat "unknown" as a design gap to close, per Mechanics #2).
Items returned per call and approximate item size in KB.
Consistency requirement (strong, eventual, or transactional).
Authorization scope — the identifier that must be verified before the call is permitted (per Data modeling #14).
The list is a numbered, ranked table. The rest of this document assumes it exists. Any modeling decision that cannot be traced back to an entry on this list is unjustified.
Per-entity operational-config inputs
This interview is required before proposing any table boundary. Producing a full multi-table design first and then backfilling "here are the assumptions I made" is a workflow violation, not a shortcut. The per-entity questions below drive the table-splitting decision via Data modeling #3; when the answers are agent-assumed rather than user-stated, the signal fires spuriously and the design ends up over-fragmented (or under-fragmented if the agent guessed "no divergence" to keep things simple). Ask first, then design.
Before grouping entities into tables, gather operational-config requirements from the user per entity (or per logical aggregate — a parent and its tightly-bound children can share one answer set). Do not assume these defaults silently, because Data modeling #3 uses operational-config divergence as a signal to split tables — if the divergence is agent-assumed rather than user-stated, the signal fires spuriously and the design ends up over-fragmented.
For each entity, ask:
Backup and recovery granularity. Does this entity need PITR? If so, what retention (default 35 days, can be shorter)? Would this entity ever be restored independently of other entities, or always together with them? (Independent-restore requirements force table separation per Data modeling #5.)
Streams consumers. Does any downstream system need change events for this entity — search indexing, analytics export, notifications, audit, CDC? Which stream view type (
NEW_AND_OLD_IMAGESis the default per Integration #3)? A "no" here is a positive answer: no Streams consumer means Streams can stay disabled, which is cheaper and simpler.Capacity mode. Does this workload's shape justify provisioned (sustained, predictable traffic over months, per Mechanics #19), or does on-demand remain the default? "Unknown" means on-demand.
TTL. Is there a per-item expiration attribute the application will set? If yes, the attribute is a Unix epoch second (per Patterns #3). If no, TTL stays off and items persist until deleted.
Encryption and IAM scope. Any non-default requirement — customer-managed KMS key, specific IAM boundary, cross-account resource policy? Default is AWS-owned KMS and standard IAM; divergence is an explicit answer.
Treat these as design inputs on par with RPS. A missing answer is a gap to close, not a value to guess. If the user says "same across all entities," record that and do not treat the entities as operationally divergent — co-location by Data modeling #1 is then unobstructed. If the user states real divergence, Data modeling #3 fires on real divergence and the tables split.
Per-entity attribute walkthrough (drives item size)
Item size is the second-largest driver of the cost estimate after RPS, and it's the place the estimate silently drifts worst. A Query declared as 20 items × 1,536 B but really returning 20 × 512 B triples the modeled cost against reality. Mechanics #2 says unknown RPS is a design gap; the same discipline applies to item size — an ungrounded guess for estimated_item_size_bytes is a design gap, not a safe default.
For each entity, before settling on a number, walk the attribute list with the user. Asking first is the preferred path; proceeding from inferred attributes is the fallback. Either way, the user has to see and sign off on the per-attribute breakdown before it becomes an input to the cost estimate — a silent fill-in is what makes item sizes drift 2–10×.
Propose an attribute list grounded in the domain. For a Waypoint, that's
waypoint_id,courier_id,lat,lng,recorded_at. For a Contract, it'sfirm_id,contract_id,title,status,body,created_by,created_at,updated_at.Per attribute, estimate bytes using these starting points:
IDs and short strings (ULIDs, UUIDs, slugs, enum values): ~40 B each. The generic
S=100heuristic incost-model-schema.mdis conservative for the free-tier storage path; for per-item size estimation, use realistic values.Titles, display names, short descriptions: 100–300 B.
Long-form content (contract body, message body, serialized JSON aggregates): ask the user explicitly. Do not guess 4 KB or 50 KB without confirmation.
Numeric attributes: ~8 B.
Timestamps as ISO strings: ~25 B. As epoch numbers: ~8 B. (Mechanics #11.)
Boolean: ~1 B. Map/List: ~200 B per instance as a rough default, but ask if the user is storing a big blob inside a Map.
Ask the corrections the user will know and you won't: "Does this item carry any denormalized parent data per Mechanics #10?" "Is there a free-text field whose length varies widely?" "Are you storing the full document or a summary?" Update the estimates from the answers.
Sum the per-attribute estimates to derive the entity's
estimated_item_size_bytes. For a Query that projects a subset (INCLUDE / KEYS_ONLY, or application-side projection), use a smaller number for the access-pattern'sestimated_item_size_bytes— the bytes billed by DynamoDB are bytes actually read from the projected view, not the full item.If the user is uncertain on a specific attribute, label that attribute as an assumption in the artifact (same discipline as unknown RPS). Do not silently pick a number.
Surface the full list in your response — always, regardless of whether this is an interactive conversation or a one-shot prompt. Emit a compact markdown table per entity with columns
attribute | type | bytes | source (user or guess). This is a reply-shape requirement, not a dialog gate. In one-shot settings where there will be no follow-up turn, the table still goes in the response so the user sees exactly what you assumed — the call-out is how they catch a 3× overshoot on a body field before it contaminates every cost number downstream. Label every uncertain estimate "guess" explicitly; do not smuggle a guess in as a user-supplied number. Explicitly invite correction: "These are my guesses where noted — please correct any that are wrong." Even when the user said "just pick reasonable values and go," emit the table.
Calibration: for the reference Contracts-app example in cost-model-schema.md, Contract is ~2 KB (not 50 KB — the 50 KB value is the worst-case body size, not the typical), and Clause is ~512 B. If a declared estimated_item_size_bytes is more than 2× the sum of the named attributes and the user hasn't explained the gap, you're guessing — revisit.
A run that skips this walkthrough can drift by 2–10× on individual patterns. The live-validation step (below) will surface that drift, but you shouldn't need live validation to get the cost estimate in the right order of magnitude.
Artifacts to produce
Produce artifacts in the order listed below. Schema + per-pattern plan are the primary outputs; cost estimate (item 7) and live validation (item 8) come after the design exists, not instead of it. A response that leads with a cost analysis and buries the schema in an appendix has the dependency backwards — the user asked for a design, and the cost is a property of the design. The access-pattern list (item 1), schema (item 2), and per-pattern plan (item 3) must be visible and discussable in the reply before any cost numbers appear. Items 1–6 are the no-AWS design itself and are the default deliverable; item 7 (cost) is produced on request or at finalization (see Cost estimation); item 8 (live validation) is the opt-in AWS fork. Putting these artifacts only in
dynamodb_data_model.jsondoes not satisfy this — the user reads your prose, not the JSON. ❌ BAD reply shape (a real failure mode): a reply that opens "## Summary for the CFO — $1,019/month" with the schema living only indynamodb_data_model.jsonon disk and the reply's only design content a trailing "artifacts produced" file list. ✅ GOOD: access-pattern list + per-table schema + per-pattern plan + per-entity byte table (per Per-entity attribute walkthrough step 6) rendered in the reply, then the cost summary, then "cost_report.mdwritten."
A complete design hands back:
The access-pattern list as above.
A schema per table: primary key (named per Data modeling #7), GSIs with their key attributes and projection type, and the operational configuration (Streams, PITR, TTL, capacity mode, Global Tables replication, encryption, IAM scope — all per Data modeling #3).
A per-pattern plan: for each access pattern in the list, the exact API call (
GetItem,Query, orBatchGetItem, per Mechanics #16), the table or GSI it targets, the key conditions, the filter expressions if any, and the projected cost using the formulas in Mechanics #18.A fan-out topology: for each table with Streams enabled, the consumers (Lambda, EventBridge Pipe, Kinesis shim), the filters at the source (Integration #4), and the on-failure destinations and retry bounds (Integration #5).
A list of deviations and their justification: any axiom or pattern not applied, with a stated reason.
Idempotency and conditional-write guards: which routes use idempotency-key middleware (Patterns #1), which
UpdateItems carryattribute_existsguards against phantom upserts (Patterns #2), and whichPutItems carryattribute_not_existsguards against double-creation.A monthly cost estimate — produced on request or at finalization, not reflexively on every design turn. While the user is still exploring or refining the model, stay in design discussion and don't run the calculator each turn. Produce the estimate (via
${SKILL_DIR}/scripts/calculate_costs.py— see Cost estimation below) when the user asks what it costs, or when the design is being settled (they signal they're committing to it / taking it to review / want the numbers). When you do produce it, use the calculator — never inline arithmetic. Skip entirely for questions too narrow to have produced a full design (a single-query sizing, a debugging thread, a pointed mechanics question).A live validation (optional, on offer, last step): after the cost estimate, ask the user whether they want to deploy this schema to an AWS account they nominate and measure real per-operation capacity, latency, and GSI amplification against live DynamoDB. If yes, follow Live validation below. Skip for narrow questions, when the cost estimate was skipped, or when the user has no sandbox account. Unlike the cost estimate, this step creates real resources and incurs real charges, so both the offer and the consent must be explicit.
Conflict-resolution ordering
When axioms point in opposite directions, apply in this priority order:
Correctness — authorization boundary alignment (Data modeling #14), consistency requirements, transactional atomicity, idempotency. A design that leaks data across tenants or serves stale data where strong consistency is required is wrong regardless of its other merits.
Operational necessity — divergent PITR, Streams, capacity, or replication configuration (Data modeling #3), recovery granularity (Data modeling #5), per-partition throughput ceilings (Mechanics #3), transaction bounds (Mechanics #14). These are physical or service constraints; preference does not override them.
Cost and performance — access-pattern co-location (Data modeling #1), dedicated GSIs (Data modeling #6), projection choice (Mechanics #7), cost formulas (Mechanics #18), capacity mode (Mechanics #19).
Style and convention — naming (Data modeling #7), single-table vs. multi-table framing (Data modeling #11) absent other signal. The cheapest to override when a higher-tier axiom disagrees.
Two concrete examples:
Data modeling #1 (co-locate by shared access) vs. Data modeling #14 (partition by authorization boundary): #14 wins. If the natural access key and the authorization key differ, key on the authorization identifier and expose the alternate access via a GSI.
Data modeling #1 (co-locate) vs. Data modeling #3 (split on divergent operational config): #3 wins. Two entities sharing a read pattern but requiring different PITR retention or Streams consumers belong in separate tables.
Glossary
Access pattern — a request the application makes against the data layer, described by its key conditions, items returned, frequency, and consistency requirement. The atomic unit of DynamoDB design.
Aggregate — a cluster of entities that are read or written together. A single item, an item collection, or a set of items under different keys can each be an aggregate; the choice is the subject of Mechanics #1.
Item collection — the set of items sharing a single partition-key value. Queries against an item collection are constant-partition and cheap; cross-partition reads are not.
Identifying relationship — a data model in which a child entity is keyed by its parent's identifier plus its own. The child has no independent existence outside the parent.
Overloaded key — a partition or sort key whose value encodes a type prefix (e.g.
USER#42,ORDER#42) so that one physical key holds multiple logical entity types.Sparse GSI — a GSI whose indexed attribute is present on only a subset of base-table items, so the index projects only those items. Useful when an access pattern would otherwise filter out most items at read time. The same semantics apply to a vector index and are the mechanism behind silent de-indexing.
Vector index — a third index type alongside GSIs and LSIs, holding vector embeddings from a named item attribute and read with
SearchVectorsrather thanQuery/Scan. Declared viaVectorIndexesonCreateTableorVectorIndexUpdatesonUpdateTable. On-demand capacity only; up to 5 per table.ANN (approximate nearest neighbour) — the search strategy a vector index uses. It trades exactness for speed, so results and their ordering can vary slightly between runs or Regions over identical data.
SearchSchema — a vector index's optional schema, listing at most one
HASHelement (the vector index partition key, which scopes and scales each search and MUST be supplied on every call) and up to 18INLINE_FILTERelements (optional equality filters applied at the storage layer). Fixed at creation. Every SearchSchema attribute must also appear in the table'sAttributeDefinitions.Distance function —
COSINE,EUCLIDEANorDOT_PRODUCT, chosen at index creation and immutable. It determines both theScoreand the sort order: lower is more similar forCOSINE/EUCLIDEAN, higher forDOT_PRODUCT, whose scores can be negative.Silent de-indexing — an item written without the vector index's SearchSchema
HASHattribute is accepted on the base table with no error but never replicated into the index, so it vanishes fromSearchVectorsresults whileGetItemstill returns it intact. Expected behaviour, not a defect; see${SKILL_DIR}/references/vector-search.md.VS / VWR — Vector Search and Vector Write request units. Vector index capacity is metered in bytes, reported as
VectorSearchRequestBytesandVectorWriteRequestBytes, separately from base-table RCU/WCU, with a 1 KB per-request minimum.GSI write amplification — the property that every write to a base-table item with projected GSI attributes produces one write per matching GSI, each billed in WCU.
LWW (last-writer-wins) — the conflict resolution strategy used by standard Global Tables: the write with the newest timestamp wins; earlier writes are silently discarded.
MRSC (Multi-Region Strong Consistency) — an opt-in Global Tables mode that provides strong consistency across replicas via consensus, at higher write latency and cost.
Hot partition — a partition receiving traffic beyond the per-partition throughput ceiling (Mechanics #3), causing throttling even when table-level capacity is available.
Poison pill — a record that a stream consumer cannot process successfully, which blocks forward progress on its shard until it is discarded, retried to exhaustion, or routed to an on-failure destination.
RCU / WCU — read and write capacity units; the provisioned-mode spelling of the per-operation throughput unit. The formulas in Mechanics #18 are written in RCU/WCU and apply identically to on-demand.
RRU / WRU — read and write request units: the on-demand (PAY_PER_REQUEST) spelling of the same per-operation unit, billed per request rather than per provisioned capacity-second. One RRU = one RCU of work and one WRU = one WCU of work — the consumption math in Mechanics #18 is identical; only the billing dimension differs. The cost references (
cost-model-schema.md) use RRU/WRU because the calculator prices on-demand; the axioms and the performance report use RCU/WCU. They are the same quantity — do not treat a model's RRU/WRU figure and the report's RCU/WCU figure as different things.
Data modeling
Co-locate data by shared access pattern, not by domain. Two entities belong in the same table only when an application request fetches or writes them together. A shared business domain — "user data," "billing data" — is not sufficient justification. Co-location without a shared query introduces coupling and yields no performance benefit.
Treat table count as an output of the design, not a target. Do not optimize for one table, nor for one table per entity. The correct number of tables is whatever the access patterns produce. If the analysis surfaces three tables, ship three tables.
Treat table-level configuration as both a modeling input and an interface declaration. Streams, point-in-time recovery, TTL, capacity mode, attached Kinesis streams, Global Tables replicas, encryption, and IAM scope all apply at the table level — they declare how the table participates in the broader system. When two entities require different operational settings — different PITR retention, different stream consumers, different replication regions, different capacity modes — that divergence is a primary signal that they belong in separate tables, not a secondary concern to be reconciled later.
DynamoDB Streams provide two concurrent consumers, no native per-entity filtering, and a 24-hour retention window. These properties constrain downstream architecture as firmly as the key schema does. Decide the fan-out topology before finalizing table layout.
Design for recovery granularity. Point-in-time recovery operates at the table level; partial restores are not supported. If two entities would never be recovered together, they should not share a table. The cost of an incident scales with the volume of unrelated data the restore has to carry.
Prefer additional GSIs to overloaded keys. The constraints that historically motivated index overloading — a low per-table GSI ceiling, per-index provisioned capacity, and no on-demand mode — generally do not apply to modern DynamoDB, which supports many GSIs per table with shared capacity and on-demand billing (consult the current AWS service-quota docs for the exact per-table GSI limit). Choose a dedicated GSI per access pattern unless a specific, measured reason argues otherwise.
Vector indexes are scarcer and serialize with GSIs. A table supports at most 5 vector indexes, against a far higher GSI allowance, so budget them deliberately rather than adding one per query shape. More importantly, only one online index operation — create or delete — runs at a time per table, and that limit is shared between GSIs and vector indexes. Two
Createactions in a singleUpdateTablefail withLimitExceededException: Subscriber limit exceeded: Only 1 online index can be created or deleted simultaneously per table. Sequence index work one at a time, and note this makes any vector index migration (Mechanics #7) slow, because each new index must finish backfilling before the next can start.Name keys for what they represent. Use
customer_id,order_created_at, andOrdersByCustomerrather thanPK,SK, andGSI1. Self-describing keys reduce onboarding time, make code reviewable without a schema reference, and let tooling introspect the model. Reserve generic overloaded keys for genuinely polymorphic hierarchies.Pre-join only entities that are read together in a single request. Item collections exist to satisfy one-shot queries such as "fetch parent with all children." If no access pattern reads two entities together, do not store them together. A pre-join without a corresponding read is coupling without benefit.
Default to not sharing tables across service boundaries. A table shared between services forces teams to coordinate GSI allocation, key conventions, and schema changes. That coordination cost compounds as the table grows. Service ownership should imply table ownership.
Plan for the model to change. Access patterns evolve. A table layout that requires a multi-table refactor to accommodate a new pattern is a liability. Prefer designs in which adding or modifying an access pattern is a localized change to a single table instead of changing the full data access layer.
Apply single-table design as a tactic, not a default. Single-table design is appropriate for tightly bound parent-child hierarchies with predictable joint access — orders and line items, tenants and sub-resources. It is not appropriate as a global rule. The pattern's value is local; applying it globally produces the operational problems documented in production post-mortems.
Move analytical workloads off the operational table. DynamoDB is an OLTP store. SQL-shaped queries, full-table aggregations, and ad-hoc BI belong in a downstream system populated by export or zero-ETL integrations. Do not reshape the operational table to serve analytics.
Design around last-writer-wins in standard Global Tables, or opt into Multi-Region Strong Consistency (MRSC) when the domain cannot tolerate it. Standard Global Tables provide multi-region active-active replication with eventual consistency and last-writer-wins conflict resolution — they are not CRDTs. Domains that cannot tolerate LWW — inventory, balances, counters — have two options: application-level conflict avoidance (region-pinned writes, shard partitioning, conditional updates), or MRSC, an opt-in mode that provides strongly consistent reads and writes across a designated replica set via a consensus protocol. MRSC requires a three-region topology (two active replicas plus a witness), carries higher write latency and cost, and is configured per table. Cross-account replicas are supported in both modes and inherit the same consistency semantics — account boundaries are an IAM and ownership surface, not a consistency one. The choice between standard Global Tables and MRSC is a modeling decision: it determines what domains the table can host.
A special case: a single hot item under contention — a flash-sale stock decrement where thousands of buyers converge on one SKU. Here the problem is throughput, not just consistency: writes to one item serialize internally, so you hit elevated latency and the per-partition WCU ceiling (Mechanics #3) before the partition's raw budget, and every failed conditional decrement still burns WCU (Fact #8). Region-pinning a single item does NOT help — it pins the contention to one region without removing it. The fix is a sharded counter: split the quantity across N sibling items (
sku#shard0..sku#shardNas the partition key so they spread across partitions), decrement a random shard under a> 0condition, retry another shard on failure, and scatter-read all shards to sum — sized to undersell rather than oversell (each shard enforces its own floor). Note this IS write-sharding the partition key (Mechanics #3), applied to sibling counter items; it is the contention fix, not a contradiction of it. Region-pinning and per-item conflict avoidance remain right for high-cardinality, low-per-item-contention data (per-account balances, where each user is their own partition key and contention is ~1), not for one contended row.Align the partition key with the authorization or tenancy boundary. The identifier you verify before permitting a read or write —
tenant_id,account_id,board_id, whatever scopes the caller's access — should be the partition key of every table holding data under that scope. When authorization and lookup share a partition, a query against a partition the caller cannot prove access to simply returns nothing; when they diverge, a caller can pass a foreign child identifier and reach data across the authorization boundary (IDOR). Data that legitimately crosses tenants belongs in a separate table with its own access rules, not as a compromise on the main one.On any design review — and first of all when the user asks what the security problem is — evaluating whether the partition key aligns with the authorization/tenancy boundary is the FIRST check to make and to state, ahead of hot-partition, projection, or cost findings (per Conflict-resolution ordering, correctness outranks cost). The canonical hole: a partition key set to a client-supplied identifier (a
review_id/order_idfrom the URL) instead of the server-derived authorization identifier (user_id,tenant_id,account_id) — that is IDOR. If the design is already aligned, say so and move on; if it diverges, lead with it and do NOT downgrade it to "add aConditionExpression" — a per-call check is discipline, not a structural fix. When you explain why aConditionExpressionis not the fix, get the mechanism right: a write-side condition is opt-in (one forgotten call path = IDOR) and it does not cover reads at all — but the reason it does not cover reads is thatGetItem/Querytake noConditionExpressionin the first place (Fact #4), NOT that a "conditional GetItem returns the item / throwsConditionalCheckFailedExceptionanyway." Do not invent a read-side condition mechanism to argue against it. The structural fix is the key schema (the foreign key is unaddressable), and reads stay safe because the caller cannot name a partition they don't own — state it that way.
Integration
Ensure Streams consumers are idempotent or deduplicated by event identifier. DynamoDB Streams deliver at-least-once — handlers will see the same record twice after a Lambda timeout, a batch retry, or replay during an incident, and must produce the same outcome whether invoked once or many times.
Two strategies are viable:
Idempotent handler: structure the operation so repeating it is a no-op. Conditional writes, set-based mutations (
ADDto a set), and idempotency tokens on downstream APIs all qualify.Explicit dedupe: record the
eventIDfrom the stream record in a dedupe store (a DynamoDB table with TTL is the usual choice) and skip records already present.
Example (dedupe by
eventID, TTL bounded to the redelivery window):def handler(event, context): for record in event["Records"]: try: dedupe_table.put_item( Item={"event_id": record["eventID"], "ttl": int(time.time()) + 86400}, ConditionExpression="attribute_not_exists(event_id)", ) except ClientError as e: if e.response["Error"]["Code"] == "ConditionalCheckFailedException": continue # already processed raise process(record)Dedupe on
eventID, not on application identifiers likeorder_id— the same aggregate legitimately produces many events, and keying on the business identifier will drop valid change records.Use the transactional outbox pattern for reliable event publication. Write the state change and an outbox record in a single
TransactWriteItemscall; have a downstream consumer read the outbox via Streams and publish the event. This eliminates the "updated but did not emit" failure mode without resorting to two-phase commit.Default the stream view to
NEW_AND_OLD_IMAGES. Zero-ETL integrations with OpenSearch, most change-data-capture consumers, and any logic that depends on diffs all require both images.Filter events at the source, not in the handler. Source-side filtering has two mechanisms — mention both when diagnosing or proposing a fix:
Lambda Event Source Mapping
FilterCriteria— filters stream records before Lambda invocation. A handler that previously ran on 10M events/day but cares about 500k of them becomes a handler that runs 500k times, billed for 500k invocations instead of 10M. Filter shape (JSON pattern matching against the DynamoDB stream record):{"dynamodb": {"NewImage": {"status": {"S": ["shipped", "delivered"]}}}}.EventBridge Pipes filters — the same source-side discarding, applied when the Pipe is the consumer. Use Pipes when the fan-out crosses multiple downstream targets (an event → Lambda + SQS + EventBridge bus), since the 2-concurrent-consumer limit on Streams (Data modeling #4) makes Pipes the common path for >2-consumer topologies.
Both mechanisms eliminate invocation cost AND the work the handler would have done filtering. Inline filter logic inside the handler pays full invocation cost for every discarded event — a filter that drops 95% of traffic means 95% of invocations are pure waste. If the question is about source-side filtering, the answer names both mechanisms and routes the recommendation to the one that fits the consumer topology.
Treat Event Source Mapping configuration as a reliability surface.
BatchSize,MaximumBatchingWindowInSeconds,ParallelizationFactor,MaximumRetryAttempts,MaximumRecordAgeInSeconds,BisectBatchOnFunctionError, and on-failure destinations together define delivery semantics. Several defaults are actively dangerous:MaximumRetryAttemptsandMaximumRecordAgeInSecondsboth default to-1, meaning a single unprocessable record will block its shard for the full 24-hour stream retention and then disappear silently. Bound retries, bound record age, and configure an on-failure destination so poison records are quarantined rather than stalling the shard or vanishing. Leaving any of these at defaults is a deliberate choice, not a neutral one.Treat stream enablement as a durable interface commitment. Disabling a stream and re-enabling it produces a new stream ARN. Any Lambda Event Source Mapping or EventBridge Pipe bound to the old ARN becomes orphaned. Stream lifecycle is part of the table's public contract and should be managed accordingly.
Distinguish TTL-driven deletes from user-driven deletes. DynamoDB delivers TTL deletions through Streams with
userIdentity.principalIdset todynamodb.amazonaws.com. Consumers should branch on this attribute to route TTL events separately — typically to archival or audit pipelines — rather than treating them as ordinary user deletes.Match the analytical or search workload to the right target; do not reshape the operational table to serve either. DynamoDB is an OLTP store for analytical workloads, and the supported path to a read-optimized system is a managed integration, not a
Scanand not a bespoke GSI. Semantic similarity is the exception and stays in DynamoDB (Fact #10). Route by workload type:Semantic similarity — "find similar", "more like this", RAG retrieval, recommendations, AI-agent memory, embedding-based anomaly detection → a vector index on the DynamoDB table itself, queried with
SearchVectors. No second datastore, no replication pipeline, no PITR prerequisite. This is the default for anything expressed as similarity over embeddings. See${SKILL_DIR}/references/vector-search.md.Full-text, keyword, typo-tolerant or fuzzy matching, faceting, aggregations, geospatial/radius search → zero-ETL to Amazon OpenSearch Service. Initial load from a PITR snapshot, ongoing change capture via DynamoDB Streams, near real-time freshness (seconds). Requires PITR enabled on the source and Streams enabled with
NEW_AND_OLD_IMAGES.SQL analytics, BI, materialized views, data sharing → zero-ETL to Amazon Redshift. Initial load from a PITR export, ongoing change capture via incremental exports every 15–30 minutes. Requires PITR enabled on the source and KMS configured with an AWS-owned or customer-managed key (AWS-managed KMS is not supported).
Open-format data lake, Apache Iceberg, multi-engine analytics (Athena, EMR, Spark, Redshift), or ML feature stores → zero-ETL to Amazon SageMaker Lakehouse. Glue-orchestrated initial export plus incremental exports that write Iceberg tables to S3 or S3 Tables, typically 15–30 minutes fresh. Requires PITR enabled and a resource-based policy granting Glue the export actions.
Point-in-time batch dump with custom downstream processing and no freshness requirement → DynamoDB export to S3 (not branded zero-ETL, but the correct fallback when no zero-ETL target fits). Consumes no RCU.
In every zero-ETL path above, PITR on the source table is a hard prerequisite — this does NOT apply to the vector-index path, which is not an integration. Do not approximate search with
begins_withqueries and bespoke GSIs. Do not scan the operational table for analytics. The integrations exist because these workloads compose with a column store, a search engine, or an Iceberg table and do not compose with a NoSQL key-value store.Split a bundled ask; do not send it all to one place. A request for "search" often bundles several kinds. Semantic similarity goes to a vector index; lexical, fuzzy and geo go to OpenSearch; SQL goes to Redshift. A workload legitimately needing both semantic and full-text search uses both a vector index and OpenSearch. Two directions are both wrong: routing semantic similarity out to OpenSearch by reflex, and routing typo-tolerant keyword or geo-radius search into a vector index. ANN over embeddings does no substring, prefix or edit-distance matching.
When one operational table feeds BOTH the Redshift (SQL/BI) and SageMaker Lakehouse (Iceberg) paths, prefer a single DynamoDB→Lakehouse Iceberg export and have Redshift read the same Iceberg tables (Redshift Spectrum / native Iceberg support) rather than running two parallel zero-ETL integrations — this avoids a second incremental export and the duplicate per-GB charge with no loss of SQL/join capability. Use two separate integrations only when the consumers genuinely need divergent freshness or isolation.
Mechanics
Select aggregate tightness by weighing how often entities are read together against how often they are written independently. Three options — embed children in a single item, group them as an item collection under a shared partition key, or store them as separate aggregates — sit on a spectrum of how tightly parent and children are bound. No single threshold governs the choice. A high read correlation argues for co-location, but a write-heavy workload with large items pushes the opposite direction, since every update rewrites the full item. Item size and whether the child count is bounded matter as much as access frequency. Rule of thumb: if order line items are fetched with orders most of the time and the line count is bounded, consider embedding or an item collection with the order as the parent; if individual line items receive frequent updates in isolation, keep them separate so each write does not rewrite the whole parent. Selecting the wrong tier is the underlying mistake behind most "single-table design gone wrong" stories.
Document RPS for every access pattern. Without a request rate, you cannot size partitions, choose between on-demand and provisioned capacity, or justify a GSI. An estimate grounded in business context is sufficient; an absent rate is not. Treat "unknown" as a design gap to be closed.
Respect the per-partition throughput ceilings. A single partition supports up to 1,000 write capacity units and 3,000 read capacity units per second. Workloads that exceed these limits must shard the partition key — typically with a hash suffix for write-heavy traffic or a time bucket for sequential keys.
Base-table key schemas allow exactly one
HASHand at most oneRANGE. When a base table requires a composite key, encode it as a concatenated string with a stable delimiter, such astenant_id#user_id. GSIs support native multi-attribute keys — up to four attributes for the partition key and up to four for the sort key — with DynamoDB hashing the PK attributes together for distribution. Prefer native multi-attribute GSI keys over synthetic concatenated keys: items are written with natural attributes from the domain model, client code does not concatenate or parse, and adding a new multi-attribute GSI to an existing table requires no backfill of synthetic attributes. Use synthetic concatenated keys only when the number of components exceeds four or when an older table is already committed to the pattern.Multi-attribute GSI keys have strict query rules. On the partition-key side, every PK attribute must be constrained with equality — a GSI with PK (
tenant_id,region) cannot be queried bytenant_idalone, and inequality operators are not allowed on any PK attribute. On the sort-key side, attributes must be constrained left-to-right in the order they are defined; a middle attribute cannot be skipped. Equality conditions must precede any inequality, and only one inequality is allowed — it must be the final condition in the key condition expression.BETWEEN,>,<,>=,<=, andbegins_withall count as inequality. Violating these rules is the most common reason a GSI fails to satisfy its intended access pattern.Consider a sparse GSI when 50% or more of items would be filtered out. Indexing the presence of an attribute is materially cheaper than indexing all items and filtering at read time. Sparse indexes also reduce write amplification, since only items carrying the indexed attribute are projected.
Project only the attributes the access pattern reads. An
ALLprojection roughly doubles storage cost and write amplification for every base-table update. UseALLif read latency is important, or if your application is read heavy in comparison to writes. If item sizes are large or the table is write heavy, useINCLUDE, and if latency warrants it, useKEYS_ONLYwith a reverse table lookup.On a vector index, projection is a hard read boundary and it is immutable.
SearchVectorscannot return an attribute that is not projected into the index, and asking for one is silently ignored — no error, the attribute is simply absent from the result. The projection cannot be changed after creation, so under-projecting means either creating a replacement index and migrating (an additive migration, cf. Fact #9) or hydrating the missing attributes from the base table withGetItem/BatchGetItemafter the search. And it is not only the projection:Dimensions,DistanceFunctionandSearchSchemaare equally immutable. So "switch this index to COSINE" or "add a partition key to the search schema" is a new-index migration too, never anUpdateTable— when a debugging thread lands on the wrong distance function, say that explicitly rather than describing only the projection case. Note also that the vector attribute itself is excluded from results by default even underALL, since vector data is large — request it explicitly withProjectionExpressiononly if you actually need it back.Avoid using mutable attributes as GSI keys. DynamoDB implements a key-attribute change as a delete followed by an insert in the index, doubling the write cost for that update. Fields that change frequently should not appear as GSI partition or sort keys.
Use identifying relationships when child access is dominantly in the context of the parent. When a child entity cannot exist independently of its parent, and fetches or updates of children typically carry the parent identifier, model the relationship on the base table with the parent identifier as the partition key and the child identifier as the sort key. This removes a GSI for the "list children by parent" pattern and typically halves write cost by eliminating the associated index amplification. The exception is a hot path that updates a single child by its own id without the parent — a webhook scoped to a line item, for example.
UpdateItemrequires the full primary key, so if callers routinely arrive with onlyorder_item_id, the identifying-relationship schema forces a GSI lookup to resolveorder_idon every write, negating the savings. In that case, invert: key the child by its own id and carry the parent on a GSI. The right shape follows the hot access pattern, not a style preference.Restrict denormalization to attributes that rarely change. Short-circuit copies — a user's display name on an order, a product SKU on a line item — are safe and eliminate additional reads. Copying mutable attributes turns every update into a multi-item fan-out and quickly outweighs the read benefit.
Choose temporal encoding based on required sort behavior. Strings sort lexicographically;
"10"precedes"2". Use ISO 8601 timestamps when natural string sort is desirable and human readability matters. Use Unix epoch numbers when compactness, arithmetic, or precision matters. Do not mix encodings within a table. When a key attribute (a table or GSI partition/sort key) is a number — an epoch timestamp, a numeric id — or binary, you MUST declare its type in the data model so the live deploy creates it correctly. DynamoDB types only key attributes, and it enforces that type at write time: a key left undeclared deploys as string (S), and the first real integer/binary write then fails withValidationException— in production, not in the benchmark (the seed may pass while live writes fail). Declare the type inentities[].attributes[]as{"name": "order_date", "type": "N"}, or table-level inattribute_definitionsas{"attribute_name": "order_date", "attribute_type": "N"}(both spellings are read by the deploy and the benchmark; anattribute_definitionsentry wins on conflict). See${SKILL_DIR}/references/cost-model-schema.md.Do not rely on TTL for time-sensitive expiration. TTL deletions are eventual — the background sweeper runs on a best-effort schedule and expired items can remain visible for hours, in practice up to ~48 hours past the TTL timestamp, until the sweeper removes them. TTL is appropriate for storage reclamation and cleanup; it is not appropriate for security-sensitive expirations such as sessions, tokens, entitlements, or real-time event triggers.
Two consequences follow:
Filter on reads. Because an item can outlive its TTL until the sweeper runs, every read path that cares about expiration must check the TTL attribute itself —
FilterExpressionof#ttl > :now, or an application-side check. Do not assume an item returned by DynamoDB is logically current.Pair with EventBridge Scheduler when timing matters. If a workload needs a precise action at expiration — revoke a session, fire a reminder, release a lock — create a one-time EventBridge Scheduler invocation at write time for the exact timestamp, and let that invocation do the work. TTL handles eventual cleanup; the scheduler handles the time-sensitive trigger. Streams can then propagate the actual TTL deletion to archival or audit (see Integration #7), but Streams is not the timing mechanism.
The TTL attribute must be a Unix epoch value in seconds, not milliseconds.
Enforce uniqueness with transactions, not application logic. Create a sibling lookup item — for example,
UNIQUE#email#[email protected]— and write it together with the entity in a singleTransactWriteItemscall. A check-then-write sequence in application code has a race window; the transaction does not.Work within transaction bounds.
TransactWriteItemsis bounded by three hard limits — always state all three when the question is about transaction bounds, never just one or two. The bounds are independent; exceeding any single one rejects the transaction:100 items per transaction — total item count across the array.
4 MB total payload — combined size of all items in the transaction. Do NOT omit this bound or fold it into the 100-item bullet; a transaction with 10 items at 500 KB each passes the 100-item bound but fails the 4 MB bound. The payload cap is 4 MB exactly — not 16 MB (that's
BatchGetItem), not 1 MB (that'sQuery/Scanpage cap), not 400 KB (that's max item size).Single region — a transaction executes in one region only; it does NOT span Global Tables replicas. If callers in different regions need to participate in the same transaction, standard Global Tables is the wrong consistency model for the workload (use MRSC or restructure so each transaction is region-local).
Operations that routinely exceed these limits indicate an aggregate boundary that is too coarse, not a database limitation to be worked around. When an import or bulk operation crosses the 100-item boundary, batch it into multiple transactions and handle partial-failure recovery explicitly (a job id, checkpointed progress, idempotent retry of each chunk).
Count items, not records, against the 100-item bound. When one logical record requires multiple DynamoDB items — a contact plus a uniqueness sentinel per Mechanics #13 plus a counter increment — every item in the
TransactWriteItemsarray counts. 3 items per contact means ~30–33 contacts fit per transaction, not 100. The 4 MB payload bound imposes a second ceiling independently; for records near 30 KB each, the 4 MB cap bites before the 100-item cap. Plan the chunk size against both bounds and pick the smaller.Bulk imports over the 100-item boundary are background jobs, not synchronous API calls. A loop of
TransactWriteItemsthat pages through tens of thousands of records will exceed any reasonable request timeout (API Gateway 30s, Lambda 15min ceiling, browser request timeouts). Model the operation as a durable job: accept the request, enqueue a job record, drive the chunked execution from a worker (Step Functions, SQS + Lambda, batch job) with checkpointed progress so restart resumes mid-run, and report completion asynchronously. Do not treat "chunked transactions" as if they compose into a single synchronous operation.Default to eventually consistent reads. Strongly consistent reads cost twice as many RCU. Use them only at boundaries that require read-your-writes semantics; do not adopt them as a global default.
When a caller updates an item and immediately needs the fresh value, prefer
ReturnValues=ALL_NEWon theUpdateItem(orALL_OLD/UPDATED_NEW/UPDATED_OLDvariants as fit) over a follow-up read. The write returns the post-update item as part of its response at no extra RCU cost and with no consistency concerns — the write's return payload is authoritative by construction. A strongly-consistentGetItemafter anUpdateItempays 1 RCU and a second round-trip for information that was already available for free. This is the cheapest read-your-writes shape on DynamoDB.Resolve every access pattern to
GetItem,Query, orBatchGetItem.Scanis appropriate for administrative operations, not for application read paths. A production access pattern that requires a scan reflects a modeling gap, not an acceptable query choice.Constrain every query. Specify sort-key ranges or filter conditions, compute expected page sizes, and paginate with
LastEvaluatedKey. The constraint can come from the key structure itself — a parent partition that holds a bounded number of children (line items under one order, members of one team) is already constrained, and "return all under this parent" is a valid access pattern against it. What is not an access pattern is a query whose result size grows without bound as the dataset grows — "all orders for a customer" over years, "all events in a log" — with no range, no page target, and no termination condition. An unbounded result set over a growing collection is a design gap, not a query choice.Compute the cost of each access pattern. Read pricing depends on the API:
GetItem / BatchGetItem: bills per item returned, rounded up to 4 KB. Eventually consistent (default) = 0.5 RCU per 4 KB per item; strongly consistent = 1 RCU per 4 KB; transactional = 2 RCU per 4 KB.
BatchGetItemoffers no per-request discount — each item is billed independently.Query: bills on the total size of items matching the key condition, rounded up to 4 KB, at the same per-4 KB rate as
GetItem.FilterExpressionapplies after billing — a filter that discards 90% of matches does not reduce cost by 90%.Scan: bills on the total size of items examined — the portion of the table or GSI actually read — again before any filter is applied. This is the underlying reason
Scanis the wrong choice on a hot path.
Write pricing:
WCU = frequency × ⌈item_size / 1 KB⌉ × copies_updated.UpdateItemis billed onmax(before, after)item size, not the delta and not just the post-update size — a shrinking update still pays for the larger pre-update size rounded up. Transactional writes cost 2×.ConditionalCheckFailedExceptionstill consumes the same WCU as a successful write of that shape; planconditional_fail_rateinto the estimate.If these formulas cannot be written down for a given pattern, the pattern is not specified precisely enough to be modeled.
Use on-demand capacity by default; switch to provisioned on evidence. On-demand removes a class of capacity-related incidents and is the correct default for new or variable workloads. At sustained high throughput with predictable patterns, provisioned capacity with autoscaling delivers a lower unit cost. The switch should be triggered by measured traffic, not anticipation.
For a table carrying a vector index, on-demand is not a default — it is mandatory. Vector indexes are not supported on provisioned tables at all. A provisioned table must be switched to
PAY_PER_REQUESTbefore the index can be created, and that switch is subject to the 24-hour capacity-mode cooldown (Fact #3), so it is a scheduling constraint rather than an instant step. This also removes the provisioned-capacity option from any design that includes semantic search, so do not later propose moving such a table to provisioned for unit-cost reasons.
Patterns
These are concrete implementation patterns that sit alongside the axioms. They are not axioms in the "correctness/operational/cost/style" sense — they are load-bearing details that a complete design needs even when no axiom explicitly requires them. Each one is worked out in full in the reference architecture; this section is the short form so the agent recognizes when it applies.
Protect mutating API routes with an
IdempotencyKeystable. EveryPOST,PUT,PATCH, orDELETEthat can be retried by a client (mobile network hiccup, load-balancer retry, user double-tap) must be wrapped by idempotency middleware. Shape: a dedicated table,PK = <user_id>:<METHOD>:<path>:<client-supplied-uuid>, attributesstatus,response_status,response_body,expiration(epoch seconds). The middleware does a conditionalPutItemwithattribute_not_exists(id)before the handler runs; on a conflict, it reads the cached response back and returns it. TTL is 24 hours; PITR off (the table is a cache). This is the mechanism that keeps a retried "create order" from double-charging a customer or double-decrementing inventory — the guaranteeTransactWriteItemsgives within a single call but not across retries.Guard mutating writes with conditional expressions. Two halves — both are mandatory guidance whenever this pattern comes up.
Half A —
attribute_exists(<pk>)onUpdateItemunless you explicitly want upsert.UpdateItemdefaults to create if missing. APATCH /orders/:idwith a guessed or staleorder_idsilently materializes a phantom order row — no error, no log, no recovery. AddConditionExpression: attribute_exists(<partition_key>)to everyUpdateItemthat operates on a row the caller is asserting exists; catchConditionalCheckFailedExceptionand translate to 404.Half B —
attribute_not_exists(<pk>)onPutItemwhen you mean "fail on overwrite."PutItemdefaults to overwrite if present. APOST /ordersthat should create a new order (or a new review, a new courier-assignment, a new uniqueness sentinel) will silently clobber an existing row keyed the same way without a guard. AddConditionExpression: attribute_not_exists(<partition_key>)to everyPutItemthat should fail if the key already exists; catchConditionalCheckFailedExceptionand translate to 409 Conflict.Conditional writes are where invariants live in the data layer. When you recommend one half, surface the other — design reviews routinely catch the missing
PutItemguard in apps that added theUpdateItemguard years earlier.TTL attributes are Unix epoch seconds. Three common format mistakes silently break expiration — enumerate all three whenever diagnosing a "TTL isn't deleting anything" report. DynamoDB TTL reads the attribute as a number and interprets it as seconds since epoch. Any of the following causes TTL to silently never fire (no error, no log — the sweeper just skips the row):
Milliseconds instead of seconds.
Date.now()in JavaScript returns milliseconds; written unchanged, the stamp is ~1000× too large and resolves to a year-50,000 timestamp that will never be in the past. Fix:Math.floor(Date.now() / 1000)in JS,int(time.time())in Python.ISO 8601 strings instead of numbers.
datetime.datetime.now().isoformat()in Python ornew Date().toISOString()in JS produces a string like"2026-05-13T09:45:00Z". DynamoDB TTL reads strings as non-numeric and skips the row entirely.Wrong attribute name. TTL is configured per-table with a specific attribute name (
ttl,expiration,expires_at, etc.). If the app writesexpires_atbut TTL is configured onttl, or if a typo splits the attribute, TTL scans an attribute that doesn't exist and skips the row. Verify the configured attribute name in the table's TTL specification matches what the application writes, exactly.
Validate at write time: flag any TTL value greater than
current_time + 50 yearsas a likely milliseconds-vs-seconds bug before it reaches the table. Standardize on the attribute name across the codebase. This pairs with Mechanics #12: the "filter on read" discipline still applies because even when TTL is configured correctly, deletion is eventual.Do not persist a
Notificationstable for transient push/email/SMS. The reflex to create a table for every domain noun is a trap here. If notifications are dispatched-and-forgotten — push to APNS/FCM, transactional email via SES — the correct shape is EventBridge → SQS → NotificationLambda → delivery service. DynamoDB is reserved for state that needs to be queried later. Only create aNotificationstable if you have a genuine read access pattern against it — a per-user in-app inbox, a delivery-status dashboard, a compliance audit. Otherwise you are paying DynamoDB write cost to never read the result. Per-user inbox, if needed:PK = user_id,SK = <createdAt>#<notificationId>, TTL on older entries.Plan cascade deletes as chunked
BatchWriteItem, not a sync API call. Deleting a parent entity in a hierarchical schema (a board, an order, a tenant, a user under GDPR) requires deleting everything keyed under it. Do it withBatchWriteItem(25 items per batch) paginated withQuery(..., Limit=25, LastEvaluatedKey=...). This is a background job, not a synchronous API response — at scale the walk is long enough to exceed a typical Lambda timeout, and partial failures must be retryable. If GDPR or regulatory deletion is a requirement of the domain, model this job explicitly as part of the design; do not assume it will fall out of the schema for free.
Cost estimation
Designing and discussing the data model is the default, no-AWS path, and a user is often there to explore options — keys, GSIs, single- vs multi-table, projection tradeoffs — without yet wanting a dollar figure. Do not run the calculator reflexively on every design turn. Produce the monthly cost estimate when either is true: (a) the user asks what it costs (or for the cost report), or (b) the design is being finalized — they signal they're settling on it, taking it to a review, or otherwise want the numbers committed. Until then, stay in design discussion.
Skip the estimate entirely when no full design was produced — a single-query sizing, a hot-partition debugging thread, a pointed mechanics question. In those cases there is nothing to estimate.
When you do produce the estimate, three steps are not optional: (a) write dynamodb_data_model.json, (b) invoke ${SKILL_DIR}/scripts/calculate_costs.py, (c) reference the generated cost_report.md in your summary. Computing numbers inline without running the calculator is never a substitute — not even for a "rough" or "realistic" figure, and not even to reconcile a headline number against a stated daily volume (for that, set avg_rps and let the calculator produce the expected-volume scenario — see below). The calculator enforces the pricing module that stays calibrated against live billing; doing the arithmetic by hand means the numbers drift silently and the user ships a quote they cannot defend.
The calculator is at ${SKILL_DIR}/scripts/calculate_costs.py. The JSON schema it consumes is documented at ${SKILL_DIR}/references/cost-model-schema.md — read that file the first time you produce a cost estimate.
The workflow:
Serialize the design to
dynamodb_data_model.json. The calculator does not reinvent the design — it reads what you already produced. The access-pattern list you built (per Mechanics #2, #18) is a near-literal translation into the JSON'saccess_patternsarray; the schema you built (per Data modeling #3, #7, Mechanics #4, #7) translates intotables,gsis, and — when the design has one —vector_indexes. If a field the calculator wants is missing (most commonlypeak_rpson a pattern), that is a gap in the design itself — close it by asking the user or by making a defensible assumption and labeling it as one, not by writing0.A vector index goes into the model as a
vector_indexesentry, and its search goes in as aSearchVectorsaccess pattern carryingtop_k. The calculator supports both (full shape in${SKILL_DIR}/references/cost-model-schema.md). Mind the asymmetry with the service API: you create an index with theVectorIndexesCreateTable parameter (Fact #10), but you cost it with thevector_indexesmodel field. Knowing the API shape is not the same as knowing the model shape, and conflating the two is how a vector index ends up costed at zero.If you catch yourself concluding "the calculator doesn't support vector search," the vector index is missing from your model — fix the model, do not fall back to arithmetic. The calculator prices vector index storage and vector write capacity and includes both in the headline; it deliberately declines to price vector search, printing
not pricedwith a footnote and the measured drivers instead. Both halves are already handled, so hand-computing either is exactly the inline arithmetic the rule above forbids. This is not a small slip: on a 1024-dimensionALL-projected index over ~3 KB items, vector writes are typically the largest single line in the estimate — measured at $1,942/month at 200 rps against $972/month for the base-table writes themselves — becauseALLcopies the entire item into the index on every write. Eyeballing that figure can understate the true top cost driver by more than an order of magnitude.When the user states an average or daily volume, set
avg_rpson the pattern — do not hand-compute a "realistic" figure. The headline is peak-sustained (every pattern atpeak_rps, 24/7); a user who hears a large peak-sustained monthly figure for a 15K-orders/day workload needs the expected number too. Settingavg_rps(e.g. 15,000 orders/day ÷ 86,400 s ≈ 0.17 rps average, vs a 600 rps Black-Friday peak) makes the calculator emit a second Expected Monthly Cost headline at that average rate. This is the only sanctioned way to produce a realistic-volume number — reconciling the peak headline against a daily volume with mental arithmetic is exactly the inline-arithmetic the rule above forbids.Run the calculator. Write the JSON to a workspace path of your choosing and invoke:
python3 ${SKILL_DIR}/scripts/calculate_costs.py --model /path/to/dynamodb_data_model.json --output /path/to/cost_report.mdThe JSON is an intermediate artifact. You do not need to present it to the user unless they ask.
Surface the cost report. Hand back
cost_report.md— either inline (if short) or as a file pointer. Highlight the top drivers the report surfaces (the "cost patterns sorted descending" table), and call out the assumptions you baked in (RPS numbers you estimated, retention you assumed, consistency choices). The disclaimer at the top of the report is part of the artifact — do not strip it.Use the cost report to challenge the design, not just to report the number. A cost estimate is diagnostic. If one pattern dominates the bill — the classic case is a high-frequency write like a GPS ping — that is a modeling signal, not just a line item: revisit the aggregate choice (Mechanics #1), the projection shape (Mechanics #7), whether a sparse-GSI-on-transition would cut amplification (Mechanics #6), or whether the pattern belongs outside DynamoDB entirely (Integration #8). Propose the alternative and re-run the calculator on the revised model. This is the main value of having the calculator bundled: the feedback loop from "here's the design" → "here's the bill" → "here's the cheaper design" becomes a turn or two, not a separate exercise.
The calculator models request and storage cost only. It does not model Streams read/write costs, PITR, backup, DAX, or data transfer. If those line items are likely material for the workload, call that out alongside the estimate so the user knows what the number excludes. Storage is priced at the full public rate — the 25 GB free tier is not assumed (it is account-wide and often already used), so never tell the user "storage is free." Set a write pattern's retention_days to size its stored data; the calculator honors it. The storage figure is a bounded-retention snapshot at that retention_days — for a table with no TTL / "kept forever" data, real storage keeps growing past the snapshot, so do not reframe a steady-state storage number as "after N years": either state the retention window the number assumes, or note that a no-TTL table accumulates without bound.
Name the artifacts in your final response. Do not let the tool calls disappear into prose. The user reads your natural-language output, not your shell history. After you run calculate_costs.py, your response must name (a) the JSON file you wrote (dynamodb_data_model.json), (b) the script you invoked (${SKILL_DIR}/scripts/calculate_costs.py), and (c) the report file it produced (cost_report.md). A response that shows cost numbers but does not name the artifact files reads as if you computed them by hand — the workflow evidence has to be in the prose, not just the tool trace.
Live validation
After the cost estimate, offer to run a live validation — deploy the tables and GSIs to an AWS account the user nominates and drive scaled-down traffic against them to measure real per-operation capacity and round-trip latency. Ask in one line, and run only on explicit agreement. Even in that one-line offer, name the two constraints the user needs in order to opt in responsibly: it runs only against a sandbox/testing account (never production or real user data), and cleanup is their responsibility (the skill hands back a teardown.sh and never auto-deletes). The full four-fact disclosure still comes before any deploy (below) — this is just so the one-line offer isn't misleadingly light. Skip for narrow questions — single-query sizing, pointed debugging, pointed mechanics — where no full design was produced and there is nothing to benchmark.
Before running, make the safety expectation explicit in your own wording (the deploy_model.py script also prints a warning banner, but that's a belt-and-braces backstop — the agent is responsible for setting the expectation). State the following four facts verbatim in your response; do not paraphrase them into a shorter summary:
Live validation creates real DynamoDB tables, a Lambda function, and an IAM role in the AWS account the user nominates.
It incurs real (small) AWS charges — single-digit cents in practice, but a real bill.
The account must be a testing or sandbox account — never a production account, never an account holding real user data.
The skill does not auto-teardown — at the end, the user is handed a
teardown.shscript to review, and the skill will execute it on their behalf only after explicit review + intent (see step 6 below).
After stating those four facts, ask the user to explicitly name the profile, the region, and confirm the account's purpose. Only then produce the benchmark_config.json and invoke deploy_model.py. If the user hasn't named a testing account, stop and ask. Shortening this to "this creates real resources, confirm your profile" is not sufficient — enumerate the four facts because the user often doesn't know which specific resources get created or that teardown is their responsibility.
The step differs from the cost estimate in two respects that change how it must be run:
Real resources, real bill. A scaled-down run costs single-digit cents in practice, but against a real account. Never run silently. Always require an explicit AWS profile, region, and caller-identity confirmation.
Teardown is two-phase. Phase 1: the skill generates
teardown.shand hands it to the user. Phase 2: the skill may execute it — but only after the user has explicitly stated they reviewed the script AND explicitly directed the skill to run it. The skill never callsDeleteTableon its own initiative, and never runsteardown.shon the basis of a bare "go" or "proceed."
Refuse to start by enumerating the four preconditions explicitly. Emit them as numbered items in your response; do not fold them into a generic "I need more information" paragraph. Surface the specific precondition that failed so the user knows what blocked the run:
access_patternsis empty, or any pattern haspeak_rpsmissing or0→ refuse with Mechanics #2 framing: "Unknown RPS is a design gap, not a benchmark input. Pattern<id>has no declared RPS — please supply an estimate before benchmarking."boto3is not installed → print exactly:pip install boto3>=1.34.AWS credentials are missing or expired → print the exact remediation, e.g.
aws sso login --profile <profile>for SSO, or the specific boto3 exception's suggested fix.Caller identity's account alias or ARN contains
prod,production,prd, orlive→ refuse and tell the user to supply a sandbox/testing profile.
Check all four before writing benchmark_config.json or invoking deploy_model.py. A generic "I need more info" paraphrase does not satisfy the contract — enumerate the specific precondition that failed.
Workflow, once the user agrees:
Reuse
dynamodb_data_model.jsonfrom the cost estimate. Do not re-serialize.Before deploying, confirm every numeric/binary key declares its type. The deploy builds each table's
AttributeDefinitionsby looking up every key attribute (table + GSI partition/sort keys) inentities[].attributes[]andattribute_definitions, defaulting any it can't find toS. A numeric epoch sort key or numeric id left undeclared deploys as a string and then rejects real writes withValidationException. Per Data modeling #11, ensure each such key carries"type": "N"(or"B") in the model before this step — and don't trust a clean seed as proof, since the failure can surface only on live writes. If you want belt-and-suspenders,DescribeTableafter deploy and confirm the key'sAttributeType.Scope the run when the design has many patterns — warn, estimate, and get confirmation. The benchmark drives patterns serially (each pattern runs its own warmup + measurement window one at a time), so wall-clock grows with pattern count: roughly
table_settle + Σ_patterns (warmup + duration). At standard windows (~10s warmup + ~60–90s duration) a 6-pattern design is a few minutes, but a 20+ pattern design is 25–45+ minutes and gets split across several sequential Lambda invocations. Before launching a run on a design with more than ~8 patterns: state a rough wall-clock estimate and get explicit confirmation ("This will benchmark all 24 patterns and take ~35–45 minutes — proceed, or would you rather test a focused subset?"). Don't silently kick off a 40-minute run.Offer to load-test only the critical patterns, and choose them with the user. A full live run rarely needs every pattern — most are cheap structured lookups whose per-op cost the calculator already nails. Offer to benchmark a focused subset and collaborate on which ones, steering toward the patterns where a live measurement actually buys something:
Highest-throughput / firehose writes — the ones whose RPS dominates the bill or stresses capacity (e.g. a location-ping write).
The transactional / multi-item core —
TransactWriteItems, multi-table writes, the checkout path (transaction 2× cost, Mechanics #18, is worth seeing live).Novel or risky key shapes — anything with a fan-out read (N parallel Queries), a hot-key risk, or a GSI you're unsure about; skip the boring
GetItem-by-id patterns.Cost-dominant patterns — whatever the cost report flagged as the top drivers.
To run a subset, benchmark a trimmed copy of the model: keep all
tables(so the schema deploys and seeds correctly) but pruneaccess_patternsto the chosen few. The cost estimate is unaffected — it always runs on the full model against declared peak; subsetting only narrows the live driven traffic, not the design or its costing. Name which patterns you're testing and which you're skipping (and why) so the user sees the coverage tradeoff.Write
benchmark_config.json— profile and region are the only fields you must set (resource_prefixis auto-generated by the deploy when omitted; everything else has a mode-driven default). Read${SKILL_DIR}/references/performance-model-schema.mdthe first time. Ask the user which mode to run BEFORE writing the file — do not pick a default silently. For a one-shot live validation, the choice is between the two unit-cost modes below; the third mode,"representative", is the iterative design loop's mode (hot-key skew at bounded scale) and is described under Iterative design loop — offer it only when the user wants to probe hot-partition/throttle risk, not for a plain cost-validation run.State the driven throughput honestly, and offer to drive harder. This is the most important thing to get right about live validation. By default
quick/standarddrive only a small fraction of declared peak (scale_factor~0.01, floored atmin_rps_per_pattern) — for a design whose peaks are modest this floors to ~1 request/second per pattern. That is a unit-cost sample: it validates per-operation capacity (cost) and unloaded latency, and it is cheap (pennies, ~1–2 min/pattern). It does NOT exercise load — at ~1 rps, zero throttles is mathematically guaranteed (on-demand baseline is ~2,000 WCU / ~4,000 RCU) and the latencies are best-case. So when you offer the run, say which it is, and offer the alternative:Unit-cost sample (default) — validates cost + unloaded latency. Cheap. Does not test load/throttling.
Drive at/near declared peak — a real load test. Set
scale_factor(fraction of declared peak, e.g.1.0for full peak) ormax_rps_per_patterninbenchmark_config.json; both are honored. Bounded by one in-region Lambda's ceiling (~1,500–2,000 rps/pattern) and gated bycost_guardrail_usd+--allow-spend(so an expensive run can't fire silently). Caveat: a freshly-created on-demand table starts at ~2,000 WCU / ~4,000 RCU baseline and adapts upward with a lag, so a high rate immediately after deploy measures cold-start adaptive-capacity warm-up, not steady state — the run already hasramp_seconds+table_settle_secondsto absorb some of this; for a clean high-rps read use a longerduration_seconds.representativemode — the loop's hot-key/throttle-risk mode (zipf skew at bounded scale); use it when hot-partition behaviour is the question (see Iterative design loop).
The extrapolated monthly cost is identical across
quick/standardand at any throughput — per-op CU is deterministic and the cost extrapolation always uses declared peak, so a higher-throughput run buys you latency-under-load and throttle signals, not a different cost number. Always tell the user the per-pattern rate you will drive before you run. Emit the mode options as a literal question and wait for the user to pick:Which mode should I run? Pick one:
"quick"— ≈1 minute per pattern window, a smoke test that confirms the design deploys and runs. Percentiles are less stable. A unit-cost sample (drives ~1 rps/pattern for a modest-peak design): validates cost + unloaded latency, not load."standard"(default) — full-length windows, stable p50/p95/p99, trustworthy extrapolated monthly cost. Still a unit-cost sample by default (same ~1 rps/pattern) — the cost number is identical toquick; you get tighter latency percentiles, not a load test. To make it a load test, tell me to drive at/near your declared peak. (There is also"representative"— hot-key/throttle-risk mode for the iterative design loop. Mention it only if hot-partition behavior is the question; see Iterative design loop.)
Write the user's chosen value into
benchmark_config.json'smodefield. If the user skips the question and just says "go," default to"standard"and name that default in your reply ("I'll runstandardmode since you didn't specify — interrupt if you meantquick").Deploy:
python3 "$DDB_SKILL_DIR/scripts/deploy_model.py" --model … --config … --manifest-out created_resources.json --yes-deploy. Prints caller identity for confirmation, aborts on prod markers, prefixes every resource withddb-skill-bench-<date>-<uuid8>-(generated for you if you didn't setresource_prefix), tags it with the run id. On an interactive terminal it waits a few seconds for a Ctrl-C abort after printing the account; in an agent/CI run it proceeds immediately on the--yes-deployconsent.Benchmark:
python3 "$DDB_SKILL_DIR/scripts/benchmark_model.py" --model … --config … --manifest created_resources.json --raw-out perf_raw.jsonl --summary-out perf_summary.json. Handles cold-start discipline automatically (see below).Execution discipline — run it in the foreground and wait. A
representativerun drives many minutes of traffic (≈ settle + per-pattern warmup + per-pattern duration across every pattern — often 15+ minutes for a multi-pattern design). Run the benchmark as a single blocking call and wait for it to finish — do NOT background it and hand control back to the user. A backgrounded run can be killed when the surrounding turn/session ends, leaving the previous run'sperf_summary.jsonin place — which is exactly how a stale summary gets read as if it were fresh. If the run risks exceeding your tool's wall-clock limit, raise the tool timeout — do not shrink the windows or background the job. You do not need to hand-sizeduration_secondsto fit a budget: the heavy work runs inside the Lambda (≤900 s server-side) andbenchmark_model.pyalready splits the load across sequential invocations (_compute_split). When it finishes, note the UTC time you launched it; you'll check the summary'sbenchmark_completed_atagainst it in step 5.Report:
python3 "$DDB_SKILL_DIR/scripts/generate_perf_report.py" --model … --summary perf_summary.json --output performance_report.md.Verify the summary is fresh AND matches the experiment you described before you interpret it (Operating discipline, applied).
perf_summary.jsonrecordsbenchmark_completed_at(UTC) and theconfigthe run actually used. Before generating or trusting the report, confirm (a)benchmark_completed_atis newer than the launch time you noted in step 4, and (b) the summary'sconfig(mode,provisioned_capacity,duration_seconds) matches what you set for this run. If either fails — or the field is absent — the benchmark did not complete and you are looking at a stale file: do not interpret it, re-run. The report's Deployment block surfacesbenchmark_completed_atand the run window so the user can see the same thing. The tell that you're reading stale data: benchmark numbers that are byte-identical to a prior run you expected to differ.Run what you said you'd run — verify config-match, not just freshness. If you told the user this run would change something (raise the provisioned ceiling to isolate skew, pin a subset of patterns, drive a higher rps, switch modes), then read the actual
configblock in the freshperf_summary.jsonand confirm it reflects that change before you report. A run that is fresh but used the old config is not the experiment you promised — interpreting it answers a different question than the one you set up. If you find you described an experiment you did not actually configure, say so plainly and either re-run with the correct config or, if the run is physically incapable of producing the signal regardless (e.g. a unit-cost/floor run at ~1 rps cannot drive any partition key hot no matter how you set capacity, so it can never isolate skew — only representative mode or driving at/near declared peak can; raising a provisioned ceiling on a floor run changes nothing — Mechanics #3, and Unit-cost / floor runs below), state that instead of quietly substituting a same-config re-run. Never present a re-run as testing a variable you did not change. The catch in practice: the user notices two "different" runs producing near-identical numbers — get ahead of it by checking the config yourself.⚠ DO NOT run
rm,rm -f,ls,find,aws dynamodb delete-table, or any other ad-hoc delete command when the user asks to "clean up," "tear down," "remove," or "get rid of" the benchmark resources. "Teardown" in this skill means invokinggenerate_teardown.pyto produceteardown.sh, never running a shell command yourself to delete local files or AWS resources. The teardown protocol is step 6, below. Local artifact files (performance_report.md,cost_report.md,dynamodb_data_model.json, etc.) are the user's to keep or delete — the skill never cleans them up.Teardown — generate first, execute only after explicit review + intent. When the user asks to tear down, clean up, or remove benchmark resources:
a. Generate the script with
python3 ${SKILL_DIR}/scripts/generate_teardown.py --manifest created_resources.json --out teardown.sh. This writesteardown.sh. It does not delete anything.b. Surface the script path, the review command, and the execute command in your reply. Verbatim form:
Review:
cat teardown.sh(or open in an editor) — the script deletes DynamoDB tables, a Lambda function, and an IAM role under theddb-skill-bench-prefix.Execute:
bash teardown.sh --confirm— the manifest's account, region, and prefix are baked into the script at generation time, and it re-checks caller identity + prefix before deleting anything. (Pass--dry-runinstead of--confirmto preview deletions; add--delete-logsto also remove the Lambda's CloudWatch log group — that log deletion is irreversible.)
c. Do not conflate "teardown" with "local cleanup." Never run
rm,rm -f performance_report.md,ls, orfindthinking the task is removing local files. Onlyteardown.shdeletes AWS resources; local files are not the skill's concern.d. The skill may execute
teardown.shon the user's behalf, but only under this exact two-part condition:The user's message must state they reviewed the script (examples: "I read it," "I reviewed teardown.sh," "I checked the contents," "looked it over"). Inferring review from context does not count; it must be stated.
The user's message must state the intent to run it (examples: "run it," "execute it," "go ahead and tear down," "proceed with the teardown").
e. When the user asks the skill to run teardown without stating review, the skill must not execute. Instead, surface the exact review command (
cat teardown.sh) and ask the user to confirm they reviewed it. One required response shape:Before I run
teardown.sh, please confirm you've reviewed the script. Runcat <PATH>/teardown.sh— it deletes the DynamoDB tables, Lambda function, and IAM role created under theddb-skill-bench-prefix. Once you've read it and want me to execute, reply with something like "I reviewed it, go ahead" and I'll runbash teardown.sh --confirm.f. When the user asks the skill to skip the review ("don't worry about it," "just run it without showing me," "trust it"), refuse. Destructive actions require the user's attested review. Surface the refusal and re-offer the review command.
g. When the conditions in (d) are met, the skill may run
bash teardown.sh --confirm, surface the full stdout/stderr, and report which resources were deleted.
Operations execute as declared, never substituted. When the design says BatchWriteItem, the benchmark issues BatchWriteItem in 25-item batches. When it says TransactWriteItems, the benchmark issues TransactWriteItems with the declared item set. Single-item substitutes (a loop of PutItem for BatchWriteItem, or individual PutItem calls for TransactWriteItems) are not used — they measure a different design. Transaction 2× write cost (Mechanics #18), batch partial-failure semantics, and transaction bounds (Mechanics #14) are exactly what live validation observes; substituting them defeats the purpose. If a user asks how the benchmark exercises batch or transactional patterns, state plainly that the declared operation runs as-declared and name the specific bounds the benchmark will observe.
Cold-start discipline. DynamoDB latency and throughput are not stable at t=0 on a new table; numbers collected without discipline here will misleadingly look worse than steady state. Three classes of bias must be removed before measurements are trustworthy:
Fresh-table capacity. On-demand tables start at baseline capacity (~2,000 WCU, ~4,000 RCU); traffic above baseline in the first minutes can throttle until adaptive capacity scales. After
table_existsreturns, wait at leasttable_settle_seconds(default 30) before any measurement call.Partition priming and client warmup. Seed writes warm the partitions the benchmark will later hit, and a fresh boto3 client carries a TLS handshake + credential refresh on its first call (tens to hundreds of ms). The benchmark pauses briefly between seed and measurement, and issues one
DescribeTableper worker thread, so those one-time costs don't show up in the percentiles.Per-pattern warmup window. Every pattern runs a
warmup_secondswindow (default 10) ahead of its measurement window. Warmup calls are recorded withphase: "warmup"and excluded from latency percentiles and the cost extrapolation. The perf report surfaces warmup p99 separately — if it's materially higher than steady-state p99, that itself characterizes cold-deploy behavior and is worth calling out.
What is and is not measured:
Measured: per-op consumed RCU/WCU from
ReturnConsumedCapacity; round-trip latency p50/p95/p99 (warmup and steady-state reported separately); throttle counts; GSI write amplification; even-distribution signal per pattern; observation-based extrapolated monthly cost.Not measured: TTL sweep cadence, stream consumer latency and cost, autoscaling under sustained burst, cross-region replication lag, long-tail traffic shapes beyond the measurement window, and any non-DDB service in the design (Lambda, API Gateway, SQS, EventBridge Pipes, OpenSearch, etc.). The perf report lists these under "Designed, not benchmarked" so the user sees plainly which parts of their architecture the numbers cover.
The report format is documented in ${SKILL_DIR}/references/performance-report-format.md. It mirrors cost_report.md's disclaimer-first, table-driven style and prints both observed and calculator-expected numbers per pattern so the user sees where their inputs (RPS, item size, consistency) diverged from reality.
Reflect on the findings and offer an iteration loop. After the report is written, read it as a diagnostic — not a final score — and ask: is there an alternative design that the measurements argue for? For each finding, classify it:
Input-accuracy finding — the observation diverged from the calculator because a declared input (RPS, item size, consistency,
conditional_fail_rate) was off. The design itself is fine; the estimate needs updated inputs. Propose updating the JSON and re-running the calculator only (cheap, no redeploy).Design finding — the observation reveals a structural issue that persists regardless of input accuracy. Examples and the axioms that govern them:
Persistent throttles that scale with partition-key skew → Mechanics #3 (per-partition ceilings, write-shard the hot key). Before attributing throttles to a hot partition, run the skew-vs-starvation gate — both parts must pass: (1) Differential test — the hot partition's p99 must be materially above the cold partitions' p99 (the report's ~1.8× rule). If hot p99 ≈ cold p99, every partition throttled uniformly: that is capacity starvation, not key skew — say so, and do not attribute it to the partition key. (2) Uniform-baseline test — using the per-op observed CU the report already prints (not hand-derived item counts), check whether uniform keys at the driven RPS would already exceed the provisioned ceiling; if they would, the run cannot isolate skew — recommend re-running with capacity set above uniform demand so only a genuinely hot partition throttles. Only once both pass is "hot partition at its ceiling" a supported verdict. Note also: write-sharding a mutable partition key (e.g. a status that changes each transition) spreads the Mechanics #3 load but the key still mutates on every write, so the Mechanics #8 (mutable GSI key) deviation persists and must stay recorded per Artifact #5 — eliminating the write amplification needs a separate single-purpose table, not a sharded suffix.
One pattern dominating the bill → Mechanics #1 (aggregate tightness), Mechanics #7 (projection), Integration #8 (move off DDB to Timestream/S3/OpenSearch/Redshift/SageMaker Lakehouse when the workload is analytical, high-frequency telemetry, or full-text).
GSI write amplification higher than the projection implies → Mechanics #6 (sparse GSI when a mutable key isn't always present), Mechanics #7 (downgrade ALL → INCLUDE/KEYS_ONLY), Mechanics #8 (mutable attribute as GSI key doubles the write).
Strong read cost observed where RYW semantics weren't actually required → Mechanics #15 (default to eventual), Patterns around
ReturnValues=ALL_NEWto avoid the post-write re-read.Unbounded query hitting the 900 KB page cap repeatedly → Mechanics #17 (constrain every query; require a time window or max page).
Cold-start p99 materially elevated at on-demand baseline → a capacity-mode or pre-warming decision (Mechanics #19), not usually a schema change.
When at least one design finding is meaningful — meaning the proposed change would plausibly reduce cost by >10%, remove a throttle source, or close an axiom deviation — tell the user:
Which findings point to design changes vs. which point to input accuracy.
For each design finding, the axiom it traces to AND a literal JSON diff on
dynamodb_data_model.json— not a verbal description. Emit the diff as a fenced code block showing the specific fields that change (new GSI object, dropped GSI, changedprojection.type, sharded PK with hash suffix, pattern moved out of theaccess_patternsarray entirely). A verbal suggestion like "consider downgrading the projection" does not satisfy this — the user has to see the exact change so they can accept or reject it. When the diff changes a GSI's key (a sharded partition key is the common case), label it as an additive-GSI migration, not a code-only change, per Fact #9 — the new index is created alongside, populated, cut over, then the old one dropped. Example shape:- { "name": "OrdersByCustomer", "projection": { "type": "ALL" } } + { "name": "OrdersByCustomer", "projection": { "type": "INCLUDE", "non_key_attributes": ["status", "total"] } }The expected effect on the top cost drivers and throttle sources.
Your spoken verdict inherits the report — never exceed it. The chat summary you give the user must not claim an outcome the report did not classify. Concretely:
If the report emitted no
key_skew_patternsfinding, do not say the design "throttles" or "hot-partitions" — the supported statement is the report's own ("no hot-partition distress observed under skew" / "not validated by this run").Restate the report's hedge; do not upgrade it. "Throttles appeared under a deliberately low ceiling" is not "the GSI partition is hot" (see the skew-vs-starvation gate above).
Attribute every number to exactly the claim it supports. A low provisioned ceiling that throttles proves the table was under-provisioned for the driven load, not that any single partition key is hot.
Worked contrast — same data, two verdicts: ❌ "Confirmed: the OrdersByStatus GSI throttles hard under load." ✅ "Under a 50-WCU ceiling, writes to the Orders table throttled — but hot-partition p99 ≈ cold-partition p99, so this is uniform capacity starvation, not key skew. The run did not isolate a hot-partition effect; to test that, re-run with capacity above uniform demand."
Unit-cost / floor runs — do not narrate them as load tests. When the run was a unit-cost sample —
quick/standardmode, or any run where every pattern's drivenbench_rpsequals themin_rps_per_patternfloor (it shows as a uniformbench_rps, typically ~1, inperf_summary.json, and the report's disclaimer says "UNIT-COST sample, not a load test") — you MUST NOT present "0 throttles" or the latency percentiles as evidence the design withstands load. At ≤ a few rps, zero throttles is mathematically guaranteed (on-demand baseline ~2,000 WCU / ~4,000 RCU) and the latencies are unloaded best-case. State what the run actually validated — per-op capacity (cost) and unloaded latency — and that throttling/ceilings were "not exercised — see the report's Not-validated list." If you reassure the user that throttling isn't a risk, give the structural reason, not the zero count: e.g. a high-cardinality partition key (user_idacross many users) spreads load so no single partition gets hot (Mechanics #3). To actually exercise throttling, drive at/near declared peak or use representative mode.Worked contrast — a ~1 rps habit-tracker run: ❌ "Validated against live DynamoDB — 0 throttles, reads <8 ms, writes <50 ms. Ready for review." ✅ "This validated per-op cost (within 2% of the model) and unloaded latency (reads <8 ms). It ran at ~1 rps, so it did not test throttling — but throttling isn't a concern for this design anyway: the partition key is
user_id, so load spreads across users and no single partition gets hot. To prove peak behaviour, we'd re-run driving your declared peak rps."
Reconcile any number the user quotes back against your own latest artifact — do not just agree. When the user restates a figure ("$189/mo is fine," "so it's ~600 WCU on that partition," "the 1,884 throttles") before you respond, check it against the number actually in your current
cost_report.md/perf_summary.json/performance_report.md— after confirming that artifact is itself fresh per step 5, since a number matching a stale summary is not a real reconciliation. If it doesn't match — even when the user sounds confident, and especially when accepting it would be the agreeable move — correct it explicitly and cite the artifact: "Quick correction: the report says $835/mo expected, not $189 — the $189 was an earlier figure. Still want to proceed?" Silently accepting a wrong number the user will then carry into a budget deck or a design review is a worse failure than a moment of friction. This applies to costs, capacity math, throttle/latency counts, and item-size assumptions alike. A user-supplied number is an input to verify, not a verdict to ratify.Do not vouch for a claim the run did not establish. Your spoken verdict may assert only what the artifact in front of you actually shows. If a number you're about to state came from your own head rather than the report (a hand-extrapolated provisioned/reserved cost, a "scales linearly so 25k rps is fine" leap from a 2k-rps run, a per-partition figure the benchmark never isolated), label it as reasoning, not measurement, and give the user the structural argument behind it — never dress an estimate as a validated result. "The calculator only models on-demand; the reserved figure is my hand-calc for a provisioned alternative (reserved capacity applies to provisioned only — Fact #7) from published rates — confirm it against the AWS calculator before the deck" is honest. Presenting that hand-calc as "validated" is not.
Then ask the user whether they want to iterate: apply one or more changes and re-evaluate. Offer three levels of re-evaluation and let the user pick. Emit the three options as three literal bulleted items — do not collapse them into prose, do not drop any of the three, and do not re-order them. The user picks one by name.
Calculator only — update the JSON, re-run
calculate_costs.py, show the newcost_report.mdand a side-by-side delta. Cheap, no AWS calls, no teardown. This is the default and should be the first offer.Calculator + live validation — do the above, then offer a second live run against the revised design. Requires the user to re-consent to AWS deployment and accept a second teardown script; the prior teardown should be run first unless the change is additive-only.
No changes — the user may reasonably decide the findings are acceptable, the cost of change outweighs the savings, or the proposed alternatives don't fit a constraint the skill doesn't know about. Record the decision as a "deviation" per Artifact #5 with the user's stated reason.
Do not iterate silently, do not iterate unboundedly. Each iteration is explicit and opt-in; if the user declines or the report surfaces no meaningful findings, stop and hand back the final artifacts.
Iterative design loop
The three re-eval levels above are run as rounds of a human-driven loop, orchestrated by ${SKILL_DIR}/scripts/iterate_design.py. This is the seamless path for "deploy → test at representative scale → get feedback → change the design → test again."
What it is — and is NOT. One invocation = one round: optionally apply a user-agreed change → decide reuse-vs-redeploy → (gated) deploy or reuse → benchmark at representative scale → report → surface findings + suggestions → STOP. The loop is human-driven: you run one round, present the findings and concrete suggestions, and hand back. The user decides what to change and prompts the next round. Never self-iterate to convergence; never start a second round in one invocation; the script enforces this (it has no logic to propose or select a change — it only applies the change it is handed).
Representative mode (honest scope). The loop runs mode: "representative" (see ${SKILL_DIR}/references/performance-model-schema.md): ~0.10–0.25× declared peak with zipf hot-key sampling and realistic item-collection cardinality. This surfaces what the 1%-scale unit-cost run cannot — hot-partition throttling (Mechanics #3), throttle-under-load, GSI amplification at volume, and Query-at-realistic-cardinality. State the scope plainly to the user: one in-region Lambda drives ~1,500–2,000 RPS/pattern max (32 driver threads), so representative scale is bounded — it surfaces hot-partition/throttle risk at bounded cost; it does not prove the design sustains declared peak. Per-op cost extrapolation stays linear and valid (per-op CU is scale-invariant, extrapolated against declared peak); the throttle/latency numbers are load-risk signals in the report's "Load-risk signals" section and must never be extrapolated linearly.
Exceeding one Lambda's ceiling (not yet built). If a design needs more than one Lambda's ~1,500–2,000 rps/pattern, the path is to invoke the same deployed benchmark function concurrently (Lambda scales horizontally for free — no need to deploy multiple functions), each invocation driving target_rps / N of the load, with seeding done once and the results merged (calls/throttles/errors add; per-op CU is a call-weighted average; latency percentiles are recomputed from the pooled per-call latencies the raw rows already carry). This load-sharding is not implemented yet — today a single Lambda is the ceiling. Don't promise rates above it; offer representative mode or note the ceiling instead.
Gated consent every round. A real deploy and a real teardown each require explicit user consent, every round. The four live-validation facts above (real resources, real charge, sandbox-only, no auto-teardown) apply per round. iterate_design.py refuses to deploy without --yes-deploy; the benchmark refuses to spend above cost_guardrail_usd without --allow-spend. The two-phase attested teardown (Live validation step 6) is unchanged and remains the only teardown path — a schema-changing round tells the user to run the prior teardown.sh first so no resources are orphaned.
On-demand vs. provisioned, and why throttles may not appear. Representative runs deploy on-demand (PAY_PER_REQUEST) by default, matching the skill's Mechanics #19 default. On on-demand, adaptive capacity isolates and absorbs a single hot key — a hot partition usually shows up as elevated tail latency on that partition, not throttles, because the table auto-scales to serve it. The key_skew_patterns signal therefore fires on either throttles or a hot-partition p99 materially above the cold-partition p99 (a baseline-free, within-pattern comparison), so on-demand designs still get the warning. To observe hard throttling, set provisioned_capacity: {"read": N, "write": M} in benchmark_config.json (or per-table) — a low provisioned ceiling has no adaptive absorption, so a hot partition throttles deterministically; this is also how you validate a planned provisioned capacity. The benchmark Lambda runs with client-side retries disabled so throttles are observed and counted, not silently retried away. Be aware a freshly-created provisioned table carries ~5 min of burst capacity that masks throttling at the very start of a short window — use a longer duration_seconds or a lower capacity to drain it.
The round protocol you follow:
Invoke
python3 "$DDB_SKILL_DIR/scripts/iterate_design.py" --model dynamodb_data_model.json --config benchmark_config.json --loop-state loop_state.json --manifest created_resources.json [--apply-change change.json] --mode representative [--timestamp <iso>] [--yes-deploy] [--allow-spend].--timestampis optional — omit it and the round stamps itself with the current UTC time; pass an ISO string only if you need the loop-state entries to carry a specific clock (e.g. reproducible tests). A round runs a representative benchmark, so the same execution discipline as Live validation step 4 applies: invoke it as a single blocking foreground call and wait — never background it (a killed round leaves stale artifacts) — and raise the tool timeout rather than shrinking windows if needed. Before relaying any result, verify freshness as in step 5 (the round'sloop_state.jsonheadline anddesign_findings.jsonderive from the sameperf_summary.json; if it didn't refresh, the round didn't complete).Read only the compact artifacts:
design_findings.json,cost_report.md, andloop_state.json. Never readperf_raw.jsonl(large) orperformance_report.md(human-facing).Relay the round's compact summary and the delta-vs-prior-round from
loop_state.json(e.g. "sharding the PK cut W1 throttles 1450 → 0, p99 −70 ms"). Present each suggested change as a literal JSON diff ondynamodb_data_model.json(the same diff requirement as the iteration offer above). Then STOP and ask the user what to change, or whether to accept.When the user picks a change, pass it as
--apply-changenext round (amergeobject or anopslist); record their decision for the prior round inloop_state.json'suser_decision.
How the three re-eval tiers map onto the loop: "Calculator only" → iterate_design.py --calculator-only (applies the change, re-costs, no AWS); "Calculator + live validation" → a full representative round (gated deploy or reuse); "No changes" → record user_decision as a deviation per Artifact #5 and stop. The loop extends the offer, it does not replace it. quick/standard modes remain for one-shot unit-cost validation; representative is the loop's mode. Loop state is documented in ${SKILL_DIR}/references/loop-state-schema.md.
Security considerations
Security is woven through the axioms above (authorization-aligned keys in Data modeling #14, conditional-write guards in Patterns #2, transactional uniqueness in Mechanics #13). This section consolidates the risks an agent following this skill MUST surface to the user regardless of which axiom is in play, plus the controls specific to the skill's executable scripts. State the relevant ones in any design review; do not assume the user already knows them.
Authorization-boundary alignment (IDOR) is the first security check. The most common DynamoDB data-exposure bug is a partition key set to a client-supplied identifier (a
review_id/order_idfrom a URL) instead of the server-derived authorization identifier (user_id,tenant_id,account_id). When they diverge, a caller can pass a foreign key and reach data across the tenancy boundary. Align the partition key with the authorization/tenancy boundary (Data modeling #14) so a foreign key is simply unaddressable; a per-callConditionExpressionis discipline, not a structural fix, and does not cover reads (Fact #4). On any design review, evaluate and state this before hot-partition, projection, or cost findings.Encryption at rest — always on; choose the key tier deliberately. DynamoDB encrypts every table at rest with no way to turn it off. The default is an AWS-owned key — no cost, no key management, and the implicit state when no
SSESpecificationis set (which is what the live-validation deploy does; the bench manifest records the at-rest state read back fromDescribeTable). It is right for ephemeral/bench tables and most workloads. For data under a compliance regime (audit of key usage, independent key rotation/revocation, cross-account key policy, or a regulatory requirement to hold the key), recommend a customer-managed KMS key (CMK) — setSSESpecificationwithSSEType: KMSand aKMSMasterKeyId(note: DynamoDB's only non-defaultSSETypeisKMS; there is noAES256type on DynamoDB), and note the per-request KMS cost and the need for a key policy. Surface the encryption choice in the design (it is one of the per-entity operational-config inputs) rather than leaving it implicit. Encryption in transit is always on for every AWS API call these scripts make: boto3 talks to DynamoDB/Lambda/IAM over TLS (HTTPS) by default and there is no plaintext path — agents replicating this pattern should treat all data-in-transit as protected and never disable TLS verification.Generated artifacts are sensitive — treat them accordingly. The deploy writes
created_resources.json, a manifest that contains the AWS account ID, resource ARNs, and the profile name used. It is not a secret store (it holds no credentials), but it identifies the account and resources, so do not paste it into a shared channel, commit it to a public repo, or include it verbatim in a report. The skill's scripts never write AWS credentials, tokens, or connection strings to any artifact — keep it that way; if you extend them, do not log credential material.Benchmark resources go only in a sandbox/test account. The live-validation deploy creates real, billable resources. It must run only against a sandbox/testing account the user names — never production, never an account holding real user data. The deploy refuses on prod-marker account aliases/ARNs (
prod/production/prd/live), but the agent is responsible for stating the four live-validation facts and confirming the account's purpose first (see Live validation).Credential hygiene. Never persist AWS profiles, access keys, session tokens, or
credential_processblobs into any file the skill generates, and never echo them to output. The scripts authenticate from the ambient AWS credential chain (a named profile, SSO, or environment credentials); they neither read arbitrary environment variables wholesale nor record what they resolved. When you writebenchmark_config.json, store only the profile name and region — not the resolved credentials behind them.Vector search does not honour fine-grained access control, and a vector index partition key is not a tenant boundary. Two distinct hazards, both of which MUST be surfaced on any multi-tenant design that includes semantic search — followed by the compensating detective control they require.
First, the
dynamodb:FGAC condition keys —dynamodb:LeadingKeys,dynamodb:Attributes,dynamodb:Select— are absent from theSearchVectorsrequest context. A statement whoseConditionreferences one therefore does not match, and the outcome is denial rather than a grant. Addingdynamodb:SearchVectorsto an existing FGAC-conditioned statement silently breaks vector search while that statement's other actions keep working normally, which presents as a broken index rather than a policy fault. Grantdynamodb:SearchVectorsin its own statement, scoped to the index resource ARN (arn:aws:dynamodb:<region>:<account-id>:table/<table>/index/<index>), with nodynamodb:FGAC conditions attached.Second, scoping searches to one tenant's partition-key value is a data-locality and performance optimization, not an access-control mechanism. Any principal holding
dynamodb:SearchVectorson the index can search any partition-key value, and because the FGAC condition keys do not apply you cannot constrain which values they reach at the IAM layer. Where strict tenant isolation at the data layer is required, use separate tables or separate indexes with distinct per-tenant IAM grants; an application layer that injects the caller's tenant identifier and never accepts it from the client is a complement, not a substitute. This is the vector-search form of the authorization-boundary check in item 1 — and unlike the base table, the key schema does not make foreign data unaddressable here, so the structural fix is grant separation rather than partitioning. Encryption at rest is inherited from the base table; there is no separate setting for a vector index.Third, because grant separation constrains which index a principal may search but never which partition-key values, recommend CloudTrail data-event logging on any multi-tenant vector-indexed table as the compensating detective control. It is the only record of which value each principal searched — the precise dimension IAM cannot gate — so a cross-tenant search from a compromised principal or a failed application-layer injection is otherwise unobservable. A CloudWatch alarm on per-principal
VectorSearchRequestBytescatches partition enumeration as a volume anomaly. Note also that the embeddings are themselves sensitive: similarity results disclose that two records are semantically close even when no raw text is returned, so the vector attribute's exclusion from results by default is a security default worth keeping, and anything logging whole items needs the same care as the table. Full treatment:${SKILL_DIR}/references/vector-search.md § Security.
Best practices
These are recommended defaults, not axioms. They do not shape the design; they are operational safeguards that should be enabled unless a specific reason argues against them. An agent producing a design should turn each of these on unless the justification for disabling is explicit in the artifacts.
Enable deletion protection on every table. DynamoDB's deletion-protection flag blocks
DeleteTableat the service boundary until it is explicitly disabled. It costs nothing, has no performance impact, and prevents an entire class of incidents — an IaC misconfiguration, a copy-paste command, a stack tear-down that should have been scoped — from destroying a production table. The one-step disable is cheap when deliberate deletion is genuinely intended; the protection should be on everywhere else.Two important details people miss:
Deletion protection defaults to FALSE. New tables are created without it unless you set
--deletion-protection-enabledoncreate-table(or the equivalent property in CloudFormation/CDK or any third-party IaC tool). This is why it is so commonly absent on tables that predate the feature or were created without deliberate thought — you have to explicitly turn it on.Service-boundary protection is not redundant with IaC guards. CDK
RemovalPolicy.RETAIN, a third-party IaC tool's prevent-destroy / retain-on-delete guard, and similar mechanisms only protect the IaC path — a directaws dynamodb delete-tableCLI call, a different IaC state file targeting the same resource, or a rogue script all bypass them. Deletion protection lives on the table itself at the service boundary and catches every destruction path. Use both together; do not treat IaC guards as sufficient on their own.
Enable point-in-time recovery (PITR) on every table. PITR provides continuous backup with 35-day retention and per-second restore granularity. Cost is a fraction of storage cost, and the day it is needed — accidental deletion, a bad migration, a logic bug that wrote corrupted values — it is the only thing that recovers the data. Disable PITR only on ephemeral tables whose contents can be rebuilt from another source (idempotency caches, dedupe stores with TTL). PITR divergence across tables is also a modeling signal per Data modeling #3.
Make the table observable — alarms and an audit trail for production tables. A table with no monitoring fails silently; the first sign of throttling or a runaway cost is often a customer complaint or the monthly bill. For every production table, recommend:
CloudWatch alarms on
ThrottledRequests,SystemErrors, andUserErrors(and, on provisioned tables, on consumed-vs-provisioned capacity). At minimum a starter alarm onThrottledRequests > 0sustained — that is the earliest signal of a hot partition or under-provisioning, and it ties directly to the Mechanics #3 ceilings.CloudTrail data-event logging for tables holding sensitive or regulated data, so item-level reads and writes are auditable. CloudTrail captures DynamoDB management events by default; item-level (data) events are opt-in per table and are what an audit or incident investigation actually needs. When the audit/operational logs themselves hold sensitive data, encrypt the CloudWatch Logs log group (and the CloudTrail trail's S3 bucket) with an AWS KMS customer-managed key so the logged data is protected at rest, not just the source table.
These are operational safeguards, not design inputs — but because this skill's guidance is replicated across many customer environments, surfacing them is part of a defensible design rather than an afterthought.
Reference architectures
Worked examples that apply these axioms end-to-end to a concrete application. Consult when a design question is easier to answer against a running model than against the axioms alone.
${SKILL_DIR}/references/reference-architecture.md— a multi-tenant kanban task-board ("TaskBoard") SaaS on AWS backed by DynamoDB. Covers the access-pattern list, table and GSI layout, Streams fan-out to OpenSearch and EventBridge, authorization-aligned partitioning, real-time delivery, and the trade-offs made at each step. Start here when a user asks about a full-stack DynamoDB design, multi-tenancy, or end-to-end composition of the services around the database.${SKILL_DIR}/references/vector-search.md— the vector index API surface (VectorIndexes,VectorIndexUpdates,SearchVectors), SearchSchema design, distance functions and score direction, the readiness predicate, projection and immutability rules, silent de-indexing, IAM including the fine-grained-access-control gap, the byte-based cost dimensions, and a troubleshooting table keyed on the service's verbatim error strings. Read this whenever a design or debugging thread involves semantic similarity, RAG retrieval, recommendations, or AI-agent memory (Fact #10, Integration #8). Skip it otherwise.${SKILL_DIR}/references/cost-model-schema.md— the JSON shape consumed by${SKILL_DIR}/scripts/calculate_costs.py. Read this the first time you produce a monthly cost estimate (see Cost estimation above). Skip it for non-cost questions.${SKILL_DIR}/references/performance-model-schema.md— the JSON shape ofbenchmark_config.jsonconsumed by the live-validation scripts. Read this the first time you run a live validation (see Live validation above). Skip it when live validation is not on the table.${SKILL_DIR}/references/performance-report-format.md— the structure ofperformance_report.mdgenerated by${SKILL_DIR}/scripts/generate_perf_report.py. Read this before interpreting the report or drafting the design-reflection section.${SKILL_DIR}/references/loop-state-schema.md— the shape ofloop_state.json, the compact per-round genealogy file written by${SKILL_DIR}/scripts/iterate_design.py. Read this when running the iterative design loop (see Iterative design loop above).
Files
- ATTRIBUTION.mdAdded by AutoGineer918 B
- LICENSELicence file added by AutoGineer9.9 KiB
- NOTICELicence file added by AutoGineer67 B
- SKILL.md136.7 KiB
- references/cost-model-schema.md24.3 KiB
- references/loop-state-schema.md4.6 KiB
- references/performance-model-schema.md17.2 KiB
- references/performance-report-format.md19.7 KiB
- references/reference-architecture.md54.5 KiB
- references/vector-search.md25.8 KiB
- scripts/benchmark_lambda.pyScript78.3 KiB
- scripts/benchmark_model.pyScript46.2 KiB
- scripts/calculate_costs.pyScript85.1 KiB
- scripts/deploy_model.pyScript46.9 KiB
- scripts/find_skill_dir.shScript3.5 KiB
- scripts/generate_perf_report.pyScript76.5 KiB
- scripts/generate_teardown.pyScript11.0 KiB
- scripts/iterate_design.pyScript29.5 KiB
scripts/calculate_costs.py
#!/usr/bin/env python3
"""DynamoDB Cost Calculator — reads dynamodb_data_model.json directly.
Reads the structured JSON data model (tables, entities, access_patterns),
calculates RRU/WRU consumption and monthly on-demand costs, outputs a
standalone cost_report.md.
Formulas were empirically verified against live DynamoDB
ReturnConsumedCapacity responses. See verify/ for the validation suite.
Rules enforced (all verified):
- WRU rounding: ceil(item_size / 1024). Item size = sum of attribute-name + value bytes.
- RRU rounding: ceil(item_size / 4096); eventually consistent = strong / 2.
- Non-existent GetItem still consumes 0.5 RRU (eventual) or 1 RRU (strong).
- Query/Scan: aggregate bytes read, rounded up to next 4 KB, then halved if eventual.
- Filters and projections DO NOT reduce capacity.
- BatchGetItem / BatchWriteItem: rounded per-item, then summed.
- PutItem / UpdateItem: sized on LARGER of before and after.
- Transactional writes: 2× multiplier applies to BASE TABLE ONLY.
GSI and LSI writes remain 1×. (Verified.)
- Transactional reads: 2× multiplier.
- Conditional write failure still consumes capacity based on the existing
item size (or new item size for PutItem on a non-existent key).
- GSI write amplification:
* Only fires if the item has the GSI's partition-key attribute.
* Only fires if a projected attribute changed (UpdateItem).
* GSI write is sized by the PROJECTED attributes, not the base item.
- Storage: (raw_bytes + 100 bytes per item) / 1_000_000_000 (decimal GB),
billed at the full public Standard rate. Global tables add +48 bytes/item.
The 25 GB free tier is intentionally NOT applied (account-wide, often
already consumed) — see _storage_cost.
Usage:
python3 calculate_costs.py --model artifacts/{app}/dynamodb_data_model.json
python3 calculate_costs.py --model artifacts/{app}/dynamodb_data_model.json \
--requirements artifacts/{app}/{app}-requirements.json
python3 calculate_costs.py --model artifacts/{app}/dynamodb_data_model.json \
--output cost_report.md
"""
from __future__ import annotations
import argparse
import json
import math
import sys
from pathlib import Path
# -----------------------------------------------------------------------------
# Pricing constants — DynamoDB Standard table class, on-demand public rates.
# Rates vary by region; confirm against the AWS DynamoDB pricing page.
# -----------------------------------------------------------------------------
SECONDS_PER_MONTH = 2_592_000
SECONDS_PER_DAY = 86_400
DAYS_PER_MONTH = 30
# Per-request prices (dollars per unit, converted from "$X per million").
WRU_PRICE = 0.625 / 1_000_000 # Standard on-demand write
RRU_PRICE = 0.125 / 1_000_000 # Strongly-consistent on-demand read
# (Eventually-consistent and transactional rates derive from these via multiplier.)
# Storage.
STORAGE_PRICE_PER_GB_MONTH = 0.25 # Standard class, full public rate
# NOTE: the 25 GB Standard storage free tier is intentionally NOT modeled — it is
# account-wide and often already consumed, so we price all storage at full rate.
BYTES_PER_GB = 1_000_000_000 # Decimal GB per AWS billing convention
# Capacity-unit sizing (bytes).
RCU_SIZE_BYTES = 4 * 1024 # 4 KB
WCU_SIZE_BYTES = 1 * 1024 # 1 KB
# Per-item storage overhead (AWS docs: "100 bytes per item for indexing").
PER_ITEM_STORAGE_OVERHEAD = 100
GLOBAL_TABLES_OVERHEAD = 48 # Additional bytes when global tables enabled
# Query/Scan pagination cap. DynamoDB returns at most 1 MB per page; we use a
# conservative 900 KB to leave headroom for metadata.
PAGE_CAP_KB = 900
# -----------------------------------------------------------------------------
# Vector index pricing.
#
# Vector index capacity is metered in BYTES, not request units, across three
# dimensions. Rates below are us-east-1 Standard, taken from the AWS Pricing API
# (usage types VectorWriteRequest / VectorSearch) rather than from a worked example.
# Standard-IA is 125% on both request dimensions (IA-VectorWriteRequest $0.65,
# IA-VectorSearch $0.0025) and 40% on storage.
#
# There is NO separate vector-storage usage type: index storage rolls into ordinary
# table storage at STORAGE_PRICE_PER_GB_MONTH.
#
# All three dimensions are modelled, but on different footings, and the report says
# which is which:
# storage, writes — derived from the design (dimensions, projection, item counts)
# searches — CALIBRATED against live measurements, because the metered bytes
# depend on index traversal rather than anything in the design
# Live validation supersedes the search figure with the observed
# VectorSearchRequestBytes. See references/vector-search.md.
# -----------------------------------------------------------------------------
VECTOR_WRITE_PRICE_PER_GB = 0.52
VECTOR_SEARCH_PRICE_PER_GB = 0.002
IA_REQUEST_MULTIPLIER = 1.25 # applies to both vector request dimensions
BYTES_PER_F32 = 4 # vectors are stored at 32-bit float precision
# 1 KB minimum. Applied ONCE PER REQUEST, per the documentation.
#
# We follow the docs here, and a measurement caveat is worth recording because it is easy
# to misread. `ConsumedCapacity` floors each index's REPORTED value at 1 KB independently
# of the request total: in a probe where one PutItem touched four indexes and the request
# total was ~26 KB (far above the floor), a 128-dimension index whose real content is 512 B
# still reported 1024. So the per-index reported numbers are a display rounding and cannot
# be summed to infer the billed total, and they cannot distinguish per-request from
# per-index billing. Settling that would need Cost Explorer / CUR data, not
# ConsumedCapacity.
#
# It barely matters in practice: the floor only binds below 256 dimensions (256 x 4 =
# 1024 B), and real embedding models are 256-3072.
VECTOR_METERING_MIN_BYTES = 1024
# ---- Search metering: measured, and deliberately NOT priced -------------------
# Measured us-west-2 2026-08-19 (see _research/verified-vector-facts.md). What is solid:
# * Metered bytes are EXACTLY linear in TopK within a configuration (0.0% midpoint error).
# * The PROJECTION is a ~170x multiplier on the per-result term: ~89 B for KEYS_ONLY,
# ~98 B for a narrow INCLUDE, and the whole projected item for ALL. A TopK=100 search
# on an ALL index measured 1.52 MB.
# * At a fixed index configuration, cost is linear in dimensions (~31 B per dimension
# measured at 256 / 1536 / 3072).
#
# What is NOT modellable, and why we refuse to emit a dollar figure: the fraction of the
# index a search examines varies by an ORDER OF MAGNITUDE with index configuration —
# 12.8% of all vector bytes on a 60-item unpartitioned index, but 1.3% on a 200-item
# partitioned one. An earlier attempt to calibrate a traversal term linearly across two
# probes was 51-69% low when tested at 256 / 1536 / 3072 dimensions, i.e. wrong at exactly
# the dimensionalities real embedding models use (Titan Text Embeddings V2, OpenAI
# text-embedding-3-large). A confidently wrong cost figure is worse than none, so the
# report gives the drivers and tells the user to measure. Live validation supplies the
# observed VectorSearchRequestBytes.
# Key/search-schema overhead a KEYS_ONLY vector index carries on top of the vector.
# Measured: a minimal single-vector item gave 4,105 B on a 1024-dim KEYS_ONLY index
# (4,096 + 9); with a SearchSchema HASH + INLINE_FILTER it gave 4,130 (+34). Use the
# larger, which is the conservative direction.
VECTOR_KEY_OVERHEAD_BYTES = 35
# A vector attribute copied into an index by ALL projection is carried in its base-table
# representation, not as f32. Measured: an unindexed 1024-dim vector added 5,633 B to an
# ALL index, i.e. ~5.5 B per number rather than 4.
BYTES_PER_BASE_TABLE_NUMBER = 5.5
VECTOR_RETURN_BYTES_KEYS_ONLY = 89
VECTOR_RETURN_BYTES_INCLUDE_BASE = 89
# Traversal slope used ONLY by the pre-spend gate in benchmark_model.py, never for a
# reported cost. Two measured points: 128 dims -> 2,647 B (20.7 B/dim) and 1,024 dims ->
# 10,588 B (10.3 B/dim) — traversal grows SUBLINEARLY in dimensions. Taking the steeper
# low-dimension slope therefore overshoots at higher dimensions, which is the correct
# direction for a guard whose job is to refuse a bill. See
# vector_search_bytes_spend_gate_upper().
VECTOR_TRAVERSAL_BYTES_PER_DIM_UPPER = 20.7
# Safety factor on the spend-gate bound. Not decoration — unfactored, the bound had
# essentially NO margin on two of the sixteen measured search points: 128-dim KEYS_ONLY at
# TopK=1 came out 2,739 B against 2,736 B measured (1.001x), and 1024-dim ALL at TopK=100
# came out 1,739,097 B against 1,524,351 B (1.14x) — i.e. thinnest exactly where the bill
# is largest. The exposure is structural: for an ALL projection the per-result term IS the
# declared item size, so the bound inherits the user's input error, and under-declaring
# item size is a common mistake. 1.5x absorbs a ~33% input shortfall.
# Applies ONLY to the gate, never to a reported cost.
VECTOR_SPEND_GATE_SAFETY = 1.5
# Service limits worth failing loudly on rather than silently mispricing.
MAX_VECTOR_INDEXES_PER_TABLE = 5
MAX_VECTOR_DIMENSIONS = 4096
MAX_VECTOR_TOP_K = 100
# -----------------------------------------------------------------------------
# Operation sets.
# -----------------------------------------------------------------------------
READ_OPS = {"GetItem", "Query", "Scan", "BatchGetItem", "TransactGetItems"}
WRITE_OPS = {"PutItem", "UpdateItem", "DeleteItem", "BatchWriteItem", "TransactWriteItems"}
TRANSACTIONAL_OPS = {"TransactGetItems", "TransactWriteItems"}
MULTI_ITEM_READ = {"Query", "Scan", "BatchGetItem", "TransactGetItems"}
# SearchVectors is deliberately NOT in READ_OPS. It consumes no RCU — it is billed on
# bytes examined — so routing it through the RCU path would silently misprice it.
VECTOR_SEARCH_OP = "SearchVectors"
# Default write_action when requirements aren't provided.
DEFAULT_WRITE_ACTION = {
"PutItem": "create",
"UpdateItem": "update",
"DeleteItem": "delete",
"BatchWriteItem": "mixed",
"TransactWriteItems": "mixed",
}
# Fraction of writes that create new items, by write_action.
CREATE_RATIO = {
"create": 1.0,
"update": 0.0,
"delete": 0.0,
"mixed": 0.3,
}
DEFAULT_RETENTION_DAYS = 30
# Fraction of effective item bytes projected, by GSI projection type.
# Used to size the GSI write that fires as amplification.
PROJECTION_WRITE_RATIO = {
"ALL": 1.0,
"INCLUDE": 0.3, # conservative default when include list is unknown
"KEYS_ONLY": 0.1, # keys + optional sort key, typically ≤ 1 KB
}
# Storage ratio relative to base table, by GSI projection type.
PROJECTION_STORAGE_RATIO = {
"ALL": 1.0,
"INCLUDE": 0.3,
"KEYS_ONLY": 0.1,
}
DISCLAIMER = (
"> **Disclaimer:** This estimate covers **read/write request costs** and "
"**storage costs** only, at DynamoDB Standard table class on-demand **full "
"public rates** (the 25 GB storage free tier is NOT applied — it is "
"account-wide and often already used). **Rates vary by region** — confirm "
"the figure against the AWS DynamoDB pricing page (or `aws pricing "
"get-products --service-code AmazonDynamoDB`) for the region you will "
"deploy in before quoting it. The headline figure "
"is **peak-sustained** — it assumes every pattern runs at its declared "
"`peak_rps` continuously, 24/7 for a month (the worst case). Real on-demand "
"spend tracks actual request volume; set `avg_rps` on patterns to also get "
"an expected average-volume figure. For up-to-date pricing, refer to "
"the [Amazon DynamoDB Pricing](https://aws.amazon.com/dynamodb/pricing/) page."
)
GSI_FOOTNOTE = (
"¹ **GSI additional writes** — When a table write changes attributes "
"projected into a GSI, DynamoDB performs an additional write to that index. "
"The additional write is sized by the projected attributes (not the base "
"item), which is why KEYS_ONLY / INCLUDE projections are cheaper. "
"[Learn more](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html"
"#GSI.ThroughputConsiderations.Writes)"
)
# What a SearchVectors row shows in the Monthly Cost column. Never a dollar amount:
# search consumes no RCU/WCU and its byte cost is deliberately unpriced, so any figure
# there would be 0.00 — and "$0.00" in the table a reader scans to find expensive
# patterns asserts that vector search is free. It is not; it is unmeasured.
VECTOR_SEARCH_CELL = "not priced²"
VECTOR_SEARCH_FOOTNOTE = (
"² **Vector search is not priced here, and is NOT $0** — `SearchVectors` consumes no "
"RCU/WCU; it bills on vector bytes examined plus bytes returned, at "
f"${VECTOR_SEARCH_PRICE_PER_GB:.3f}/GB. The examined fraction is not derivable from a "
"design (measured to vary by an order of magnitude across index configurations), so no "
"figure is emitted rather than a wrong one. **This means the headline total above "
"excludes vector search cost.** See *Vector Index Capacity* below for the measured "
"drivers and how to obtain the real number."
)
# =============================================================================
# Core capacity-consumption formulas (verified against live DynamoDB).
# =============================================================================
def wru_for_item(item_size_bytes: int) -> int:
"""WRUs consumed for a standard write of a given item size.
Empirically verified: ceil(size / 1024), with a minimum of 1 WRU.
"""
if item_size_bytes <= 0:
return 1
return math.ceil(item_size_bytes / WCU_SIZE_BYTES)
def rru_for_item(item_size_bytes: int, strong: bool = False) -> float:
"""RRUs consumed reading a single item.
Strong: ceil(size / 4096), min 1. Eventual: half of strong, min 0.5.
Non-existent items still consume the minimum (verified).
"""
strong_rru = max(1, math.ceil(max(1, item_size_bytes) / RCU_SIZE_BYTES))
return float(strong_rru) if strong else strong_rru / 2.0
def rru_for_query(total_bytes_read: int, strong: bool = False) -> float:
"""RRUs for a Query/Scan that reads `total_bytes_read` from storage.
Query/Scan aggregates all items, rounds the TOTAL to next 4 KB, then
halves for eventual consistency. (Verified: 5×1KB items = 1 RCU eventual,
10×512B items = 1 RCU eventual, etc.)
"""
strong_rru = max(1, math.ceil(max(1, total_bytes_read) / RCU_SIZE_BYTES))
return float(strong_rru) if strong else strong_rru / 2.0
def rru_for_batch_read(item_sizes: list[int], strong: bool = False) -> float:
"""BatchGetItem rounds each item individually, then sums. Verified."""
total = 0.0
for sz in item_sizes:
total += rru_for_item(sz, strong=strong)
return total
def wru_for_batch_write(item_sizes: list[int]) -> int:
"""BatchWriteItem rounds each item individually, then sums. Verified."""
return sum(wru_for_item(sz) for sz in item_sizes)
# =============================================================================
# GSI amplification (verified empirically).
# =============================================================================
def gsi_write_wru(
base_item_size_bytes: int, projection_type: str, projection_ratio_override: float | None = None
) -> int:
"""WRU consumed for the GSI write that amplifies from a base-table write.
The GSI item size is driven by PROJECTED attributes only. Empirically:
- ALL projection on a 4 KB base: GSI consumed = 4 WCU (full item)
- KEYS_ONLY / INCLUDE on a 4 KB base: GSI consumed = 1 WCU (tiny projected item)
"""
ratio = (
projection_ratio_override
if projection_ratio_override is not None
else PROJECTION_WRITE_RATIO.get(projection_type, 1.0)
)
projected_size = max(1, int(base_item_size_bytes * ratio))
return wru_for_item(projected_size)
# =============================================================================
# Model loading and helpers.
# =============================================================================
def load_model(path: str) -> dict:
with open(path) as f:
return json.load(f)
def _build_entity_attr_sizes(tables: list[dict]) -> dict:
"""Build {entity_name: {attr_name: size_bytes}} from entity definitions."""
# Heuristic sizes used by modeling instructions.
type_sizes = {
"S": 100,
"N": 8,
"BOOL": 1,
"B": 256,
"L": 200,
"M": 200,
"SS": 200,
"NS": 200,
"BS": 200,
"NULL": 1,
}
# Attribute-name overhead. Average name ~10 bytes.
name_overhead = 10
result = {}
for t in tables:
for ent in t.get("entities", []):
attr_sizes = {}
for attr in ent.get("attributes", []):
attr_sizes[attr["name"]] = (
type_sizes.get(attr.get("type", "S"), 100) + name_overhead
)
result[ent["entity_name"]] = attr_sizes
return result
def _cap_items_per_request(ap: dict, entity_sizes: dict) -> int:
"""Cap items_per_request at the 1 MB page limit using projected attribute sizes."""
items = ap.get("items_per_request", 1)
if ap["operation"] not in ("Query", "Scan"):
return items
projection = ap.get("projection")
if not projection:
return items
projected_bytes = 0
for attr in projection:
found = False
for ent_key, attr_sizes in entity_sizes.items():
if attr in attr_sizes:
projected_bytes += attr_sizes[attr]
found = True
break
if not found:
projected_bytes += 100 # default fallback per attribute
if projected_bytes <= 0:
return items
page_cap_bytes = PAGE_CAP_KB * 1024
items_per_page = max(1, page_cap_bytes // projected_bytes)
return min(items, items_per_page)
def _resolve_write_action(ap: dict, requirements: dict | None) -> str:
wa = ap.get("write_action")
if wa:
return wa
if requirements:
source_req = ap.get("source_requirement", "")
updates = requirements.get("updates", {})
if source_req in updates:
wa = updates[source_req].get("write_action")
if wa:
return wa
return DEFAULT_WRITE_ACTION.get(ap["operation"], "mixed")
def _resolve_retention_days(ap: dict, tables: list[dict], requirements: dict | None) -> int:
# A per-pattern retention_days (documented in cost-model-schema.md) is the
# most specific signal and wins over the requirements lookup and the default.
# Without this, a write pattern declaring e.g. retention_days: 365 was
# silently priced at the 30-day default, understating storage ~12×.
if "retention_days" in ap:
return int(ap["retention_days"])
if not requirements:
return DEFAULT_RETENTION_DAYS
entity_name = None
table_name = ap.get("table", "")
for t in tables:
if t["table_name"] == table_name:
entities = t.get("entities", [])
if entities:
entity_name = entities[0].get("entity_name")
break
if entity_name:
ent_def = requirements.get("entities", {}).get(entity_name, {})
ret = ent_def.get("retention_days")
if ret is not None:
return ret
ret = requirements.get("metadata", {}).get("retention_days_default")
if ret is not None:
return ret
return DEFAULT_RETENTION_DAYS
def _fmt(cost: float) -> str:
return f"${cost:.2f}"
def _padded_table(headers: list[str], rows: list[list[str]]) -> str:
widths = [len(h) for h in headers]
for row in rows:
for i, cell in enumerate(row):
if i < len(widths):
widths[i] = max(widths[i], len(cell))
lines = [
"| " + " | ".join(h.ljust(widths[i]) for i, h in enumerate(headers)) + " |",
"| " + " | ".join("-" * w for w in widths) + " |",
]
for row in rows:
lines.append(
"| "
+ " | ".join(
row[i].ljust(widths[i]) if i < len(row) else "".ljust(widths[i])
for i in range(len(headers))
)
+ " |"
)
return "\n".join(lines)
# =============================================================================
# Per-access-pattern capacity and cost.
# =============================================================================
def calc_pattern_capacity(ap: dict) -> dict:
"""Compute {rcus, wcus, notes} for one access pattern.
Returns a dict with:
op, rcus, wcus, strong, transactional, notes
Only read OR write is non-zero for a given pattern.
"""
op = ap["operation"]
size = ap.get("estimated_item_size_bytes", 1024)
items = ap.get("items_per_request", 1)
strong = ap.get("consistency", "eventual") == "strong"
transactional = op in TRANSACTIONAL_OPS
notes = []
rcus = 0.0
wcus = 0.0
if op == "GetItem":
rcus = rru_for_item(size, strong=strong)
if ap.get("non_existent_rate"):
notes.append("Non-existent items still consume the minimum RRU.")
elif op == "Query":
total_bytes = size * items
rcus = rru_for_query(total_bytes, strong=strong)
if ap.get("filter"):
notes.append("FilterExpression does not reduce cost.")
if ap.get("projection"):
notes.append("ProjectionExpression does not reduce cost.")
elif op == "Scan":
total_bytes = size * items
rcus = rru_for_query(total_bytes, strong=strong)
notes.append("Scan reads every evaluated item regardless of filter.")
elif op == "BatchGetItem":
item_sizes = ap.get("item_sizes") or [size] * items
rcus = rru_for_batch_read(item_sizes, strong=strong)
elif op == "TransactGetItems":
item_sizes = ap.get("item_sizes") or [size] * items
rcus = rru_for_batch_read(item_sizes, strong=True) * 2.0
notes.append("Transactional reads consume 2× RRUs.")
elif op == "PutItem":
# Replacement costs the LARGER of before/after.
before = ap.get("previous_item_size_bytes", 0)
wcus = wru_for_item(max(size, before))
elif op == "UpdateItem":
before = ap.get("previous_item_size_bytes", size)
wcus = wru_for_item(max(size, before))
elif op == "DeleteItem":
wcus = wru_for_item(size)
elif op == "BatchWriteItem":
item_sizes = ap.get("item_sizes") or [size] * items
wcus = float(wru_for_batch_write(item_sizes))
elif op == "TransactWriteItems":
item_sizes = ap.get("item_sizes") or [size] * items
# Txn multiplier applies to base table only. GSI amplification is
# billed elsewhere (standard 1× per GSI write).
wcus = float(wru_for_batch_write(item_sizes)) * 2.0
notes.append(
"Transactional writes: 2× multiplier on base table; "
"GSI amplification remains 1× per GSI write."
)
# Conditional-failure handling
cond_fail_rate = ap.get("conditional_fail_rate", 0.0)
if cond_fail_rate and op in WRITE_OPS:
# Failed writes still consume capacity sized by existing/new item.
# We assume the rate is a fraction of total calls; those calls still
# compute WRUs the same way.
notes.append(
f"Conditional failures (rate={cond_fail_rate}) still " f"consume the same WRUs."
)
return {
"op": op,
"rcus": rcus,
"wcus": wcus,
"strong": strong,
"transactional": transactional,
"notes": notes,
}
def _entity_attr_names(table_def: dict) -> dict:
"""{entity_name: set(attribute names)} for one table.
A DynamoDB item only appears in (and only amplifies a write to) a GSI whose
key attributes it actually CARRIES. In a single-table design with
heterogeneous entities, each GSI is keyed on an attribute only some entities
have — so this per-entity attribute-name set is what decides GSI membership.
"""
out = {}
for ent in table_def.get("entities", []) or []:
name = ent.get("entity_name")
if not name:
continue
out[name] = {a["name"] for a in ent.get("attributes", []) or [] if a.get("name")}
return out
def _gsi_key_attrs(gsi: dict) -> tuple:
"""(pk_attr, sk_attr|None) for a GSI, tolerating key-name spellings."""
pk = gsi.get("partition_key") or gsi.get("hash_key")
sk = gsi.get("sort_key") or gsi.get("range_key")
return pk, sk
def _table_key_attrs(table_def: dict) -> set:
"""The table's own PK/SK attribute names — every item carries these."""
ks = table_def.get("key_schema") or {}
return {a for a in (ks.get("partition_key"), ks.get("sort_key")) if a}
def _item_carries_gsi_key(attr_names: set, gsi: dict, table_key_attrs: set | None = None) -> bool:
"""True iff an item with these attributes is INDEXED by this GSI.
Membership rule, verified live (ReturnConsumedCapacity=INDEXES): an item is
written to a GSI only if it carries the GSI partition key AND, for a
composite GSI, the GSI sort key. An item carrying the PK but missing the SK
of a composite GSI is NOT indexed (live: a g2-without-g2sk put fired ZERO
GSI writes). A GSI key that is also a TABLE key is carried by every item.
"""
tk = table_key_attrs or set()
pk, sk = _gsi_key_attrs(gsi)
if not pk or (pk not in attr_names and pk not in tk):
return False
if sk and sk not in attr_names and sk not in tk:
return False
return True
def _gsi_membership_determinable(table_def: dict, gsi: dict) -> bool:
"""Can we reason about which items belong to this GSI from the model?
Determinable only when the GSI's partition key is positively declared
somewhere we can attribute to items: by at least one entity's attribute
list, or as a table key (every item carries table keys). A GSI key declared
nowhere — or only at table level (`attribute_definitions`, which can't say
WHICH entity carries it) — is INDETERMINATE: we must not infer absence and
silently drop the GSI's cost. Callers fall back to the conservative
charge-everything bound for such a GSI.
"""
pk, sk = _gsi_key_attrs(gsi)
if not pk:
return False
table_keys = _table_key_attrs(table_def)
declared = set(table_keys)
for ent in table_def.get("entities") or []:
declared |= {a["name"] for a in (ent.get("attributes") or []) if a.get("name")}
# Both the GSI PK and (for a composite GSI) the GSI SK must be declared
# somewhere we can attribute to items, or we cannot reason about which items
# are members and must fall back to the conservative full-size charge.
if pk not in declared:
return False
if sk and sk not in declared:
return False
return True
def _projection_type(gsi: dict) -> str:
proj = gsi.get("projection", {}) or {}
ptype = proj.get("type", "ALL")
return ptype.upper() if isinstance(ptype, str) else "ALL"
def _resolve_written_items(ap: dict, table_def: dict) -> tuple[list, bool]:
"""Resolve the per-item entity attribute-name sets a single call writes.
Returns (item_attr_sets, resolved). Each element of item_attr_sets is the
attribute-name set of ONE written item, so a transaction creating 1 order +
3 line items yields 4 sets. `resolved=False` means we could not identify the
written entities and the caller must fall back to a conservative upper bound.
Resolution order:
1. explicit `entities_written` on the access pattern — a list of entity
names (repeats allowed) or `[{"entity": "X", "count": n}]`.
2. exactly one entity declared on the table — the unambiguous single-entity
/ classic-multi-table case; each of the items_per_request written items
is that entity.
3. otherwise unresolved (multi-entity single-table design with no
entities_written): conservative upper bound.
"""
ent_attrs = _entity_attr_names(table_def)
items_per = int(ap.get("items_per_request", 1) or 1)
ew = ap.get("entities_written")
if ew:
flat = []
for e in ew:
if isinstance(e, dict):
name = e.get("entity") or e.get("entity_name")
cnt = int(e.get("count", 1) or 1)
else:
name, cnt = e, 1
# A named entity we can't find contributes items that carry no known
# key — i.e. amplify nowhere. That's the honest reading of a typo'd
# or undeclared entity name; it surfaces as a visibly low number
# rather than silently inheriting the full fan-out.
attrs = ent_attrs.get(name, set())
flat.extend([attrs] * max(1, cnt))
return flat, True
if len(ent_attrs) == 1:
only = next(iter(ent_attrs.values()))
return [only] * items_per, True
return [], False
def _gsi_index_writes_for_item(
item_attrs: set, gsi: dict, op: str, attrs_written, table_key_attrs: set | None = None
) -> int:
"""How many index writes ONE item's write costs against this GSI: 0, 1, or 2.
Membership (carries the GSI key) is necessary for every op. Verified live
against ReturnConsumedCapacity=INDEXES:
- Put / create / Delete, or an UpdateItem with no attrs_written info:
1 index write per member item (the projected-size write).
- UpdateItem that CHANGES the GSI's own key attribute (PK or SK): **2**
index writes — DynamoDB deletes the old index entry and inserts a new
one (a "ByStatus" KEYS_ONLY index cost 2 WCU when `status` changed).
- UpdateItem that changes a NON-key but projected attribute:
ALL → 1 (the whole item is re-projected);
INCLUDE → 1 if a projected attr changed, else 0;
KEYS_ONLY → 0 (nothing projected but the keys, which didn't move).
- UpdateItem touching nothing the GSI projects and no key it indexes: 0.
Each returned write is later sized by gsi_write_wru (ALL=full item,
INCLUDE/KEYS_ONLY=small).
"""
if not _item_carries_gsi_key(item_attrs, gsi, table_key_attrs):
return 0
if op == "UpdateItem" and attrs_written is not None:
ptype = _projection_type(gsi)
pk, sk = _gsi_key_attrs(gsi)
keys = {k for k in (pk, sk) if k}
key_moved = bool(set(attrs_written) & keys)
if key_moved:
# Old entry deleted + new entry inserted on this index.
return 2
if ptype == "ALL":
return 1
if ptype == "INCLUDE":
proj = gsi.get("projection", {}) or {}
inc = set(
proj.get("attributes")
or proj.get("non_key_attributes")
or proj.get("NonKeyAttributes")
or []
)
return 1 if (set(attrs_written) & inc) else 0
if ptype == "KEYS_ONLY":
return 0 # keys didn't move (handled above) and nothing else projected
return 1
def calc_gsi_amplification_wru(ap: dict, table_def: dict) -> tuple[float, list[str]]:
"""Compute per-request WRU consumed by GSI amplification for this write.
Returns (total_wru_amp, details_strings).
A write amplifies to a GSI only when the written item CARRIES that GSI's key
attributes (PK, plus SK if the GSI is composite) — verified live against
ReturnConsumedCapacity=INDEXES. In a single-table design with heterogeneous
entities, each GSI is keyed on an attribute only some entities have, so a
given write touches only the few GSIs its entity is a member of — not all of
them. The per-GSI write is sized by the PROJECTED attributes (ALL = full
item; INCLUDE / KEYS_ONLY = small), which the live run also confirmed.
"""
if ap["operation"] not in WRITE_OPS:
return 0.0, []
gsis = table_def.get("gsis", []) or []
if not gsis:
return 0.0, []
op = ap["operation"]
size = ap.get("estimated_item_size_bytes", 1024)
attrs_written = ap.get("attributes_written") # optional: list of attr names
items_per = int(ap.get("items_per_request", 1) or 1)
written_items, resolved = _resolve_written_items(ap, table_def)
table_keys = _table_key_attrs(table_def)
total = 0.0
details: list[str] = []
if resolved:
# Accurate path: each written item amplifies only to the GSIs whose key
# it carries. A GSI whose key is declared NOWHERE attributable to an item
# (not on any entity, not a table key) is INDETERMINATE — we can't infer
# absence, so charge it conservatively (fires for every item, honoring
# the UpdateItem projection gate) rather than silently dropping its cost.
per_gsi_writes: dict = {} # index_name -> total index writes (1 or 2 each)
indeterminate: list[str] = []
member_but_gated = [] # member items whose UpdateItem projection gate excluded them
for gsi in gsis:
name = gsi.get("index_name", "?")
determinable = _gsi_membership_determinable(table_def, gsi)
gsi_gated_member = False
for item_attrs in written_items:
if determinable:
is_member = _item_carries_gsi_key(item_attrs, gsi, table_keys)
writes = _gsi_index_writes_for_item(
item_attrs, gsi, op, attrs_written, table_keys
)
if is_member and writes == 0:
gsi_gated_member = True
else:
# Indeterminate membership → conservative: count it unless the
# UpdateItem projection gate provably excludes it.
pk_a, sk_a = _gsi_key_attrs(gsi)
synthetic = {pk_a} | ({sk_a} if sk_a else set())
writes = _gsi_index_writes_for_item(
synthetic, gsi, op, attrs_written, table_keys
)
if writes:
total += gsi_write_wru(size, _projection_type(gsi)) * writes
per_gsi_writes[name] = per_gsi_writes.get(name, 0) + writes
if not determinable and name in per_gsi_writes:
indeterminate.append(name)
if gsi_gated_member and name not in per_gsi_writes:
member_but_gated.append(name)
if per_gsi_writes:
details = [
f"{name} (×{n} index write{'s' if n != 1 else ''}"
+ (
", membership unverified — GSI key not declared on any entity"
if name in indeterminate
else ""
)
+ ")"
for name, n in per_gsi_writes.items()
]
elif member_but_gated:
details = [
f"no GSI amplification — {op} touched no attribute "
f"projected into {', '.join(member_but_gated)} "
f"(projection gate)"
]
else:
details = ["no GSI amplification — written item(s) carry no GSI key"]
return total, details
# Unresolved: multi-entity single-table design with no `entities_written`.
# We cannot know which entity each written item is, so we keep the
# conservative UPPER BOUND (every item is assumed a member of every GSI) and
# say so loudly — declaring `entities_written` on the pattern refines it. A
# synthetic full-key item drives the same write-count logic (incl. the
# UpdateItem projection gate and the key-move 2× factor) as the accurate path.
for gsi in gsis:
pk_a, sk_a = _gsi_key_attrs(gsi)
synthetic = {a for a in (pk_a, sk_a) if a}
writes = _gsi_index_writes_for_item(synthetic, gsi, op, attrs_written, table_keys)
total += gsi_write_wru(size, _projection_type(gsi)) * writes
if op in ("BatchWriteItem", "TransactWriteItems"):
total *= items_per
details.append(
f"UPPER BOUND — '{ap.get('table', '?')}' has multiple entities and this "
f"pattern has no `entities_written`, so every written item is charged "
f"against every GSI. Declare `entities_written` to bill only the GSIs "
f"each written entity is a member of."
)
return total, details
# =============================================================================
# Vector index costs.
#
# Modelled honestly: storage and writes are derivable from the design, searches are
# not. Every function here labels which side of that line it is on, and the report
# carries the caveat so a user never mistakes the search figure for a measurement.
# =============================================================================
def _vector_indexes(table_def: dict | None) -> list:
return (table_def or {}).get("vector_indexes", []) or []
def _ia_multiplier(table_def: dict | None) -> float:
"""Standard-IA charges 125% on both vector request dimensions.
Matching on "IA" alone was wrong, and wrong in the silent direction. DynamoDB's actual
table-class enum is `STANDARD_INFREQUENT_ACCESS`, which does NOT contain the substring
"IA" -- so a model written with the real AWS value was priced at 1.0x and under-reported
Standard-IA vector requests by 25% with every other number looking correct. "INFREQUENT"
is the reliable token; "IA" stays for the shorthand spellings an author might reasonably
write by hand.
"""
tclass = ((table_def or {}).get("table_class") or "STANDARD").upper()
return IA_REQUEST_MULTIPLIER if "INFREQUENT" in tclass or "IA" in tclass else 1.0
def _vector_projection(vi: dict) -> dict:
"""The index's projection block, accepting the bare-string shorthand.
`"projection": "KEYS_ONLY"` is the natural thing to write — it is how the API-level value
reads — and treating it as a dict raised a bare `AttributeError: 'str' object has no
attribute 'get'` that said nothing about the model. Normalised here rather than at each
call site, and it mirrors the `attribute_definitions` `{"name","type"}` shorthand these
scripts already accept.
"""
proj = vi.get("projection", {}) or {}
return {"type": proj} if isinstance(proj, str) else proj
def _vector_item_bytes(vi: dict, table_def: dict, entity_attr_sizes: dict | None) -> int:
"""Bytes one item contributes to one vector index. Measured semantics, not a ratio.
A controlled probe (two indexes on the SAME attribute and dimensions, differing only in
projection) established:
KEYS_ONLY flat. Insensitive to item content: 4,105 / 4,104 / 4,104 B on a 1024-dim
index across a minimal item, a +10 KB item, and an item with a second
vector. So it is dimensions x 4 plus a small key overhead — NOT a fraction
of item size, which is what an earlier ratio-based model wrongly assumed.
ALL dimensions x 4 plus essentially the whole rest of the item. Adding a
10,240 B payload added 10,246 B. It also copies UNINDEXED vector
attributes, in their base-table representation (an unindexed 1024-dim
vector added 5,633 B, ~5.5 B/number rather than 4).
INCLUDE dimensions x 4 plus the named attributes.
Consequence worth stating plainly: on a minimal item ALL costs the SAME as KEYS_ONLY.
The cost of ALL is entirely the cost of the rest of the item.
CONVENTION: `estimated_item_size_bytes` EXCLUDES the vector attributes. The ALL branch
adds the declared item size to this index's own `dimensions x 4`, so a declared size
that already counted the embedding bills it twice and overstates an ALL index by
roughly 2x. This is the direction the probe above was measured in, and it is the more
accurate one -- `dimensions` is exact where an item-size estimate is not. Documented in
references/cost-model-schema.md against both the field and the vector section; the
alternative (subtracting a vector footprint from the declared size) would silently
under-count every model that already follows the documented convention.
"""
dims = int(vi.get("dimensions", 0) or 0)
vector_bytes = dims * BYTES_PER_F32
proj = _vector_projection(vi)
ptype = (proj.get("type") or "ALL").upper()
if ptype == "KEYS_ONLY":
return int(vector_bytes + VECTOR_KEY_OVERHEAD_BYTES)
if ptype == "INCLUDE":
named = proj.get("attributes", []) or []
extra, _ = _named_attr_bytes(table_def, named)
return int(vector_bytes + VECTOR_KEY_OVERHEAD_BYTES + extra)
# ALL: the whole rest of the item rides along on every vector write.
entities = table_def.get("entities", []) or []
if entities:
non_vector = sum(e.get("estimated_item_size_bytes", 1024) for e in entities) / len(entities)
else:
non_vector = 1024.0
# Other vector attributes declared on this table also get copied, in base-table form.
own_attr = vi.get("vector_attribute")
others = 0.0
for other in table_def.get("vector_indexes", []) or []:
if other.get("vector_attribute") and other.get("vector_attribute") != own_attr:
others += int(other.get("dimensions", 0) or 0) * BYTES_PER_BASE_TABLE_NUMBER
return int(vector_bytes + VECTOR_KEY_OVERHEAD_BYTES + non_vector + others)
def _vector_indexed_item_count(vi: dict, table_def: dict) -> tuple[int, bool]:
"""(item count in this vector index, was it declared explicitly?).
Only items carrying the vector attribute — and the SearchSchema partition key, if
one is defined — are replicated into the index. The model can declare that directly
with `estimated_indexed_items`; otherwise we fall back to the table's entity counts,
which is a conservative upper bound, and say so in the report.
"""
declared = vi.get("estimated_indexed_items")
if declared is not None:
return int(declared), True
total = sum(
e.get("estimated_item_count", 100_000) for e in (table_def.get("entities", []) or [])
)
return int(total or 100_000), False
def vector_write_bytes_per_call(ap: dict, table_def: dict) -> tuple[float, list[str]]:
"""Bytes replicated into a table's vector indexes by one write call.
Mirrors the GSI amplification gate: a write only pays for an index whose vector
attribute it actually touched. When `attributes_written` is absent we cannot tell,
so we stay at the conservative upper bound (assume it did) and flag it — the same
convention the GSI path uses.
The 1 KB minimum is applied ONCE across all of the table's vector indexes, because
the floor is per request, not per index.
"""
vis = _vector_indexes(table_def)
if not vis:
return 0.0, []
written = ap.get("attributes_written")
details: list[str] = []
raw_bytes = 0.0
for vi in vis:
attr = vi.get("vector_attribute") or ""
touched = True
if written is not None:
touched = attr in set(written)
if not touched:
details.append(f"{vi.get('index_name')}: not touched by this write")
continue
b = _vector_item_bytes(vi, table_def, None)
raw_bytes += b
note = "" if written is not None else " (upper bound: attributes_written absent)"
details.append(f"{vi.get('index_name')}: {b:,} B{note}")
if raw_bytes <= 0:
return 0.0, details
return max(float(raw_bytes), float(VECTOR_METERING_MIN_BYTES)), details
def _named_attr_bytes(table_def: dict, names: list) -> tuple[float, bool]:
"""(summed size of the named attributes, were they all actually declared?).
Uses the sizes declared in entities[].attributes[] where available. Falls back to the
generic S=100 heuristic for anything undeclared, which is coarse — measurement showed
a 6-character Title contributes ~9 B per result, not 100 — so an undeclared attribute
list overstates INCLUDE cost. The report flags when the fallback was used.
"""
type_sizes = {
"S": 100,
"N": 8,
"BOOL": 1,
"B": 256,
"L": 200,
"M": 200,
"SS": 200,
"NS": 200,
"BS": 200,
"NULL": 1,
}
declared = {}
for ent in table_def.get("entities", []) or []:
for a in ent.get("attributes", []) or []:
if a.get("name"):
declared[a["name"]] = type_sizes.get(a.get("type", "S"), 100)
total, all_declared = 0.0, True
for n in names:
if n in declared:
total += declared[n]
else:
total += 100.0
all_declared = False
return total, all_declared
def _vector_return_bytes_per_result(vi: dict, table_def: dict) -> tuple[float, str]:
"""Bytes returned per search result, driven by the index projection.
Measured per result: ~89 B for KEYS_ONLY, ~98 B for a narrow INCLUDE, and the whole
projected item for ALL (15,243 B on the probe item — roughly 170x KEYS_ONLY). This is
the dominant term for ALL and negligible for KEYS_ONLY, which is why projection choice
drives search cost far more than index size does.
"""
proj = _vector_projection(vi)
ptype = (proj.get("type") or "ALL").upper()
if ptype == "KEYS_ONLY":
return float(VECTOR_RETURN_BYTES_KEYS_ONLY), "KEYS_ONLY, keys only (~89 B measured)"
if ptype == "INCLUDE":
named = proj.get("attributes", []) or []
extra, all_declared = _named_attr_bytes(table_def, named)
note = "" if all_declared else " (some sizes defaulted to 100 B — declare them to tighten)"
return (
VECTOR_RETURN_BYTES_INCLUDE_BASE + extra,
f"INCLUDE, keys + {len(named)} attribute(s){note}",
)
return (
float(_vector_item_bytes(vi, table_def, None)),
"ALL, the whole projected item is returned per result",
)
def vector_search_bytes_per_call(ap: dict, vi: dict, table_def: dict) -> tuple[float, str]:
"""Bytes RETURNED by one SearchVectors call — the part that is soundly measurable.
This is deliberately NOT the full metered figure. Metering also includes the vector
data the search examines during traversal, and measurement showed that term varies by
an order of magnitude with index configuration (see the constants block). So this
returns only the returned-data component, which is exactly linear in TopK and driven by
the projection, and the caller reports it as a DRIVER rather than a cost.
"""
top_k = int(ap.get("top_k", 10) or 10)
per_result, proj_basis = _vector_return_bytes_per_result(vi, table_def or {})
return top_k * per_result, f"TopK {top_k} x {per_result:,.0f} B/result — {proj_basis}"
def vector_search_bytes_spend_gate_upper(ap: dict, vi: dict, table_def: dict) -> float:
"""Deliberately HIGH byte estimate for one SearchVectors call — pre-spend gate ONLY.
Do not use this for a reported cost. vector_search_bytes_per_call() is the reported
driver and is a LOWER bound: returned data only, no traversal term. A spend gate that
under-estimates fails in the expensive direction, so this adds a traversal term at the
steeper of the two measured dimension slopes, then a safety factor.
Measured against the search points from the live probe (declared item size set to the
13,048 B the ALL index actually copies — non-vector payload plus the other vector
attributes in base-table form — so the estimate is not fed the measured answer):
projection dims TopK bound measured ratio
ALL 1024 100 2,608,645 1,524,351 1.71x
ALL 1024 1 57,564 15,276 3.77x
KEYS_ONLY 1024 100 45,145 19,488 2.32x
KEYS_ONLY 1024 1 31,929 10,677 2.99x
KEYS_ONLY 128 1 4,108 2,736 1.50x
INCLUDE 1024 100 60,145 20,437 2.94x
Loosest where the absolute cost is trivial, tightest where the money is — the same
shape the cost model has. See VECTOR_SPEND_GATE_SAFETY for why the factor is load-
bearing rather than padding. The gate is allowed to be loose. It is not allowed to be
low.
"""
returned, _ = vector_search_bytes_per_call(ap, vi, table_def)
dims = int(vi.get("dimensions") or 0)
traversal = dims * VECTOR_TRAVERSAL_BYTES_PER_DIM_UPPER
bound = (returned + traversal) * VECTOR_SPEND_GATE_SAFETY
return max(float(VECTOR_METERING_MIN_BYTES), bound)
def _as_int(value: object) -> int | None:
"""``int(value)`` or None. Never raises.
Everything this function's callers inspect is user-authored JSON, so a non-numeric
value has to become an actionable error string rather than a traceback. A bare
``not dims`` guard does not achieve that: ``"dimensions": "1k"`` is truthy, so it
short-circuits past the guard and reaches ``int("1k")``, crashing the one function
whose whole contract is to fail loudly instead of pricing something impossible.
``bool`` is rejected on purpose. ``int(True)`` is 1, so ``"dimensions": true`` would
otherwise validate as a legal 1-dimension index.
"""
if isinstance(value, bool):
return None
try:
return int(value) # type: ignore[call-overload]
except (TypeError, ValueError):
return None
def validate_vector_model(tables: list, access_patterns: list) -> list[str]:
"""Hard validation. These are service constraints, so a violation means the design
cannot be deployed — better to fail loudly than to price something impossible."""
errors: list[str] = []
for t in tables:
vis = _vector_indexes(t)
if not vis:
continue
tname = t.get("table_name", "<unnamed>")
if len(vis) > MAX_VECTOR_INDEXES_PER_TABLE:
errors.append(
f"{tname}: {len(vis)} vector indexes exceeds the per-table "
f"limit of {MAX_VECTOR_INDEXES_PER_TABLE}"
)
if t.get("provisioned_capacity"):
errors.append(
f"{tname}: vector indexes require on-demand capacity; this "
f"table declares provisioned_capacity"
)
by_attr: dict[str, set] = {}
for vi in vis:
name = vi.get("index_name", "<unnamed>")
# `index_name` is what a SearchVectors pattern's `index` resolves against, so
# a missing one is unpriceable rather than merely untidy. Called out on its own
# because the plausible wrong spelling is a bare `name`, which leaves every
# other field looking correct — observed in a real run.
if not vi.get("index_name"):
errors.append(
f"{tname}: a vector index is missing `index_name` "
f"(found keys: {sorted(vi)}) — the field is `index_name`, not `name`"
)
dims = vi.get("dimensions")
dims_int = _as_int(dims)
if dims_int is None or not (1 <= dims_int <= MAX_VECTOR_DIMENSIONS):
errors.append(
f"{tname}.{name}: dimensions must be an integer 1-"
f"{MAX_VECTOR_DIMENSIONS}, got {dims!r}"
)
fn = (vi.get("distance_function") or "").upper()
if fn not in {"COSINE", "EUCLIDEAN", "DOT_PRODUCT"}:
errors.append(
f"{tname}.{name}: distance_function must be COSINE, "
f"EUCLIDEAN or DOT_PRODUCT, got {vi.get('distance_function')!r}"
)
if not vi.get("vector_attribute"):
errors.append(f"{tname}.{name}: vector_attribute is required")
by_attr.setdefault(vi.get("vector_attribute") or "", set()).add(dims_int or 0)
for attr, dimset in by_attr.items():
if len(dimset) > 1:
errors.append(
f"{tname}: indexes on attribute {attr!r} declare differing "
f"dimensions {sorted(dimset)} — DynamoDB rejects this "
f"('Attributes cannot be redefined')"
)
# Only NAMED indexes are resolvable targets. Including unnamed ones would put
# (table, None) in this set, and a pattern that omits `index` also looks up
# (table, None) — so a missing target would silently MATCH a missing name and the
# error below would never fire. Two bugs cancelling out is not a passing design.
index_names = {
(t.get("table_name"), vi.get("index_name"))
for t in tables
for vi in _vector_indexes(t)
if vi.get("index_name")
}
for ap in access_patterns:
if ap.get("operation") != VECTOR_SEARCH_OP:
continue
pid = ap.get("pattern_id", "<unnamed>")
top_k = ap.get("top_k", 10)
top_k_int = _as_int(top_k)
if top_k_int is None or not (1 <= top_k_int <= MAX_VECTOR_TOP_K):
errors.append(f"{pid}: top_k must be an integer 1-{MAX_VECTOR_TOP_K}, got {top_k!r}")
if not ap.get("table"):
errors.append(
f"{pid}: operation SearchVectors has no `table` (found keys: "
f"{sorted(ap)}) — a vector index belongs to a table, so the pattern must "
f"name it just as every other access pattern does"
)
continue
if not ap.get("index"):
errors.append(
f"{pid}: operation SearchVectors has no `index` "
f"(found keys: {sorted(ap)}) — the field is `index`, not `vector_index`; "
f"it must name an entry in the table's vector_indexes"
)
continue
key = (ap.get("table"), ap.get("index"))
if key not in index_names:
errors.append(
f"{pid}: operation SearchVectors targets "
f"{ap.get('table')}.{ap.get('index')}, which is not declared "
f"in that table's vector_indexes"
)
return errors
def pattern_monthly_cost(
ap: dict,
table_def: dict | None,
entity_attr_sizes: dict | None = None,
) -> dict:
"""Return per-pattern monthly cost and capacity detail for one access pattern.
Inputs:
ap — one access-pattern dict (see cost-model-schema.md).
table_def — the table-def dict (used for GSI amplification); may be None.
entity_attr_sizes — output of _build_entity_attr_sizes(tables); needed to cap
Query/Scan items_per_request at the 900 KB page limit.
Returns:
{
"ap": the effective ap dict (post-cap),
"cap": output of calc_pattern_capacity,
"base_cost": monthly dollars for base-table RCU/WCU,
"gsi_amp_wru": observed-equivalent GSI amp WRU per call,
"gsi_amp_details": list[str] of "IndexName (PROJ): +X WRU",
"gsi_amp_cost": monthly dollars for GSI amplification,
"total_cost": base_cost + gsi_amp_cost,
}
No AWS calls; no side effects. The same formulas the CLI uses.
"""
# SearchVectors consumes no RCU/WCU — it is billed on bytes examined — so it takes
# its own path rather than going through calc_pattern_capacity.
if ap["operation"] == VECTOR_SEARCH_OP:
return _search_vectors_monthly_cost(ap, table_def)
if ap["operation"] in ("Query", "Scan") and entity_attr_sizes is not None:
capped = _cap_items_per_request(ap, entity_attr_sizes)
ap = dict(ap, items_per_request=capped)
cap = calc_pattern_capacity(ap)
rps = ap.get("peak_rps", 0)
# Optional expected/average-volume scenario. `avg_rps` defaults to peak_rps,
# so a model without it produces byte-identical numbers to before. When set,
# `expected_*` is the same per-op CU (scale-invariant) driven at the lower
# average rate — the realistic monthly figure, computed by the calculator
# instead of hand-derived.
avg_rps = ap.get("avg_rps", rps)
per_op_cu_cost = cap["rcus"] * RRU_PRICE + cap["wcus"] * WRU_PRICE
base_cost = per_op_cu_cost * rps * SECONDS_PER_MONTH
expected_base_cost = per_op_cu_cost * avg_rps * SECONDS_PER_MONTH
gsi_amp_wru = 0.0
gsi_amp_details: list[str] = []
if ap["operation"] in WRITE_OPS and table_def:
gsi_amp_wru, gsi_amp_details = calc_gsi_amplification_wru(ap, table_def)
gsi_amp_cost = gsi_amp_wru * WRU_PRICE * rps * SECONDS_PER_MONTH
expected_gsi_amp_cost = gsi_amp_wru * WRU_PRICE * avg_rps * SECONDS_PER_MONTH
cond_fail_rate = ap.get("conditional_fail_rate", 0.0)
if cond_fail_rate and ap["operation"] in WRITE_OPS:
fail_multiplier = cond_fail_rate
base_cost *= 1.0 + fail_multiplier
gsi_amp_cost *= 1.0 + fail_multiplier
expected_base_cost *= 1.0 + fail_multiplier
expected_gsi_amp_cost *= 1.0 + fail_multiplier
# Vector write capacity, if this write touches a vector-indexed attribute. Billed in
# bytes at a different rate from WCU, so it is a separate line rather than folded in.
vec_write_bytes = 0.0
vec_write_details: list[str] = []
vec_write_cost = expected_vec_write_cost = 0.0
if ap["operation"] in WRITE_OPS and table_def and _vector_indexes(table_def):
vec_write_bytes, vec_write_details = vector_write_bytes_per_call(ap, table_def)
rate = (VECTOR_WRITE_PRICE_PER_GB / BYTES_PER_GB) * _ia_multiplier(table_def)
vec_write_cost = vec_write_bytes * rate * rps * SECONDS_PER_MONTH
expected_vec_write_cost = vec_write_bytes * rate * avg_rps * SECONDS_PER_MONTH
if cond_fail_rate:
vec_write_cost *= 1.0 + cond_fail_rate
expected_vec_write_cost *= 1.0 + cond_fail_rate
return {
"ap": ap,
"cap": cap,
"base_cost": base_cost,
"gsi_amp_wru": gsi_amp_wru,
"gsi_amp_details": gsi_amp_details,
"gsi_amp_cost": gsi_amp_cost,
"vector_write_bytes": vec_write_bytes,
"vector_write_details": vec_write_details,
"vector_write_cost": vec_write_cost,
"total_cost": base_cost + gsi_amp_cost + vec_write_cost,
"expected_cost": expected_base_cost + expected_gsi_amp_cost + expected_vec_write_cost,
}
def _vector_search_cap(ap: dict, notes: list[str]) -> dict:
"""A zero-capacity `cap` matching calc_pattern_capacity's shape.
SearchVectors consumes no RCU/WCU, but the report renders every pattern through
the same columns, so the shape must match or rendering breaks.
"""
return {
"op": ap.get("operation", VECTOR_SEARCH_OP),
"rcus": 0.0,
"wcus": 0.0,
"strong": False,
"transactional": False,
"notes": notes,
}
def _search_vectors_monthly_cost(ap: dict, table_def: dict | None) -> dict:
"""Monthly cost for one SearchVectors pattern.
The dollar figure here is an UPPER BOUND, not a model — see
vector_search_bytes_per_call. It is surfaced with its basis string so the report can
say plainly how it was derived and that live validation supersedes it.
"""
# No rate needed: search contributes no priced line, only reported drivers.
vi = next(
(v for v in _vector_indexes(table_def) if v.get("index_name") == ap.get("index")), None
)
if vi is None:
return {
"ap": ap,
"cap": _vector_search_cap(
ap, ["SearchVectors target index not found in the model — cost not estimated"]
),
"base_cost": 0.0,
"gsi_amp_wru": 0.0,
"gsi_amp_details": [],
"gsi_amp_cost": 0.0,
"vector_write_bytes": 0.0,
"vector_write_details": [],
"vector_write_cost": 0.0,
"vector_search_bytes": 0.0,
"vector_search_basis": "target index not declared",
"vector_search_cost": 0.0,
"is_vector_search": True,
"total_cost": 0.0,
"expected_cost": 0.0,
}
per_call, basis = vector_search_bytes_per_call(ap, vi, table_def or {})
# No dollar figure: the examined-data term is not modellable from a design.
cost = expected = 0.0
return {
"ap": ap,
"cap": _vector_search_cap(
ap, ["SearchVectors consumes no RCU/WCU — billed on vector bytes examined"]
),
"base_cost": 0.0,
"gsi_amp_wru": 0.0,
"gsi_amp_details": [],
"gsi_amp_cost": 0.0,
"vector_write_bytes": 0.0,
"vector_write_details": [],
"vector_write_cost": 0.0,
"vector_search_bytes": per_call,
"vector_search_basis": basis,
"vector_search_cost": cost,
"is_vector_search": True,
"total_cost": cost,
"expected_cost": expected,
}
def _gsi_membership_byte_fraction(table_def: dict, gsi: dict) -> float:
"""Fraction of the base table's BYTES that this GSI actually indexes.
A GSI holds only the items that carry its key (PK + SK if composite), so its
storage is base_storage × (member-entity bytes / all-entity bytes), NOT the
full base table. Computed from entity declarations; an ALL-projection GSI on
a single-entity table yields 1.0 (unchanged from legacy behavior).
Guard: the refinement only applies when the GSI's partition key is declared
by at least one entity's attribute list — proof that attributes are
specified meaningfully. If no entity declares the GSI PK at all (attributes
underspecified, or a key that lives only in `attribute_definitions`), we
cannot reason about membership and return 1.0 (conservative — never silently
zeroes a GSI's storage on a thin model).
"""
ents = table_def.get("entities") or []
if not ents:
return 1.0
# Indeterminate membership (GSI key declared on no entity and not a table
# key) → conservative full size; never silently shrink a GSI's storage on a
# thin model. A GSI key that IS a table key is carried by every item → 1.0.
if not _gsi_membership_determinable(table_def, gsi):
return 1.0
table_keys = _table_key_attrs(table_def)
total = 0.0
member = 0.0
for ent in ents:
cnt = ent.get("estimated_item_count", 100_000)
sz = ent.get("estimated_item_size_bytes", 1024)
b = cnt * sz
total += b
attrs = {a["name"] for a in (ent.get("attributes") or []) if a.get("name")}
if _item_carries_gsi_key(attrs, gsi, table_keys):
member += b
if total <= 0:
return 1.0
return member / total
def _storage_cost(tables: list, by_table: dict) -> tuple[list, float]:
"""Storage rows + total dollars for a given per-table byte map.
Factored out so the peak and the expected (average-volume) scenarios run
through identical projection-ratio logic — no drift between the two
headlines. `by_table` maps table_name → steady-state bytes (write-driven);
tables with no write traffic fall back to entity-count storage.
All storage is billed at the **full public rate** — the 25 GB Standard free
tier is deliberately NOT applied. The free tier is account-wide and is often
already consumed by other tables in the same account, so assuming it here
understates the bill and produces a "$0.00, storage is free" claim a user
cannot safely repeat. Pricing at full rate is the honest, account-agnostic
default.
"""
rows = []
total = 0.0
for t in tables:
tname = t["table_name"]
total_bytes = by_table.get(tname, 0)
# Fallback: if no write patterns hit this table, use entity counts.
if total_bytes == 0:
for ent in t.get("entities", []):
item_size = ent.get("estimated_item_size_bytes", 1024)
item_count = ent.get("estimated_item_count", 100_000)
overhead = PER_ITEM_STORAGE_OVERHEAD
if t.get("global_tables"):
overhead += GLOBAL_TABLES_OVERHEAD
total_bytes += item_count * (item_size + overhead)
gb = total_bytes / BYTES_PER_GB
sc = gb * STORAGE_PRICE_PER_GB_MONTH
rows.append([tname, "Table", f"{gb:.2f}", _fmt(sc)])
total += sc
for g in t.get("gsis", []):
proj = g.get("projection", {}) or {}
ptype = proj.get("type", "ALL")
ptype = ptype.upper() if isinstance(ptype, str) else "ALL"
ratio = PROJECTION_STORAGE_RATIO.get(ptype, 1.0)
# A GSI stores only the items that carry its key (PK + SK if
# composite), not the whole base table. Scale by the fraction of
# base BYTES whose entity is a member of this index. On a
# single-entity table (or an underspecified model) this is 1.0, so
# legacy single-table-single-entity output is unchanged.
membership = _gsi_membership_byte_fraction(t, g)
ggb = gb * ratio * membership
gsc = ggb * STORAGE_PRICE_PER_GB_MONTH
rows.append([g["index_name"], "GSI", f"{ggb:.2f}", _fmt(gsc)])
total += gsc
# Vector index storage. Sized from item COUNT rather than base-table bytes,
# because the vector portion is a fixed dimensions × 4 regardless of how large
# the rest of the item is. Priced at the same per-GB rate as table storage —
# there is no separate vector-storage usage type.
for vi in _vector_indexes(t):
n_items, declared = _vector_indexed_item_count(vi, t)
per_item = _vector_item_bytes(vi, t, None)
vgb = (n_items * per_item) / BYTES_PER_GB
vsc = vgb * STORAGE_PRICE_PER_GB_MONTH
label = "Vector index" if declared else "Vector index*"
rows.append([vi.get("index_name", "<unnamed>"), label, f"{vgb:.2f}", _fmt(vsc)])
total += vsc
return rows, total
# =============================================================================
# Main reporting.
# =============================================================================
def calculate_and_report(model: dict, requirements: dict | None = None) -> str:
tables = model.get("tables", [])
access_patterns = model.get("access_patterns", [])
# Lookups.
table_map = {t["table_name"]: t for t in tables}
entity_attr_sizes = _build_entity_attr_sizes(tables)
# Per-pattern costs.
results = []
for ap in access_patterns:
table_def = table_map.get(ap.get("table", ""))
pc = pattern_monthly_cost(ap, table_def, entity_attr_sizes)
eff_ap = pc["ap"]
cap = pc["cap"]
results.append(
{
"pattern_id": eff_ap["pattern_id"],
"description": eff_ap.get("description", ""),
"op": eff_ap["operation"],
"table": eff_ap.get("table", ""),
"index": eff_ap.get("index"),
"rps": eff_ap.get("peak_rps", 0),
"rcus": cap["rcus"],
"wcus": cap["wcus"],
"strong": cap["strong"],
"transactional": cap["transactional"],
"base_cost": pc["base_cost"],
"gsi_amp_cost": pc["gsi_amp_cost"],
"gsi_amp_details": pc["gsi_amp_details"],
"total_cost": pc["total_cost"],
"expected_cost": pc["expected_cost"],
"notes": cap["notes"],
# Vector capacity. Writes and storage are priced into the headline;
# search is carried through as a diagnostic ceiling only.
"vector_write_bytes": pc.get("vector_write_bytes", 0.0),
"vector_write_cost": pc.get("vector_write_cost", 0.0),
"vector_write_details": pc.get("vector_write_details", []),
"is_vector_search": pc.get("is_vector_search", False),
"vector_search_bytes": pc.get("vector_search_bytes", 0.0),
"vector_search_basis": pc.get("vector_search_basis", ""),
"vector_search_cost": pc.get("vector_search_cost", 0.0),
}
)
# -------------------------------------------------------------------------
# Storage estimates.
# -------------------------------------------------------------------------
storage_by_table: dict = {}
# Parallel accumulation at avg_rps so the expected (average-volume) headline
# uses average-rate storage growth, not peak — otherwise storage (often
# ~10-15% of the bill) would keep the "expected" number peak-inflated.
expected_storage_by_table: dict = {}
for ap in access_patterns:
if ap["operation"] not in WRITE_OPS:
continue
write_action = _resolve_write_action(ap, requirements)
create_ratio = CREATE_RATIO.get(write_action, 0.3)
if create_ratio <= 0:
continue
rps = ap.get("peak_rps", 0)
if rps <= 0:
continue
avg_rps = ap.get("avg_rps", rps)
raw_item_size = ap.get("estimated_item_size_bytes", 1024)
overhead = PER_ITEM_STORAGE_OVERHEAD
# Optional: add global tables overhead if configured on the table.
table_name = ap.get("table", "")
table_def = table_map.get(table_name, {})
if table_def.get("global_tables"):
overhead += GLOBAL_TABLES_OVERHEAD
item_size = raw_item_size + overhead
retention_days = _resolve_retention_days(ap, tables, requirements)
create_rate = rps * create_ratio
steady_state_bytes = item_size * create_rate * SECONDS_PER_DAY * retention_days
storage_by_table[table_name] = storage_by_table.get(table_name, 0) + steady_state_bytes
expected_bytes = item_size * (avg_rps * create_ratio) * SECONDS_PER_DAY * retention_days
expected_storage_by_table[table_name] = (
expected_storage_by_table.get(table_name, 0) + expected_bytes
)
storage_rows, storage_total = _storage_cost(tables, storage_by_table)
# Expected (average-volume) storage uses the same logic on avg-rate bytes.
# Only computed/rendered when avg_rps is in play; otherwise identical to peak.
_, expected_storage_total = _storage_cost(tables, expected_storage_by_table)
rw_total = sum(r["total_cost"] for r in results)
total = storage_total + rw_total
# Expected (average-volume) headline. `avg_rps` defaults to peak_rps per
# pattern, so when no pattern sets it, expected_total == total exactly and
# the two-headline block is suppressed (legacy byte-identical output).
any_avg_rps = any("avg_rps" in ap for ap in access_patterns)
expected_rw_total = sum(r["expected_cost"] for r in results)
expected_total = expected_storage_total + expected_rw_total
# Sort results by cost descending.
results.sort(key=lambda r: r["total_cost"], reverse=True)
# Cost concentration note: if a single pattern drives >30% of the RW
# total, item-size drift on that pattern could move the headline number
# materially. Surface it so the user knows where to scrutinise inputs.
concentration_notes = []
if rw_total > 0:
for r in results:
share = r["total_cost"] / rw_total
if share > 0.30:
concentration_notes.append(
f"- `{r['pattern_id']}` contributes {share*100:.0f}% of "
f"read/write cost. A 50% error in `estimated_item_size_bytes` "
f"on this pattern would move the headline number by "
f"~{share*50:.0f}%. Revisit its attribute walkthrough if "
f"you haven't already, or rely on live validation to "
f"confirm the number."
)
# Build report.
lines = [
"# DynamoDB Cost Report",
"",
DISCLAIMER,
"",
]
if any_avg_rps:
# Two labeled headlines: the peak-sustained worst case and the
# expected figure at the stated average volume. This is the calculator
# producing the "realistic" number so the agent never hand-derives it.
lines += [
f"**Peak-Sustained Monthly Cost: {_fmt(total)}** *(every pattern at "
"its declared `peak_rps`, sustained 24/7 — the worst case)*",
"",
f"**Expected Monthly Cost (your stated average volume): "
f"{_fmt(expected_total)}** *(patterns driven at `avg_rps`; on-demand "
"bills per request, so this is the realistic figure)*",
"",
_padded_table(
["Source", "Peak-Sustained", "Expected"],
[
["Storage", _fmt(storage_total), _fmt(expected_storage_total)],
["Read and write requests", _fmt(rw_total), _fmt(expected_rw_total)],
],
),
]
else:
lines += [
f"**Peak-Sustained Monthly Cost: {_fmt(total)}** *(every pattern at "
"its declared `peak_rps`, sustained 24/7; set `avg_rps` on patterns "
"for a realistic average-volume figure)*",
"",
_padded_table(
["Source", "Monthly Cost"],
[
["Storage", _fmt(storage_total)],
["Read and write requests", _fmt(rw_total)],
],
),
]
if concentration_notes:
lines += [
"",
"> **Cost concentration — item-size sensitivity.** One or more "
"patterns drive a disproportionate share of the estimate. The "
"calculator formulas are empirically verified; the remaining risk "
"is in the inputs — particularly `estimated_item_size_bytes`. If "
"any of these patterns' item sizes were guessed without an "
"attribute walkthrough, the headline number could be off by "
"tens of percent.",
"",
*concentration_notes,
]
lines += [
"",
"## Storage Costs",
"",
f"**Monthly Cost:** {_fmt(storage_total)}",
"",
"*Priced at the full public Standard rate ($0.25/GB-month). The 25 GB "
"free tier is not applied — it is account-wide and often already "
"consumed by other tables.*",
"",
_padded_table(["Resource", "Type", "Storage (GB)", "Monthly Cost"], storage_rows),
"",
"## Access Pattern Costs",
"",
f"**Monthly Cost:** {_fmt(rw_total)}",
"",
]
has_gsi_footnote = False
has_vector_search_footnote = False
detail_headers = ["Pattern", "Operation", "Table/Index", "Peak RPS", "RRU/WRU", "Monthly Cost"]
detail_rows = []
for r in results:
target = r["table"]
if r["index"]:
target += f" / {r['index']}"
ru = r["wcus"] if r["wcus"] > 0 else r["rcus"]
flag = ""
if r["transactional"]:
flag += " (txn)"
if r["strong"] and r["wcus"] == 0:
flag += " (strong)"
detail_rows.append(
[
r["pattern_id"],
r["op"] + flag,
target,
f"{r['rps']:.1f}",
f"{ru:.2f}",
# SearchVectors consumes no RCU/WCU AND its capacity is deliberately
# not priced, so BOTH base_cost and vector_search_cost are 0. Rendering
# either as "$0.00" states that search is free — in the one table a
# reader scans to find the expensive patterns. Render the carve-out
# instead and footnote it. This is the whole point of the section below:
# do not let an unpriced dimension read as a free one.
(VECTOR_SEARCH_CELL if r.get("is_vector_search") else _fmt(r["base_cost"])),
]
)
if r.get("is_vector_search"):
has_vector_search_footnote = True
if r["gsi_amp_cost"] > 0:
detail_rows.append(
[
f"{r['pattern_id']}¹",
"GSI writes",
r["table"],
f"{r['rps']:.1f}",
"-",
_fmt(r["gsi_amp_cost"]),
]
)
has_gsi_footnote = True
lines.append(_padded_table(detail_headers, detail_rows))
if has_gsi_footnote:
lines += ["", GSI_FOOTNOTE]
if has_vector_search_footnote:
lines += ["", VECTOR_SEARCH_FOOTNOTE]
lines += _vector_report_section(results)
return "\n".join(lines)
def _vector_report_section(results: list) -> list[str]:
"""The vector-capacity section: what was priced, and what deliberately was not.
Vector index storage and vector WRITE capacity are modelled and already included in
the headline. Vector SEARCH capacity is not, and this section says so explicitly
rather than leaving a $0.00 row unexplained. It reports a byte ceiling as a
diagnostic and tells the user how to obtain the real figure.
"""
searches = [r for r in results if r.get("is_vector_search")]
writes = [
r for r in results if r.get("vector_write_bytes", 0) and not r.get("is_vector_search")
]
if not searches and not writes:
return []
out = ["", "## Vector Index Capacity", ""]
if writes:
out += [
"Vector **write** capacity is metered in bytes at "
f"${VECTOR_WRITE_PRICE_PER_GB:.2f}/GB, separately from base-table WCU, and "
"**is included** in the headline above. A 1 KB minimum applies per request "
"(not per index), so low-dimension vectors do not meter proportionally lower.",
"",
]
rows = [
[
r.get("pattern_id", "?"),
f"{r['vector_write_bytes']:,.0f}",
_fmt(r["vector_write_cost"]),
"; ".join(r.get("vector_write_details") or []) or "-",
]
for r in writes
]
out.append(
_padded_table(["Pattern", "Bytes/write", "Monthly Cost", "Per-index detail"], rows)
)
out.append("")
if searches:
out += [
"Vector **search** capacity is **not priced here**, and that is deliberate. It "
"is metered on the vector data a search examines inside the index plus the data "
"returned, and measurement showed the examined fraction varies by an order of "
"magnitude with index configuration — 12.8% of all vector bytes on a 60-item "
"unpartitioned index against 1.3% on a 200-item partitioned one. A calibration "
"fitted across configurations came out 51-69% low when tested at 256 / 1536 / "
"3072 dimensions, which is exactly where real embedding models sit. A "
"confidently wrong figure is worse than none.",
"",
"What IS measured, and what you can act on:",
"",
"- **`TopK` is exactly linear.** Halving `TopK` halves the returned-data term.",
"- **The projection is a ~170x multiplier** on bytes returned per result: ~89 B "
"for `KEYS_ONLY`, ~98 B for a narrow `INCLUDE`, and the whole projected item "
"for `ALL`. A `TopK=100` search on an `ALL` index measured **1.52 MB**; the same "
"search on `KEYS_ONLY` measured **19 KB**.",
"- **Dimensions scale it linearly** at a fixed index configuration (~31 B per "
"dimension measured at 256 / 1536 / 3072).",
"- **Index size is not a linear driver.** ANN prunes: within one index, going "
"from 10 to 200 vectors in the searched partition barely moved the figure. "
'That is not the same as "population never matters" — an EMPTY 1024-dim '
"index measured ~2x a populated one, so do not extrapolate this shape down "
"to a sparse or freshly-created index.",
"",
"The returned-data component below is the part that follows soundly from your "
"design. Treat it as a **lower bound on bytes**, not a cost.",
"",
]
rows = [
[
r.get("pattern_id", "?"),
f"{r['vector_search_bytes'] / 1_000:,.1f} KB",
r.get("vector_search_basis", "-"),
]
for r in searches
]
out.append(_padded_table(["Pattern", "Returned bytes/search", "Basis"], rows))
out += [
"",
f"**Get the real number before you quote one.** Vector search bills at "
f"${VECTOR_SEARCH_PRICE_PER_GB:.3f}/GB. Set `ReturnConsumedCapacity` on your "
"`SearchVectors` calls and read `VectorSearchRequestBytes`, or chart the "
"`VectorSearchRequestBytes` CloudWatch metric (dimensioned by `TableName` and "
"`VectorIndexName`). A live validation run measures it against your own data "
"and reports the observed value.",
"",
"**The cheapest lever is the projection.** If search cost turns out to matter, "
"narrowing an `ALL` vector index to `KEYS_ONLY` or a tight `INCLUDE` and "
"hydrating the rest with `BatchGetItem` cuts search bytes by orders of "
"magnitude. The projection is immutable, so that is a new-index migration.",
]
return out
# =============================================================================
# CLI.
# =============================================================================
def main():
parser = argparse.ArgumentParser(description="DynamoDB Cost Calculator — reads data model JSON")
parser.add_argument("--model", required=True, help="Path to dynamodb_data_model.json")
parser.add_argument(
"--requirements",
"-r",
help="Path to requirements artifact JSON " "(enables write-action storage modeling)",
)
parser.add_argument(
"--output", "-o", help="Output file (default: cost_report.md " "in same directory)"
)
args = parser.parse_args()
model_path = Path(args.model)
try:
model = load_model(str(model_path))
except (json.JSONDecodeError, FileNotFoundError) as e:
print(f"Error loading model: {e}", file=sys.stderr)
sys.exit(1)
requirements = None
if args.requirements:
try:
requirements = load_model(args.requirements)
except (json.JSONDecodeError, FileNotFoundError) as e:
print(f"Warning: Could not load requirements ({e}), using defaults.", file=sys.stderr)
if not model.get("access_patterns"):
print("Error: No access patterns found in the data model.", file=sys.stderr)
sys.exit(1)
# Hard-fail a malformed or impossible vector design BEFORE pricing it. Without this
# gate the failure is silent and expensive: a vector index missing `vector_attribute`,
# or a SearchVectors pattern whose `index` does not resolve, still produces a
# plausible-looking report with vector write capacity priced at $0 — and $0 on the
# line that is usually the largest in the estimate reads as "vectors are cheap"
# rather than "the model is wrong". Observed exactly that: a model using `name`
# instead of `index_name`, omitting `vector_attribute`, and `vector_index` instead of
# `index` costed the whole vector dimension at zero without a word of complaint.
vector_errors = validate_vector_model(
model.get("tables") or [], model.get("access_patterns") or []
)
if vector_errors:
print(
"Error: the vector portion of this data model cannot be priced as written.",
file=sys.stderr,
)
# Not `e`: this function already binds `e` in two `except ... as e` blocks above,
# and Python deletes that name on leaving the handler.
for problem in vector_errors:
print(f" - {problem}", file=sys.stderr)
# Print the correct shape, not just the complaint. Observed recovery behaviour
# when given only an error: the vector index gets DELETED from the model so the
# calculator will run, and the vector cost silently leaves the estimate — a worse
# outcome than the original mistake. A copyable snippet makes fixing the field
# names the path of least resistance.
print(
"\nFix the model — do NOT delete the vector index to get past this, and do "
"NOT estimate the vector cost by hand. Both drop a real cost that is often "
"the largest line in the estimate. The correct shape is:\n"
'\n "tables": [{\n'
' "table_name": "Products",\n'
' "vector_indexes": [{\n'
' "index_name": "ProductEmbeddingIndex",\n'
' "vector_attribute": "Embedding",\n'
' "dimensions": 1024,\n'
' "distance_function": "COSINE",\n'
' "projection": { "type": "ALL" },\n'
' "search_schema": { "partition_key": "category" }\n'
" }]\n"
" }],\n"
' "access_patterns": [{\n'
' "pattern_id": "FindSimilarProducts",\n'
' "operation": "SearchVectors",\n'
' "table": "Products",\n'
' "index": "ProductEmbeddingIndex",\n'
' "peak_rps": 60,\n'
' "top_k": 50\n'
" }, {\n"
' "pattern_id": "PutProduct",\n'
' "operation": "PutItem",\n'
' "table": "Products",\n'
' "peak_rps": 200,\n'
' "attributes_written": ["Embedding"]\n'
" }]\n"
"\nNote `index_name` (not `name`), `index` (not `vector_index`), and that a "
"write storing the embedding lists it in `attributes_written`. Full schema: "
"references/cost-model-schema.md ('Vector index' and 'Access pattern').",
file=sys.stderr,
)
sys.exit(1)
report = calculate_and_report(model, requirements)
output_path = Path(args.output) if args.output else model_path.parent / "cost_report.md"
output_path.write_text(report)
print(f"Cost report written to {output_path}")
print(
f"Analyzed {len(model['access_patterns'])} access patterns "
f"across {len(model.get('tables', []))} tables."
)
if __name__ == "__main__":
main()
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Needs an MCP server, Needs Python
Security
What our scanner noticed· Low risk
- Instructions that use the network
- Scripts that your agent may run
This skill includes scripts that your agent may run. Read them before you use it.
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Published by
- AWS
- Publisher
- AWS (Official)
- Source
- aws/agent-toolkit-for-aws · skills/specialized-skills/database-skills/amazon-dynamodb · commit 0a218f8
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- NOTICE (added by AutoGineer from NOTICE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by AWS.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download amazon-dynamodb.zip