AutoGineer
Generated banner for Prompt Optimizer
Generated logo for Prompt Optimizer

Prompt Optimizer

SentryOfficialPublished by Sentry

  • Skill
  • Workflow
  • Agent tooling
  • Apache-2.0
  • No issues found
Download ZIP

From the author

Creates, optimizes, and iteratively refines agent prompts, system prompts, developer prompts, and reusable prompt templates. Use when asked to improve a prompt, optimize a system prompt, rewrite an agent prompt, tune prompt wording, make a prompt more reliable, port prompts between OpenAI, Claude, or Gemini, or build prompt evals.
— the description by Sentry

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Prompt Optimizer provides a rigorous, systematic approach to prompt engineering. Rather than relying on intuition, it mandates the creation of evaluation sets (evals) to measure performance, identify failure modes, and validate improvements. The workflow guides users through capturing contract requirements, inventorying external context, choosing model-specific strategies, and executing a disciplined optimization loop. It emphasizes separating stable policy from task-local context and provides a structured method for comparing candidate prompts to ensure changes are causal and effective.

What it does

  • Defines a structured contract for prompt requirements, success criteria, and constraints.
  • Establishes a repeatable optimization loop using evaluation sets to cluster and address failure modes.
  • Provides patterns for layering prompts to separate stable policy from dynamic user input.
  • Offers strategies for porting prompts across different model families like OpenAI, Claude, and Gemini.
  • Maintains an optimization log to track hypotheses, edits, and results to prevent regression.
  • Validates prompt candidates against holdout cases to ensure robustness and prevent overfitting.

When to use it

  • When you need to improve the reliability or performance of an existing agent or system prompt.
  • When porting a prompt between different LLM providers or model versions.
  • When a prompt is failing consistently on specific edge cases or ambiguous inputs.
  • When you need to build a reusable prompt template that maintains consistent behavior across tasks.
  • When you want to move from ad-hoc prompt editing to a data-driven, evidence-based development process.

Advantages

  • Reduces guesswork by grounding all edits in concrete evaluation results.
  • Prevents prompt bloat by enforcing clear separation between policy, examples, and context.
  • Ensures behavioral consistency by using holdout validation to check for regressions.
  • Provides a clear, repeatable framework for team-based prompt development.
  • Helps identify when failures are due to model limitations rather than prompt wording.

Usage examples

“Optimize my customer support agent prompt to reduce hallucinations regarding refund policies.”
The agent will guide you to define success criteria, create a small eval set of refund queries, cluster current failure modes, and iteratively refine the prompt's policy section until performance stabilizes.
“Port this system prompt from GPT-4 to Claude 3.5 Sonnet.”
The agent will analyze the original prompt, identify model-specific dependencies, create a portable base, and generate adapter notes to ensure consistent behavior on the new model family.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/getsentry/prompt-optimizer/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip prompt-optimizer.zip -d .agents/skills

Documentation by Sentry

Prompt Optimizer

Optimize prompts with evals. Keep every instruction, example, and external context reference causal.

Load Only What You Need

NeedRead
New promptreferences/core-patterns.md, references/model-family-notes.md, references/transformed-examples.md
Existing promptreferences/meta-optimization-loop.md, references/core-patterns.md, references/model-family-notes.md
Model-family portreferences/model-family-notes.md, references/core-patterns.md
Repeated failuresreferences/meta-optimization-loop.md, references/core-patterns.md
Weak or ambiguous draftreferences/transformed-examples.md
ProvenanceSOURCES.md

Step 1: Capture Contract

Record before editing:

  • task type: new, refine, port, or debug

  • target model family and snapshot, if known

  • prompt surface: system, developer, user, tool descriptions, examples, schemas

  • layer owners: platform, deployer/persona, retrieved context, user payload

  • objective and non-goals

  • inputs, tools, and external files available

  • required output shape

  • success criteria and failure cases

  • hard constraints: latency, verbosity, safety, budget, tool use, style

If success criteria or examples are missing, create a small eval set first. If the bottleneck is model choice, retrieval, tool schema, or missing evals, say so before rewriting.

Step 2: Inventory External Context

For repo or agent prompts, list stable context by exact path:

Context typeExamples
Agent rulesAGENTS.md, CLAUDE.md
Specsspecs/*.md, docs/api.md
PoliciesSECURITY.md, docs/releasing.md
Examplesexamples/, tests/fixtures/

Rules:

  • Reference stable files by repo-relative path instead of copying them.

  • Paste only excerpts needed for the prompt or eval case.

  • Mark whether a file is loaded, referenced, or out of scope.

  • Avoid vague context pointers such as "read the docs".

Step 3: Choose Model Strategy

Read references/model-family-notes.md.

  • Known family: optimize for that family.

  • Unknown family: write a portable base plus short adapter notes.

  • Snapshot changes: rerun evals.

  • Cross-family divergence: specialize only the failing layer.

Step 4: Shape Prompt

Read references/core-patterns.md.

  • Put stable policy in system or developer.

  • Put task-local facts, retrieved context, and variables in user-facing sections.

  • Keep one owner per behavior rule.

  • Use headings or tags only to separate content types.

  • Put tool policy in prompt text; keep schemas in provider-native tools.

  • Keep persona light unless it changes behavior.

  • Use the shortest wording that preserves the constraint.

  • Cut filler, repeated reminders, dead examples, and rationale that does not affect evals.

Step 5: Optimize

Read references/meta-optimization-loop.md for refinements.

  1. Baseline the current prompt on the same eval slice.

  2. Cluster failures by root cause.

  3. Write concrete edit criticisms.

  4. Generate two to four candidates:

    • minimal-diff repair

    • structure-first rewrite

    • examples-first or tool-rule variant

    • provider adapter when needed

  5. Compare candidates on the same cases.

  6. Keep a short optimization log.

  7. Validate the winner on holdout cases.

  8. Stop on plateau, oscillation, overfit, excessive cost, or non-prompt bottleneck.

Step 6: Return Package

Return:

  1. Target

  2. Success Criteria

  3. External Context

  4. Optimized Prompt

  5. Adapter Notes

  6. Eval Set

  7. Optimization Log

  8. Residual Risks

For existing prompts, include a concise diff-style note of the main behavioral changes.

Failure Modes

  • editing before defining the eval target

  • mixing policy, examples, and raw context without boundaries

  • duplicating rules across layers

  • putting durable policy in user payloads

  • asking for chain-of-thought

  • keeping contradictory legacy instructions

  • overfitting to one or two examples

  • retaining examples that no longer improve evals

  • fixing tool-use failures only in prompt text when tool descriptions or schemas are weak

  • adding markup that does not reduce ambiguity

  • using persona as a substitute for behavior rules

Files

SOURCES.md

Sources

This file tracks the material synthesized into prompt-optimizer.

Selected profile

  • skills/skill-writer/references/examples/workflow-process-skill.md

Why: this skill is a repeatable prompt-optimization workflow with explicit preconditions, ordered execution, validation, and failure handling.

Current source inventory

SourceTypeTrust tierRetrievedConfidenceContributionUsage constraintsNotes
skills/skill-writer/SKILL.mdlocal canonicalcanonical2026-04-19highBaseline synthesis, authoring, and registration workflowlocal repository authorityPrimary workflow source
skills/skill-writer/references/*.mdlocal canonicalcanonical2026-04-19highStructure, depth gates, transformed example requirements, validation expectationslocal repository authorityIncludes authoring and workflow patterns
README.mdrepo conventioncanonical2026-04-18highSkill template, naming, registration conventionsrepository-local policyCanonical public skill inventory
CONTRIBUTING.mdrepo conventioncanonical2026-04-18highLocal testing and registration checklistrepository-local policyConfirms registration steps
AGENTS.mdrepo conventioncanonical2026-04-18highMandatory use of skill-writer, registration checklist, portability conventionsrepository-local policyHighest-priority local instruction source
https://agents.md/official format guidecanonical2026-05-04highAGENTS.md purpose, common sections, nested files, closest-file precedencepublic format guidanceSupports exact external file references in repo agent prompts
https://developers.openai.com/codex/guides/agents-mdofficial product docscanonical2026-05-04highCodex AGENTS.md discovery, scope, merge order, size cap, verificationOpenAI-specific behaviorUsed for agent prompt and repo instruction layering guidance
https://platform.openai.com/docs/guides/prompt-optimizer/official docscanonical2026-05-04highDataset-backed prompt optimization, annotations, graders, manual reviewverify product surface before dashboard-specific instructionsConfirms eval-first and manual-review loop
https://platform.openai.com/docs/guides/promptingofficial docscanonical2026-05-04highPrompt objects, versioning, variables, linked evals, prompt rolesproduct-specific features may evolveSupports reusable prompt packages and eval reruns
https://developers.openai.com/api/docs/guides/reasoning-best-practicesofficial docscanonical2026-04-18highReasoning-model prompting differences, simplicity, delimiters, no explicit CoTverify product syntax if exact API behavior mattersUsed for OpenAI family notes
https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overviewofficial docscanonical2026-04-18highSuccess-criteria-first prompting and reminder that not every failure is a prompt problemproduct syntax may evolveSupports precondition checks
https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practicesofficial docscanonical2026-04-18highXML tags, role prompting, examples, long-context ordering, output controlprovider-specific behaviorUsed for Claude family notes and marker guidance
https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-toolsofficial docscanonical2026-04-18mediumPrompt improver inputs: prompt, failure feedback, ideal examplesconsole workflow may changeReinforces critique-plus-example loop
https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practicesofficial docscanonical2026-04-18highConcision as a context-budget discipline; "context window is a public good" framingskill-oriented source, generalized carefully to prompt authoringSupports prompt compaction guidance
https://ai.google.dev/gemini-api/docs/prompting-strategiesofficial docscanonical2026-04-18highClear instructions, example consistency, positive patterns, prompt iterationprovider-specific behaviorUsed for Gemini family notes
https://ai.google.dev/gemini-api/docs/text-generationofficial docscanonical2026-04-18highSystem instruction surface for Geminiproduct syntax may evolveConfirms system-instruction layer
https://ai.google.dev/gemini-api/docs/thinkingofficial docscanonical2026-04-18highDynamic thinking defaults and controllable reasoning settingsproduct syntax may evolveUsed for Gemini adapter notes
https://ai.google.dev/gemini-api/docs/long-contextofficial docscanonical2026-04-18highMany-shot in-context learning and long-context prompt placement notesprovider-specific behaviorSupports many-shot adapter note
https://arxiv.org/abs/2309.03409research papercanonical2026-04-18highOPRO candidate search loop and score-driven prompt optimizationresearch result, not product guaranteeSupports candidate beam design
https://arxiv.org/abs/2305.03495research papercanonical2026-04-18highTextual gradients, minibatch critiques, beam searchresearch result, not product guaranteeSupports critique-driven edits
https://arxiv.org/abs/2303.17651research papercanonical2026-04-18highFEEDBACK -> REFINE loop and test-time improvementresearch result, not product guaranteeSupports iterative refinement loop
https://arxiv.org/abs/2303.11366research papercanonical2026-04-18highReflection memory across trials for agentsresearch result, not product guaranteeSupports optimization log and reflection memory
https://dspy.ai/official project docscanonical2026-04-18highCurrent prompt optimizers such as GEPA and MIPROv2; score-driven instruction search; composable optimizationframework-specific guidanceSupports modern optimizer framing

Decisions

  1. Use an eval-first workflow before prompt edits. Status: adopted Why: Anthropic explicitly recommends success criteria, tests, and a first draft before prompt engineering, and OpenAI's prompt optimizer centers graders and annotations.

  2. Keep a provider-agnostic base prompt plus model-family adapters. Status: adopted Why: OpenAI, Anthropic, and Gemini each recommend materially different prompting strategies around reasoning, XML tags, examples, and thinking controls.

  3. Use markers and tags selectively, not automatically. Status: adopted Why: OpenAI and Anthropic both recommend delimiters for clarity, but over-structuring adds noise and reduces readability.

  4. Use a candidate-based optimization loop instead of one linear rewrite. Status: adopted Why: OPRO, ProTeGi, and DSPy all support score-driven search across multiple prompt variants.

  5. Keep an explicit optimization log and reflective memory across rounds. Status: adopted Why: Self-Refine and Reflexion both show the value of critique and retained reflection across iterations.

  6. Optimize examples, tool rules, and output contracts together with the core prompt. Status: adopted Why: Provider docs and DSPy both treat examples and instructions as first-class levers, not optional decoration.

  7. Make prompt compaction and de-duplication a first-class optimization step. Status: adopted Why: OpenAI recommends simple, direct prompts; Gemini explicitly says to avoid passing tokens you do not need; Anthropic frames concision as a context-budget issue.

  8. Make context ordering explicit rather than implicit. Status: adopted Why: Gemini and Anthropic both document that long-context prompts perform better when evidence comes before the final query.

  9. Inventory stable external context by exact path. Status: adopted Why: AGENTS.md and Codex guidance favor explicit project instruction sources, and path inventories prevent prompts from copying stale docs or using vague "read the docs" pointers.

  10. Add External Context to the returned prompt package. Status: adopted Why: another engineer should see which specs, docs, policies, and examples were loaded, referenced, or left out of scope.

Coverage matrix

DimensionCoverage statusEvidence
Preconditions and eval inputscompleteAnthropic overview, OpenAI prompt optimizer, repo authoring guidance
Ordered optimization flowcompleteworkflow-process example, OPRO, ProTeGi, Self-Refine, DSPy
Failure handling and stop conditionscompleteAnthropic overview, Self-Refine, Reflexion, DSPy
Model-family variancecompleteOpenAI, Anthropic, Gemini docs
Prompt compaction and deduplicationcompleteOpenAI reasoning best practices, Anthropic skill best practices, Gemini long-context guidance
Context ordering and query placementcompleteAnthropic prompting docs, Gemini long-context guidance
External file inventoriescompleteAGENTS.md format guidance, OpenAI Codex AGENTS.md docs, user concern
Safety and escalation boundariescompleteprovider docs plus repo workflow conventions
Output and acceptance checkscompleteOpenAI prompting and optimizer docs, skill-writer output patterns
Transformed example artifactscompletereferences/transformed-examples.md
Future-family coverage beyond OpenAI/Claude/Geminipartialcurrently deferred until there is a concrete repo need

Description QA

Should trigger

  • "Improve this system prompt for our coding agent."

  • "Optimize an agent prompt for Claude."

  • "Rewrite this OpenAI developer prompt."

  • "Port this prompt from GPT to Gemini."

  • "Make this tool-using prompt more reliable."

  • "Tune this prompt wording with a proper eval loop."

Should not trigger

  • "Write a new skill."

  • "Review my pull request."

  • "Fix this bug in the repository."

  • "Summarize this document."

  • "Design a new model architecture."

  • "Tune only the temperature and top-p settings."

Open gaps

  1. Add provider notes for additional families only when the repository has a real downstream need.

  2. If this skill starts getting heavy use, add an optional script or eval template for capturing prompt scores across rounds.

  3. Re-check provider docs when major model-family updates land, especially around tool use and reasoning defaults.

  4. Add durable before/after examples for external context inventories if future prompt changes regress into vague file references.

Stopping rationale

Further retrieval is currently low-yield for this first version. The source pack already covers:

  • official prompt guidance from the three major model families most likely to matter in practice

  • current optimizer framing from a live prompt-programming framework

  • foundational prompt-optimization research for candidate search, critique loops, and reflection memory

  • local repository conventions for canonical skill authoring and registration

Changelog

  • 2026-04-18: Created the initial prompt-optimizer skill, references, and provenance record.

  • 2026-04-18: Added an explicit prompt learnings pass covering compaction, deduplication, and context ordering.

  • 2026-04-18: Folded the prompt learnings back into the core shaping and iteration guidance to keep the workflow compact.

  • 2026-05-04: Added exact external context inventories, External Context output, concise runtime workflow, and SPEC.md.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Sentry
Publisher
Sentry (Official)
Source
getsentry/skills · skills/prompt-optimizer · commit e94f222
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Sentry.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download prompt-optimizer.zip