Phoenix CLI

GitHubOfficialCreated by arize-aiDistributed by GitHub

Skill
Tool integration
Software development
Apache-2.0
No issues found

From the author

Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing traces or spans, investigating LLM/agent failures, deciding what to do after instrumenting an app, building failure taxonomies, choosing what evals to write, or asking "what's going wrong", "what kinds of mistakes", or "where do I focus" — even without naming a technique.

— the description by arize-ai

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Phoenix CLI is a command-line tool designed for inspecting and debugging LLM-powered applications. It allows developers to fetch and analyze traces, spans, and sessions, facilitating the identification of failure modes. The tool supports structured workflows like open coding and axial coding to help developers build failure taxonomies, prioritize fixes, and design evaluation targets based on real-world application traffic.

What it does

Fetches and lists traces, spans, and sessions for LLM applications.
Supports annotation of artifacts to track observations and failure modes.
Enables structured qualitative analysis through open and axial coding workflows.
Provides access to datasets, experiments, and annotation configurations.
Allows custom data extraction and analysis via a GraphQL API.
Manages multiple connection profiles for local, staging, and production environments.

When to use it

When you need to investigate why an LLM agent or application is failing.
When you want to analyze LLM traffic to identify patterns in errors or performance.
When you are building a failure taxonomy to prioritize development fixes.
When you need to determine which evaluation metrics to implement for your application.
When you need to switch between different Phoenix project environments efficiently.

Advantages

Provides a standardized way to perform qualitative analysis on LLM outputs.
Supports complex filtering and data extraction using standard CLI tools like jq.
Enables consistent tracking of failure modes across different development runs.
Simplifies the transition from raw observation to structured evaluation targets.

Usage examples

“List the last 20 traces that resulted in an error.”: The CLI executes a trace list command filtered by status, outputting the results in a raw format suitable for further processing with jq.
“Add a note to trace ID 12345 indicating that the context was lost.”: The CLI records the note against the specified trace, optionally tagging it with a coding annotation identifier for later grouping.
“Switch to the staging profile and check the connection status.”: The CLI updates the active configuration to the staging profile and verifies connectivity to the configured endpoint.
“Fetch all spans for a specific trace ID.”: The CLI retrieves the span data for the given trace, allowing for granular inspection of individual operations within the call graph.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/github/phoenix-cli/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip phoenix-cli.zip -d .agents/skills

Documentation by arize-ai

Phoenix CLI

Invocation

px <resource> <action>                          # if installed globally
npx @arizeai/phoenix-cli <resource> <action>    # no install required

The CLI uses singular resource commands with subcommands like list and get:

px trace list
px trace get <trace-id>
px trace annotate <trace-id>
px trace add-note <trace-id>
px trace-annotations delete
px span list
px span annotate <span-id>
px span add-note <span-id>
px span-annotations delete
px session list
px session get <session-id>
px session annotate <session-id>
px session add-note <session-id>
px session-annotations delete
px dataset list
px dataset get <name>
px project list
px project get <name>
px annotation-config list
px auth status
px profile list
px profile show [name]
px profile create <name>
px profile use <name>
px profile edit <name>
px profile delete <name>

Setup

export PHOENIX_HOST=http://localhost:6006
export PHOENIX_PROJECT=my-project
export PHOENIX_API_KEY=your-api-key  # if auth is enabled

Always use --format raw --no-progress when piping to jq.

Quick Reference

Task	Files
Look at sampled traces, spans, or sessions and write specific notes about what went wrong (no taxonomy yet)	references/open-coding
Group those notes into a structured failure taxonomy and quantify what matters	references/axial-coding

Both stages tag every artifact with one shared coding annotation identifier (descriptive shape, e.g. coding-run:chatbot-context-loss-2026-05-06) so the run is queryable, reversible, and viewable as a unit. Pass --identifier <value> explicitly on every px call — shell inheritance is unreliable across agent harnesses. Open coding writes notes via px ... add-note and records a small local JSONL sidecar at .px/coding/<sanitized-identifier>.jsonl; axial coding reads that sidecar as the deterministic handoff and records labels in .px/coding/<sanitized-identifier>-axial.jsonl. Pick the identifier once per run (see references/open-coding.md), then share the Phoenix UI link from the wrap-up section. Revert is opt-in and runs three identifier-bound DELETEs only after explicit user confirmation.

Workflow term vs. server annotation name. The skill prose calls this value the coding annotation identifier (shell-variable hint: CODING_ANNOTATION_IDENTIFIER). The server-side annotation NAME used for the UI filter is unchanged — coding_session_id — for data compatibility with rows already written by previous runs. Don't try to rename the server-side annotation; treat the asymmetry as load-bearing.

Workflows

"What do I do after instrumenting?" / "Where do I focus?" / "What's going wrong?" open-coding → axial-coding → build evals for the top categories.

Reference Categories

Prefix	Description
`references/open-coding`	Free-form notes against sampled traces, spans, or sessions — reach for it whenever the user wants to make sense of LLM traffic but has no failure categories yet. Includes a unit-of-analysis diagnostic so the workflow runs at the level the failure modes actually live at (trace for stateless single-shot calls, session for multi-turn agents, span for mechanical/in-isolation failures).
`references/axial-coding`	Inductive grouping of notes into a MECE taxonomy with counts — reach for it whenever the user has observations and needs categories or eval targets

Auth

px auth status                                # check connection and authentication
px auth status --endpoint http://other:6006   # check a specific endpoint
px auth status --profile staging              # check a named profile's connection

Profiles

Named profiles let you switch between multiple Phoenix instances (local, staging, cloud) without juggling environment variables. Profiles are stored in ~/.px/settings.json (or $XDG_CONFIG_HOME/px/settings.json).

Configuration priority (highest to lowest): CLI flags > env vars > active profile > built-in defaults.

px profile list                              # list all profiles (shows active profile)
px profile show                              # show the active profile's settings
px profile show staging                      # show a named profile's settings
px profile create prod --endpoint https://app.phoenix.arize.com --api-key <key> --activate
px profile create local --endpoint http://localhost:6006 --project my-app
px profile use prod                          # switch the active profile
px profile edit prod                         # open profile JSON in $EDITOR (validates on save)
px profile delete prod --yes                 # delete a profile (--yes skips confirmation)

Use --profile <name> on any command to target a specific profile without changing the active one:

px trace list --profile staging --limit 10 --format raw --no-progress | jq .
px auth status --profile prod

px profile create options: --endpoint <url>, --project <name>, --api-key <key>, --header <key=value> (repeatable), --activate.

Projects

px project list                                            # list all projects (table view)
px project list --format raw --no-progress | jq '.[].name' # project names as JSON
px project get my-project --format raw --no-progress       # single record by exact name
px project get my-project --format raw --no-progress | jq -r '.id'  # extract project id

project get exits with ExitCode.FAILURE (1) on a name miss and writes a StructuredError {error, code: "FAILURE", hint} to stderr in --format json|raw.

Traces

px trace list --limit 20 --format raw --no-progress | jq .
px trace list --last-n-minutes 60 --limit 20 --format raw --no-progress | jq '.[] | select(.status == "ERROR")'
px trace list --since 2025-01-15T00:00:00Z --limit 50 --format raw --no-progress | jq .
px trace list --format raw --no-progress | jq 'sort_by(-.duration) | .[0:5]'
px trace list --include-notes --format raw --no-progress | jq '.[].notes'
px trace get <trace-id> --format raw | jq .
px trace get <trace-id> --format raw | jq '.spans[] | select(.status_code != "OK")'
px trace get <trace-id> --include-notes --format raw | jq '.notes'
px trace annotate <trace-id> --name reviewer --label pass
px trace annotate <trace-id> --name reviewer --score 0.9 --format raw --no-progress
px trace annotate <trace-id> --name reviewer --label pass --identifier "<coding-annotation-id>"  # tag with a coding annotation identifier
px trace add-note <trace-id> --text "needs follow-up"
px trace add-note <trace-id> --text "needs follow-up" --identifier "<coding-annotation-id>"  # tag + upsert on identifier
px trace-annotations delete --identifier "<coding-annotation-id>" --all -y            # nuke every annotation tied to this coding annotation identifier

px <entity>-annotations delete requires --all or both --start-time and --end-time and emits {deleted: true, target, filter} on success.

Trace JSON shape

Trace
  traceId, status ("OK"|"ERROR"), duration (ms), startTime, endTime
  annotations[] (with --include-annotations, excludes note)
    name, result { score, label, explanation }
  notes[] (with --include-notes)
    name="note", result { explanation }
  rootSpan  — top-level span (parent_id: null)
  spans[]
    name, span_kind ("LLM"|"CHAIN"|"TOOL"|"RETRIEVER"|"EMBEDDING"|"AGENT"|"RERANKER"|"GUARDRAIL"|"EVALUATOR"|"UNKNOWN")
    status_code ("OK"|"ERROR"|"UNSET"), parent_id, context.span_id
    notes[] (with --include-notes)
      name="note", result { explanation }
    attributes
      input.value, output.value          — raw input/output
      llm.model_name, llm.provider
      llm.token_count.prompt/completion/total
      llm.token_count.prompt_details.cache_read
      llm.token_count.completion_details.reasoning
      llm.input_messages.{N}.message.role/content
      llm.output_messages.{N}.message.role/content
      llm.invocation_parameters          — JSON string (temperature, etc.)
      exception.message                  — set if span errored

Spans

px span list --limit 20                                    # recent spans (table view)
px span list --last-n-minutes 60 --limit 50                # spans from last hour
px span list --since 2025-01-15T00:00:00Z --limit 50       # spans since a timestamp
px span list --span-kind LLM --limit 10                    # only LLM spans
px span list --status-code ERROR --limit 20                # only errored spans
px span list --name chat_completion --limit 10             # filter by span name
px span list --trace-id <id> --format raw --no-progress | jq .   # all spans for a trace
px span list --parent-id null --limit 10                   # only root spans
px span list --parent-id <span-id> --limit 10              # only children of a span
px span list --include-annotations --limit 10              # include annotation scores
px span list --include-notes --limit 10                    # include span notes
px span list --attribute llm.model_name:gpt-4 --limit 10  # filter by string attribute
px span list --attribute llm.token_count.total:500 --limit 10  # filter by numeric attribute
px span list --attribute 'user.id:"12345"' --limit 10     # force string match for numeric-looking value
px span list --attribute session.id:sess:abc:123 --limit 20  # colon in value OK (split on first colon only)
px span list --attribute llm.model_name:gpt-4 --attribute session.id:abc --limit 10  # AND multiple filters
px span list output.json --limit 100                       # save to JSON file
px span list --format raw --no-progress | jq '.[] | select(.status_code == "ERROR")'
px span annotate <span-id> --name reviewer --label pass
px span annotate <span-id> --name checker --score 1 --annotator-kind CODE
px span annotate <span-id> --name reviewer --label pass --identifier "<coding-annotation-id>"  # tag with a coding annotation identifier
px span add-note <span-id> --text "verified by agent"
px span add-note <span-id> --text "verified by agent" --identifier "<coding-annotation-id>"  # tag + upsert on identifier
px span-annotations delete --identifier "<coding-annotation-id>" --all -y           # nuke every annotation tied to this coding annotation identifier

Span JSON shape

Span
  name, span_kind ("LLM"|"CHAIN"|"TOOL"|"RETRIEVER"|"EMBEDDING"|"AGENT"|"RERANKER"|"GUARDRAIL"|"EVALUATOR"|"UNKNOWN")
  status_code ("OK"|"ERROR"|"UNSET"), status_message
  context.span_id, context.trace_id, parent_id
  start_time, end_time
  attributes
    input.value, output.value          — raw input/output
    llm.model_name, llm.provider
    llm.token_count.prompt/completion/total
    llm.input_messages.{N}.message.role/content
    llm.output_messages.{N}.message.role/content
    llm.invocation_parameters          — JSON string (temperature, etc.)
    exception.message                  — set if span errored
  annotations[] (with --include-annotations, excludes note)
    name, result { score, label, explanation }
  notes[] (with --include-notes)
    name="note", result { explanation }

Sessions

px session list --limit 10 --format raw --no-progress | jq .
px session list --order asc --format raw --no-progress | jq '.[].session_id'
px session list --include-annotations --include-notes --format raw --no-progress | jq '.[].notes'
px session get <session-id> --format raw | jq .
px session get <session-id> --include-annotations --format raw | jq '.session.annotations'
px session get <session-id> --include-notes --format raw | jq '.session.notes'
px session annotate <session-id> --name reviewer --label pass
px session annotate <session-id> --name reviewer --score 0.9 --format raw --no-progress
px session annotate <session-id> --name reviewer --label pass --identifier "<coding-annotation-id>"  # tag with a coding annotation identifier
px session add-note <session-id> --text "verified by agent"
px session add-note <session-id> --text "verified by agent" --identifier "<coding-annotation-id>"  # tag + upsert on identifier
px session-annotations delete --identifier "<coding-annotation-id>" --all -y              # nuke every annotation tied to this coding annotation identifier

Session JSON shape

SessionData
  id, session_id, project_id
  start_time, end_time
  token_count_prompt, token_count_completion, token_count_total  — cumulative across all LLM spans in the session (int, default 0)
  annotations[] (with --include-annotations, excludes note)
    name, result { score, label, explanation }
  notes[] (with --include-notes)
    name="note", result { explanation }
  traces[]
    id, trace_id, start_time, end_time

Datasets / Experiments / Prompts

px dataset list --format raw --no-progress | jq '.[].name'
px dataset get <name> --format raw | jq '.examples[] | {input, output: .expected_output}'
px dataset get <name> --split train --format raw | jq .    # filter by split
px dataset get <name> --version <version-id> --format raw | jq .
px experiment list --dataset <name> --format raw --no-progress | jq '.[] | {id, name, failed_run_count}'
px experiment get <id> --format raw --no-progress | jq '.[] | select(.error != null) | {input, error}'
px prompt list --format raw --no-progress | jq '.[].name'
px prompt get <name> --format text --no-progress   # plain text, ideal for piping to AI

Annotation Configs

px annotation-config list                                           # list all configs (table view)
px annotation-config list --format raw --no-progress | jq '.[].name' # config names as JSON

GraphQL

For ad-hoc queries not covered by the commands above. Output is {"data": {...}}.

px api graphql '{ projectCount datasetCount promptCount evaluatorCount }'
px api graphql '{ projects { edges { node { name traceCount tokenCountTotal } } } }' | jq '.data.projects.edges[].node'
px api graphql '{ datasets { edges { node { name exampleCount experimentCount } } } }' | jq '.data.datasets.edges[].node'
px api graphql '{ evaluators { edges { node { name kind } } } }' | jq '.data.evaluators.edges[].node'

# Introspect any type
px api graphql '{ __type(name: "Project") { fields { name type { name } } } }' | jq '.data.__type.fields[]'

Key root fields: projects, datasets, prompts, evaluators, projectCount, datasetCount, promptCount, evaluatorCount, viewer.

Docs

Download Phoenix documentation markdown for local use by coding agents.

px docs fetch                                # fetch default workflow docs to .px/docs
px docs fetch --workflow tracing             # fetch only tracing docs
px docs fetch --workflow tracing --workflow evaluation
px docs fetch --dry-run                      # preview what would be downloaded
px docs fetch --refresh                      # clear .px/docs and re-download
px docs fetch --output-dir ./my-docs         # custom output directory

Key options: --workflow (repeatable, values: tracing, evaluation, datasets, prompts, integrations, sdk, self-hosting, all), --dry-run, --refresh, --output-dir (default .px/docs), --workers (default 10).

Files

ATTRIBUTION.mdAdded by AutoGineer988 B
LICENSEAdded by AutoGineer10.0 KiB
SKILL.md15.9 KiB
references/axial-coding.md14.7 KiB
references/open-coding.md20.0 KiB

references/axial-coding.md

Axial Coding

Group open-ended observations into structured failure taxonomies. Axial coding turns notes, trace observations, or open-coding output into named categories with counts, supporting downstream work like eval design and fix prioritization. It works well after open coding, but can start from any set of open-ended observations.

Reach for this whenever the user has observations and needs structure — e.g., "what categories of failures do we have", "what should I build evals for", "how do I prioritize fixes", "group these notes", "MECE breakdown", or any framing that asks for categories or counts grounded in real traces rather than invented top-down.

Coding annotation identifier (reuse the open-coding value)

Reuse the coding annotation identifier chosen in open coding — every annotate call below passes --identifier "$CODING_ANNOTATION_IDENTIFIER" explicitly. In a fresh shell or fresh agent invocation, set CODING_ANNOTATION_IDENTIFIER to the same value (recoverable from the wrap-up UI URL or by listing .px/coding/*.jsonl); don't mint a new id. See open-coding.md#coding-annotation-identifier-pick-this-first for the rationale and the sanitization rule.

Workflow term vs. server annotation name. The skill calls this value the coding annotation identifier; the server annotation NAME used for the UI filter stays coding_session_id for data compatibility. Don't try to rename the server-side key.

CODING_ANNOTATION_IDENTIFIER="coding-run:chatbot-context-loss-2026-05-06"
SLUG=$(echo -n "$CODING_ANNOTATION_IDENTIFIER" | sed 's/[^a-zA-Z0-9_-]/-/g')
NOTES_SIDECAR=".px/coding/${SLUG}.jsonl"
AXIAL_SIDECAR=".px/coding/${SLUG}-axial.jsonl"

Choosing the unit

Open coding's diagnostic in open-coding.md#choosing-the-unit-of-analysis commits to a unit (trace, span, or session). Axial coding inherits that unit by default — if open coding ran at the session level, axial labels will too; same for trace and span.

An axial label can live at a different level than the note that informed it — that's a feature, and it works in every direction:

Trace → span: a trace-level note "answered shipping when asked about returns" can produce a span-level annotation on the retrieval span once a pattern reveals retrieval as the consistent culprit.
Trace → session: a batch of trace-level notes describing single-turn confusion can produce a session-level annotation once you see the pattern is "the agent doesn't track the user's stated context across turns."
Session → trace: a session-level note about cross-turn drift may, on closer reading, attribute to one specific turn where the agent dropped the thread; a trace-level annotation can name that turn.

Whichever level you write the axial label on, write the matching coding_session_id UI-filter annotation on the same entity (see UI-filter annotation below) so the UI link picks it up.

Process

Set the coding annotation identifier — set CODING_ANNOTATION_IDENTIFIER to the value used in open coding and re-derive SLUG, NOTES_SIDECAR, AXIAL_SIDECAR (see Coding annotation identifier)
Gather — read open-coding notes from $NOTES_SIDECAR (at the unit committed in open coding); no server round-trip
Pattern — group notes with common themes
Name — create actionable category names
Attribute — decide what level each category lives at; an axial label can move up (trace → session) or down (trace → span) from the source note's level to the level the pattern actually implicates
Record — px {trace,span,session} annotate ... --name axial_coding_category --label <cat> --identifier "$CODING_ANNOTATION_IDENTIFIER", add/update one JSONL sidecar row for the label, then write the matching coding_session_id UI-filter annotation
Quantify — count failures per category from $AXIAL_SIDECAR

Example Taxonomy

failure_taxonomy:
  content_quality:
    hallucination: [invented_facts, fictional_citations]
    incompleteness: [partial_answer, missing_key_info]
    inaccuracy: [wrong_numbers, wrong_dates]

  communication:
    tone_mismatch: [too_casual, too_formal]
    clarity: [ambiguous, jargon_heavy]

  context:
    user_context: [ignored_preferences, misunderstood_intent]
    retrieved_context: [ignored_documents, wrong_context]

  safety:
    missing_disclaimers: [legal, medical, financial]

Reading

1. Gather — read this run's open-coding notes from the sidecar

Open-coding wrote one JSONL line per note to $NOTES_SIDECAR (.px/coding/${SLUG}.jsonl). Read it directly — no server round-trip is needed. Each line has entity_kind, entity_id, note, identifier, and ts. If the same (entity_kind, entity_id) appears more than once, use the newest ts as the current note.

Missing-file behavior. An absent $NOTES_SIDECAR means open coding hasn't run for this coding annotation identifier in this CWD — stop and run open coding first, do not silently treat it as zero notes.

Malformed lines. Each line is independently parseable JSON. If jq reports a parse error, fix or drop that line manually; do not edit other lines.

Notes outside this run. The sidecar only carries notes this CWD wrote. To pull notes another reviewer or earlier run wrote, fetch them via px {trace,span,session} list --include-notes (embeds notes into row output) — the workflow's sidecar is intentionally per-CWD-per-coding-identifier.

2. Group — synthesize categories

Review the note text collected above. Manually identify recurring themes and draft candidate category names. Aim for MECE coverage: each note should fit exactly one category.

3. Record — write axial-coding labels

Write one annotation per entity using px {trace,span,session} annotate, passing --identifier "$CODING_ANNOTATION_IDENTIFIER" explicitly on every call, and record one JSONL row in $AXIAL_SIDECAR so Quantify below can count without a server round-trip. The level can differ from where the source note lives — see Recording below.

4. Quantify — count per category from the axial sidecar

Counts come from $AXIAL_SIDECAR (populated by Record). No server query, no project-wide history mixed in — the sidecar holds exactly the labels this run wrote. Count the current rows by axial_label; if an entity appears more than once, use the newest ts.

Same missing-file and malformed-line rules as $NOTES_SIDECAR: a missing axial sidecar means no labels have been written yet (run Record); malformed lines are line-local — fix or drop, don't edit neighbors.

Recording

Use the matching annotate command for the level the label belongs at — which may differ from where the source note lives (see Choosing the unit). Every call carries --identifier "$CODING_ANNOTATION_IDENTIFIER" and --format raw --no-progress, and is paired with a JSONL row in $AXIAL_SIDECAR.

Axial sidecar JSONL line shape (one per annotate):

{"entity_kind":"trace","entity_id":"<trace-id>","annotation_name":"axial_coding_category","axial_label":"<label>","explanation":"<optional explanation>","identifier":"<original identifier value, unsanitized>","ts":"<ISO-8601 UTC>"}

Fields:

entity_kind — "trace", "span", or "session" (matches the annotate subcommand)
entity_id — the entity argument passed to annotate
annotation_name — always "axial_coding_category" for axial labels (the workflow's reserved annotation name)
axial_label — the --label value, verbatim; this is what Quantify groups on
explanation — optional, but include it when the annotate call used --explanation
identifier — the original $CODING_ANNOTATION_IDENTIFIER value, unsanitized; the sanitized form lives only in the filename
ts — ISO-8601 UTC timestamp of the local append

If you revise a label for the same entity under the same coding annotation identifier, either replace that row or append a newer row. When duplicate (entity_kind, entity_id, annotation_name) rows exist, the newest ts is the current label. This matches the server upsert behavior of annotate --identifier.

Minimal trace example:

px trace annotate <trace-id> \
  --name axial_coding_category \
  --label answered_off_topic \
  --explanation "asked about returns; answer covered shipping" \
  --annotator-kind HUMAN \
  --identifier "$CODING_ANNOTATION_IDENTIFIER" \
  --format raw --no-progress

Then add a matching JSONL row to $AXIAL_SIDECAR using the line shape above. For span or session labels, change entity_kind, entity_id, and the px subcommand accordingly.

Accepted flags: --name, --label, --score, --explanation, --annotator-kind (HUMAN, LLM, CODE), --identifier. There is no --sync flag — the CLI passes sync=true itself.

UI-filter annotation

Write a coding_session_id annotation at the same level as the axial label — see open-coding.md#ui-filter-annotation for why the Phoenix UI filter requires a name-based annotation rather than the bare --identifier. If open coding already wrote coding_session_id on the same entity, this call upserts (idempotent). The annotation NAME coding_session_id is unchanged; only the workflow's spoken term is "coding annotation identifier".

# Same level as the axial label above
px trace annotate <trace-id> \
  --name coding_session_id \
  --label "$CODING_ANNOTATION_IDENTIFIER" \
  --identifier "$CODING_ANNOTATION_IDENTIFIER"
# or px span annotate / px session annotate at matching levels

Recording discipline

Axial coding categorizes the entities you took notes on during open coding. Use $NOTES_SIDECAR as the source of candidate entities and write labels only after reading the note text and surrounding trace/span/session context. Do not filter by --status-code ERROR — that captures only spans where Python raised, which excludes most failure modes (hallucination, wrong tone, retrieval miss). See open-coding.md for the full reasoning.

Fallback paths: REST POST /v1/{trace,span,session}_annotations and @arizeai/phoenix-client's addSpanAnnotation / addSessionAnnotation (no addTraceAnnotation is exported today — use REST or px trace annotate). The GraphQL endpoint rejects mutations.

Wrapping up

After axial coding finishes, share the Phoenix UI link with the user. The link points to the project's traces table filtered by the coding_session_id annotation — annotations['coding_session_id'].label == '<coding-annotation-id>'. The UI route /projects/:projectId expects an encoded GraphQL node ID, not a project name — resolve it via px project get:

project_id=$(px project get "$PHOENIX_PROJECT" --format raw --no-progress | jq -r '.id')
encoded=$(python3 -c 'import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))' \
  "annotations['coding_session_id'].label == '$CODING_ANNOTATION_IDENTIFIER'")
echo "Phoenix UI: $PHOENIX_HOST/projects/$project_id/traces?filterCondition=$encoded"

If the user wants to discard everything this run produced (open-coding notes, axial-coding labels, and coding_session_id annotations on the server, plus the local sidecars), three identifier-bound deletes handle the server side and one rm handles the local sidecars. Confirm before running — destructive. Each px <entity>-annotations delete call requires --all to authorize the unbounded sweep; --identifier only narrows. Set PHOENIX_CLI_DANGEROUSLY_ENABLE_DELETES=true first if not already exported:

for kind in trace span session; do
  px "$kind-annotations" delete \
    --identifier "$CODING_ANNOTATION_IDENTIFIER" \
    --all -y \
    --format raw --no-progress
done
rm -f "$NOTES_SIDECAR" "$AXIAL_SIDECAR"

Each px <entity>-annotations delete call removes notes, axial-coding labels, and coding_session_id annotations together because they share the underlying annotation table; the rm clears the local sidecars.

Agent Failure Taxonomy

agent_failures:
  planning: [wrong_plan, incomplete_plan]
  tool_selection: [wrong_tool, missed_tool, unnecessary_call]
  tool_execution: [wrong_parameters, type_error]
  state_management: [lost_context, stuck_in_loop]
  error_recovery: [no_fallback, wrong_fallback]

Transition Matrix — jq sketch

To find where failures occur between agent states, identify the last non-error span before each first-error span within a trace. Note: OTel leaves most spans at status_code == "UNSET" and only sets "OK" when code explicitly does so — match != "ERROR" rather than == "OK" so the matrix works on typical OTel data.

px span list --format raw --no-progress | jq '
  group_by(.context.trace_id)
  | map(
      sort_by(.start_time)
      | { trace_id: .[0].context.trace_id,
          last_non_error: map(select(.status_code != "ERROR")) | last | .name,
          first_err:      map(select(.status_code == "ERROR")) | first | .name }
    )
  | [ .[] | select(.first_err != null) ]
  | group_by([.last_non_error, .first_err])
  | map({ transition: "\(.[0].last_non_error) → \(.[0].first_err)", count: length })
  | sort_by(-.count)
'

Use the output to tally which state-to-state transitions are most failure-prone and add them to your taxonomy.

What Makes a Good Category

A useful category is:

Named for the cause, not the symptom ("wrong_tool_selected", not "bad_output")
Tied to a fix — if you can't name a remediation, the category is too vague
Grounded in data — emerged from actual note text, not assumed upfront

Principles

One coding annotation identifier per run — every annotate call and every sidecar line carries $CODING_ANNOTATION_IDENTIFIER, the same value open coding used; never mint a new id mid-run.
Pass --identifier explicitly — every px call gets --identifier "$CODING_ANNOTATION_IDENTIFIER"; do not rely on inherited env vars.
Sidecar reads, server writes — Gather and Quantify read $NOTES_SIDECAR and $AXIAL_SIDECAR locally; Record writes to the server and updates the sidecar. If an entity appears more than once, the newest ts wins.
MECE — Each failure fits ONE category.
Actionable — Categories suggest fixes.
Bottom-up — Let categories emerge from data.
UI-filter annotation always paired — never write axial_coding_category without writing the matching coding_session_id annotation; the UI link depends on it.

Requirements and compatibility

Written for: GitHub Copilot

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key, Needs Node.js

The author says: Requires Node.js (for npx) or global install of @arizeai/phoenix-cli. Optionally requires jq for JSON processing.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

arize-ai

Publisher

GitHub (Official)

Distributed by

GitHub

Source

github/awesome-copilot · skills/phoenix-cli · commit 797337f

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by arize-ai or GitHub.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download phoenix-cli.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by arize-ai

Phoenix CLI

Invocation

Setup

Quick Reference

Workflows

Reference Categories

Auth

Profiles

Projects

Traces

Trace JSON shape

Spans

Span JSON shape

Sessions

Session JSON shape

Datasets / Experiments / Prompts

Annotation Configs

GraphQL

Docs

Files

references/axial-coding.md

Axial Coding

Coding annotation identifier (reuse the open-coding value)

Choosing the unit

Process

Example Taxonomy

Reading

1. Gather — read this run's open-coding notes from the sidecar

2. Group — synthesize categories

3. Record — write axial-coding labels

4. Quantify — count per category from the axial sidecar

Recording

UI-filter annotation

Recording discipline

Wrapping up

Agent Failure Taxonomy

Transition Matrix — jq sketch

What Makes a Good Category

Principles

Requirements and compatibility

Security

Attribution and licence

Related skills

Diagnosing Bugs

Systematic Debugging

Performance Optimization

Debugging And Error Recovery

Animation Performance

Parallel Debugging

Download ZIP