AutoGineer
Generated banner for Tao Validate Dataset Format
Generated logo for Tao Validate Dataset Format

Tao Validate Dataset Format

NVIDIAOfficialCreated by NVIDIA Corporation

  • Skill
  • Review and audit
  • Software development
  • Apache-2.0
  • Medium risk
Download ZIP

From the author

Run `tao-daft validate` to check NVIDIA TAO DAFT datasets for structure, schema, and cross-reference errors. Do not use for non-DAFT formats. Use when the user asks to validate a DAFT dataset, check DAFT schema, validate a TAO dataset format, or run `tao-daft validate`.
— the description by NVIDIA Corporation

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides an interface to the tao-daft validate command, enabling users to verify the integrity of NVIDIA TAO DAFT datasets. It guides the user through discovering supported dataset formats, identifying the correct validation subcommands, and executing checks against local dataset paths. The skill ensures that validation is performed according to the specific requirements of the installed version of the nvidia-tao-daft package.

What it does

  • Checks dataset structure, schema, and cross-references for DAFT-formatted data.
  • Discovers available validation formats via the tao-daft CLI.
  • Supports validation of individual datasets or entire directory trees.
  • Provides guidance on identifying dataset formats based on directory markers.
  • Interprets validation results and failure codes from the CLI output.

When to use it

  • When you need to verify that a dataset conforms to the DAFT format requirements.
  • When you encounter schema or cross-reference errors in a TAO DAFT dataset.
  • When you need to confirm the validity of a dataset before running further TAO operations.
  • When you are unsure which validation flags are required for a specific DAFT format.

Advantages

  • Uses the official CLI as the source of truth for validation specifications.
  • Helps prevent errors by guiding the user through format-specific flag discovery.
  • Automates the process of checking complex dataset trees.
  • Provides clear feedback on validation success or failure.

Usage examples

“Validate my DAFT dataset located at /data/my_dataset.”
The agent will identify the dataset format, run the appropriate tao-daft validate command with the --path flag, and report the validation results.
“How do I check the schema for a metropolis-v3.0 dataset?”
The agent will list the available flags for the metropolis-v3.0 format and execute the validation command on the specified path.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/nvidia/tao-validate-dataset-format/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip tao-validate-dataset-format.zip -d .agents/skills

Documentation by NVIDIA Corporation

Validate a TAO DAFT Dataset

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

Quick start

tao-daft validate <format> --path <dataset-or-parent-dir>

<format> is a positional subcommand (e.g. metropolis-v3.0, cosmos-reason-v1.0); --path is required. Discover supported formats and per-format flags via tao-daft validate --help and the leaf --help (see "CLI conventions" below).

Preflight

python -c "import nvidia_tao_daft" 2>/dev/null || {
  echo "MISSING: tao-daft not installed. Run:"
  echo "  pip install nvidia-tao-daft"
  exit 1
}

Quick Start

Discover the installed validator formats before choosing a format slug, then run validation with the target passed through --path:

tao-daft --version
tao-daft validate --help
tao-daft validate <format> --help
tao-daft validate <format> --path /path/to/daft-dataset

Purpose

Drive tao-daft validate against a DAFT dataset (or a tree of them). The CLI is the spec; the skill picks subcommand + flags and explains the result.

Trigger when the user mentions "TAO DAFT", "DAFT format", validating a DAFT dataset, schema/cross-reference errors, or tao-daft validate. Do not trigger for non-DAFT layouts (COCO, YOLO, Data Factory JSONL), or for tao-daft info / tao-daft convert — those have their own skills.

If the user's opening is ambiguous, run a few --help commands first to ground yourself, then come back and confirm the task.

Prerequisites

  • nvidia-tao-daft installed (pip install nvidia-tao-daft; the wheel is enough, no source repo). Confirm with tao-daft --version.

  • A DAFT dataset, or a parent directory of them, on local disk.

Instructions

CLI conventions

tao-daft is nested argparse subcommands. Names and flags drift across versions, so discover the current surface from --help rather than trusting any list in this doc.

  1. Format is a positional subcommand, not --format: tao-daft validate <format> [flags]. List current formats via tao-daft validate --help; slugs look like metropolis-v3.0, cosmos-reason-v1.0.

  2. Target is --path PATH, not positional. It accepts a single dataset/scene or a parent directory — the validator walks the tree.

  3. Flags are per-format; run the leaf help, e.g. tao-daft validate metropolis-v3.0 --help, before choosing them. Don't assume a flag from one format exists on another.

So the loop is: tao-daft --version → tao-daft validate --help → pick format (infer if unspecified, see below) → tao-daft validate <format> --help → run → interpret.

Format inference

Use directory markers, not filenames:

  • meta.json next to media/ and text/ ⇒ cosmos-reason-v1.0.

  • A directory (or nested directories) containing contextual/, typically alongside raw/ and task/ ⇒ metropolis-v3.0.

  • Neither marker present ⇒ ask the user; do not guess.

Reading errors

The CLI ends every run with a VALIDATION RESULTS block, then ✅ VALIDATION PASSED or ❌ VALIDATION FAILED, and exits non-zero on failure (safe to chain in scripts).

Output can be large on big trees — capture the full output to a file and read it in slices rather than scrolling inline.

Limitations

  • Validates DAFT only. Non-DAFT layouts (COCO, YOLO, Data Factory JSONL, etc.) belong in the upstream converter skills.

  • Supported formats are whatever tao-daft validate --help reports for the installed version; older slugs may have been retired.

  • Covers validate only. Defer to the dedicated skills for tao-daft info and tao-daft convert.

  • Don't reimplement validation in Python; the CLI is the spec.

Troubleshooting

  • tao-daft: command not found — wheel not installed in the active env. pip install nvidia-tao-daft; verify tao-daft --version.

  • error: argument --path is required — path passed positionally. Move it behind --path.

  • invalid choice: '<format>' — slug isn't wired up in this version. Re-run tao-daft validate --help and pick from the list.

  • Auto-detection (raw type / contextual set) is wrong — override via the format's scope-restriction flag; discover the name from the leaf --help.

  • CI wants warnings to fail — add --strict.

Files

BENCHMARK.md

Skill Benchmark: tao-validate-dataset-format

✅ Overall verdict: PASS — Recommended for publication

Publication Recommendation

Recommended for publication based on the completed evaluation evidence in this report.

Evaluation Metadata

  • Skill: tao-validate-dataset-format

  • Evaluation date: 2026-09-17

  • Evaluator version: 1.5.6

  • Agents: Claude Code (aws/anthropic/bedrock-claude-opus-4-8), Codex (openai/openai/gpt-5.5)

  • Tasks: 1 evaluation tasks (1 positive)

  • Dataset digest: sha256:a10c158d485aae405af8bf9fa4683b3c863dd7573f2c4be03d871bd0797abf39 (skill-evaluator-dataset-snapshot/1)

  • Attempts per task: 3

  • Environment: k8s-sandbox

  • Tier 2 evidence: required for publication

  • Tier 3 evidence: required for publication

Each task attempt ran in its own isolated sandbox pod.

What This Report Answers

The three-tier evaluation checks whether the skill:

  • is safe to use;

  • produces correct answers;

  • is discovered and activated when needed;

  • helps the agent complete the user's goal and expected workflow; and

  • avoids wasted skill and tool usage.

Results at a Glance

MeasureClaude Code (Baseline → Skill Uplift)Codex (Baseline → Skill Uplift)
Overall94.6% — baseline ran, but no comparable score was available; uplift unavailable62.4% — baseline ran, but no comparable score was available; uplift unavailable
Security100.0% → 100.0% (±0.0 points)100.0% → 100.0% (±0.0 points)
Correctness0.0% → 100.0% (+100.0 points)0.0% → 60.0% (+60.0 points)
Discoverability100.0% — baseline ran, but no comparable score was available; uplift unavailable0.0% — baseline ran, but no comparable score was available; uplift unavailable
Effectiveness5.6% → 100.0% (+94.4 points)16.7% → 53.3% (+36.6 points)
Efficiency72.8% — baseline ran, but no comparable score was available; uplift unavailable99.7% → 98.6% (-1.1 points)

How to read this table: baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.

Example: 47.0% → 92.0% (+45.0 points) means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.

A partial dimension was calculated from only the available configured signals; review the detailed report before relying on it.

Token Usage

Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.

AgentDataset caseWith skillWithout skillDeltaChangeCoverage
claude-codeAll cases98,229826,994N/AN/Askill 1/1; base 3/3
claude-codetao-validate-dataset-format-basic98,229826,994N/AN/Askill 1/1; base 3/3
codexAll cases13,71840,651N/AN/Askill 1/1; base 3/3
codextao-validate-dataset-format-basic13,71840,651N/AN/Askill 1/1; base 3/3
ALL AGENTSDataset aggregate111,947867,645N/AN/Askill 2/2; base 6/6

Prompt tokens include cached reads, so total tokens are prompt + completion (cached is not added twice). The Efficiency score uses (prompt - cached) + completion. N/A means the relevant trajectory counters were not available; coverage is never estimated.

Tier Status

TierPurposeStatusEvidence
Tier 1Static validationPASSED WITH OBSERVATIONS11 validator(s); 7 finding(s)
Tier 2Semantic deduplicationPASSED WITH OBSERVATIONS2 validator(s); 1 finding(s)
Tier 3Live agent evaluationPASS2 agent(s); 1 task(s)

Findings and Observations

<details> <summary>Show detailed findings and successful checks</summary>

  • HIGH DUPLICATE/duplicate: Duplicate content found within SKILL.md: "# Validate a TAO DAFT Dataset" in SKILL.md (lines 1-4) vs "## Quick start" in SKILL.md (lines 5-14) vs "## Quick Start" in SKILL.md (lines 24-35) vs "## Purpose" in SKILL.md (lines 36-49) vs "### CLI conventions" in SKILL.md (lines 58-77) vs "## Limitations" in SKILL.md (lines 96-105) (SKILL.md:1)

  • MEDIUM QUALITY/quality_correctness: SKILL_SPEC recommended field missing: 'metadata.tags' (skills/data/tao-validate-dataset-format/SKILL.md)

  • MEDIUM SCHEMA/frontmatter_field_placement: Root field 'tags' is ignored; use 'metadata.tags' (skills/data/tao-validate-dataset-format/SKILL.md)

  • MEDIUM SCHEMA/folder_hierarchy: Unexpected nesting depth for general skill (skills/data/tao-validate-dataset-format)

  • MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (skills/data/tao-validate-dataset-format/SKILL.md)

  • 3 additional finding(s) are available in the full evaluation artifacts.

</details>

Scoring Methodology

<details> <summary>Show dimension definitions, source signals, and thresholds</summary>

DimensionQuestionScored signals
SecurityIs it safe to use?security (100%)
CorrectnessIs the answer correct?accuracy (100%)
DiscoverabilityWas the right skill loaded when needed?skill_execution (100%)
EffectivenessDid the skill help complete the task?goal_accuracy (50%) + behavior_check (50%)
EfficiencyDid it avoid wasted tool calls and token usage?skill_efficiency (50%) + token_efficiency (50%)

  • Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.

  • Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.

  • Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.

  • The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.

  • Effectiveness is the equal-weight mean of goal completion (goal_accuracy) and expected workflow adherence (behavior_check).

  • Efficiency is 50% tool-call productivity (the backward-compatible skill_efficiency wire id) and 50% token_efficiency. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.

Signals present in this run:

  • security (Security): unsafe operations, secret leakage, and unauthorized access.

  • skill_execution (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.

  • skill_efficiency (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).

  • accuracy (Accuracy): final-answer correctness against the reference answer.

  • goal_accuracy (Goal Accuracy): whether the user's goal was achieved.

  • behavior_check (Behavior Check): whether the expected workflow behavior was followed.

  • token_efficiency (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).

</details>

Freshness

Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Declares allowed tools, Needs Python

The author says: Requires Python 3.10+ and the nvidia-tao-daft package (pip install nvidia-tao-daft).

Security

What our scanner noticed· Medium risk

  • Large base64-encoded blobs
  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
NVIDIA Corporation
Publisher
NVIDIA (Official)
Source
NVIDIA/skills · skills/tao-validate-dataset-format · commit fce3cca
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE-APACHE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by NVIDIA Corporation or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download tao-validate-dataset-format.zip