I4h Workflow Dataset Teleop

NVIDIAOfficialCreated by Isaac for Healthcare TeamDistributed by NVIDIA

Skill
Workflow
Automation and integrations
Apache-2.0
Medium risk

From the author

Record demonstrations through a workflow's teleop Task into workflow HDF5. Use for keyboard, leader, VR, or bus input; do not use for policy evaluation or autonomous rule-based Tasks.

— the description by Isaac for Healthcare Team

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured procedure for recording human-controlled teleoperation demonstrations within the Isaac for Healthcare (i4h) framework. It facilitates the capture of robot actions, state, camera feeds, and task outcomes into standardized HDF5 files. The process requires an active human operator using supported input devices such as keyboards, VR controllers, or bus inputs to perform tasks in a simulated environment.

What it does

Resolves workflow repositories and teleoperation device configurations.
Executes simulation environments for interactive human-led task performance.
Records demonstration data including actions, state, and camera streams into HDF5 format.
Provides validation steps to inspect recorded segments and action motion.
Supports custom run directories for integration into larger data pipelines.

When to use it

When you need to collect human-demonstration data for training or analysis.
When working with i4h-compatible workflows that support teleoperation modes.
When you have a human operator available to perform tasks via supported input devices.
When you need to ensure recorded data adheres to the standard HDF5 contract.

Advantages

Ensures consistent data structure across different teleoperation sessions.
Integrates directly with existing simulation runners and inspection tools.
Supports multiple input modalities including VR and keyboard.
Provides clear verification steps to confirm data integrity and task success.

Usage examples

“Record 5 keyboard teleop demonstrations for locomanip tray pick and place.”: The agent initiates the specified workflow in teleop mode using the keyboard device, prompts the operator to complete 5 successful episodes, and saves the data to an HDF5 file for verification.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/nvidia/i4h-workflow-dataset-teleop/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip i4h-workflow-dataset-teleop.zip -d .agents/skills

Documentation by Isaac for Healthcare Team

Record Teleop Demonstrations

Purpose

Run the workflow's declared teleop graph through the shared SimulationRunner so actions, state, cameras, segments, attempts, and outcomes use the normal HDF5 contract.

Instructions

Resolve the base checkout and live teleop device contract.
Choose a supported human-input device.
Record in the foreground through run.sh.
Inspect visible motion and HDF5 content.

Resolve support

export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
  [ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT"
./run.sh list
./run.sh show <workflow> --mode teleop

Treat the resolver above as part of the skill contract: a hosted copy may run outside the base repository, so never assume the current checkout contains workflows/i4h_workflows. I4H_WORKFLOWS_REPO_URL selects the clone source. When I4H_WORKFLOWS is unset, derive the fallback directory from that URL; set I4H_WORKFLOWS only to reuse or choose a specific destination. Never replace an existing checkout.

Require teleop in the live mode list. Read the workflow builder, its scene manifest teleop override, and the embodiment manifest's teleop_devices. Do not maintain a static support table in the skill.

Choose input

Use the named device when supported.
Otherwise use the workflow builder's default device.
Keep interactive keyboard/leader/VR/bus sessions visible and in the foreground. Surface the device controls and require the human operator to complete the task.
Never pretend to provide human input in an unattended shell.

Record

./run.sh <workflow> --teleop <device> \
  --episodes <N> --attempts 3 \
  --record

Omit <device> to use the workflow default. Bare --record writes demos.hdf5 inside the launcher's automatic run directory. Read the absolute directory from the ==> run dir ... line or run.json; do not recreate its timestamp in the shell. When a larger pipeline requires a caller-selected shared directory, pass --run-dir "$RUN_DIR" --record demos.hdf5; the launcher creates the directory and anchors the relative recording name inside it. An absolute --record path remains supported.

Verify

Require the final N/N episodes succeeded summary. Then inspect content:

RUN_DIR="<absolute run_dir from run.json or launcher output>"
uv run --project tools/dataset i4h-dataset inspect "$RUN_DIR/demos.hdf5" --segments
uv run --project tools/dataset i4h-dataset actions "$RUN_DIR/demos.hdf5"

Visually confirm that the operator completes the requested task, robot motion matches the input device, and all expected cameras record the same behavior. Treat zero saved episodes, missing observations, absent action motion, or an unsuccessful task outcome as failure. Stop leftovers with ./stop.sh all.

Troubleshooting

On device or width errors, compare the workflow teleop builder, Scene mode override, and embodiment devices.

Prerequisites

Require a workflow with teleop, a supported device, a working simulator, and a present operator for interactive input.

Limitations

Teleop records human input and requires an operator for interactive devices. Record autonomous rule-based Tasks with i4h-workflow-validate instead.

Examples

Record 5 keyboard teleop demonstrations for locomanip tray pick and place. → use G1's supported keyboard device, require a human operator, and verify the recorded action motion.

Completion gate

Report workflow, mode/device, controls, requested/saved episodes, attempts, visual result, HDF5 path, dimensions/segments, and whether a human operator completed the task.

Files

ATTRIBUTION.mdAdded by AutoGineer958 B
BENCHMARK.md7.0 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md4.6 KiB
evals/evals.json2.1 KiB
skill-card.md3.8 KiB
skill.oms.sig4.5 KiB

BENCHMARK.md

Skill Benchmark: i4h-workflow-dataset-teleop

✅ Overall verdict: PASS — Recommended for publication

Publication Recommendation

Recommended for publication based on the completed evaluation evidence in this report.

Evaluation Metadata

Skill: i4h-workflow-dataset-teleop
Evaluation date: 2026-09-21
Evaluator version: 1.5.6
Agents: Claude Code (aws/anthropic/bedrock-claude-opus-4-8), Codex (openai/openai/gpt-5.5)
Tasks: 3 evaluation tasks (3 positive)
Dataset digest: sha256:7e5d819eae66b55ccd3c97e712d134cb73d73d8e5a746f5c03faf7907c3d98c9 (skill-evaluator-dataset-snapshot/1)
Attempts per task: 3
Environment: k8s-sandbox
Tier 2 evidence: required for publication
Tier 3 evidence: required for publication

Each task attempt ran in its own isolated sandbox pod.

What This Report Answers

The three-tier evaluation checks whether the skill:

is safe to use;
produces correct answers;
is discovered and activated when needed;
helps the agent complete the user's goal and expected workflow; and
avoids wasted skill and tool usage.

Results at a Glance

Measure	Claude Code (Baseline → Skill Uplift)	Codex (Baseline → Skill Uplift)
Overall	66.0% — baseline ran, but no comparable score was available; uplift unavailable	71.8% — baseline ran, but no comparable score was available; uplift unavailable
Security	88.9% → 66.7% (-22.2 points)	71.4% → 83.3% (+11.9 points)
Correctness	4.4% → 66.7% (+62.3 points)	31.4% → 93.3% (+61.9 points)
Discoverability	91.7% — baseline ran, but no comparable score was available; uplift unavailable	75.0% — baseline ran, but no comparable score was available; uplift unavailable
Effectiveness	3.3% → 21.7% (+18.4 points)	10.9% → 36.3% (+25.4 points)
Efficiency	83.3% — baseline ran, but no comparable score was available; uplift unavailable	71.2% — baseline ran, but no comparable score was available; uplift unavailable

How to read this table: baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.

Example: 47.0% → 92.0% (+45.0 points) means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.

Token Usage

Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.

Agent	Dataset case	With skill	Without skill	Delta	Change	Coverage
claude-code	All cases	1,038,495	4,435,856	N/A	N/A	skill 3/3; base 9/9
claude-code	i4h-workflow-dataset-teleop-g1-keyboard-five	272,645	2,956,670	N/A	N/A	skill 1/1; base 3/3
claude-code	i4h-workflow-dataset-teleop-specialty-endoluminal-patient-twin	308,462	724,135	N/A	N/A	skill 1/1; base 3/3
claude-code	i4h-workflow-dataset-teleop-table-keyboard-five	457,388	755,051	N/A	N/A	skill 1/1; base 3/3
codex	All cases	3,869,512	2,533,623	N/A	N/A	skill 3/3; base 7/7
codex	i4h-workflow-dataset-teleop-g1-keyboard-five	1,327,813	114,717	+1,213,096	+1057.47%	skill 1/1; base 1/1
codex	i4h-workflow-dataset-teleop-specialty-endoluminal-patient-twin	878,644	1,000,629	N/A	N/A	skill 1/1; base 3/3
codex	i4h-workflow-dataset-teleop-table-keyboard-five	1,663,055	1,418,277	N/A	N/A	skill 1/1; base 3/3
ALL AGENTS	Dataset aggregate	4,908,007	6,969,479	N/A	N/A	skill 6/6; base 16/16

Prompt tokens include cached reads, so total tokens are prompt + completion (cached is not added twice). The Efficiency score uses (prompt - cached) + completion. N/A means the relevant trajectory counters were not available; coverage is never estimated.

Tier Status

Tier	Purpose	Status	Evidence
Tier 1	Static validation	PASSED WITH OBSERVATIONS	11 validator(s); 2 finding(s)
Tier 2	Semantic deduplication	PASSED	2 validator(s); 0 finding(s)
Tier 3	Live agent evaluation	PASS	2 agent(s); 3 task(s)

Findings and Observations

<details> <summary>Show detailed findings and successful checks</summary>

MEDIUM SECURITY/Unknown (SQP-2): The skill instructs the agent to automatically clone a remote git repository into the user's home directory (~/$I4H_REPO (SKILL.md:33)
LOW QUALITY/quality_reliability: Inputs are used but no dedicated Inputs section is documented (team-skills/holoscan/i4h-workflows/i4h-workflow-dataset-teleop/SKILL.md)

</details>

Scoring Methodology

<details> <summary>Show dimension definitions, source signals, and thresholds</summary>

Dimension	Question	Scored signals
Security	Is it safe to use?	`security` (100%)
Correctness	Is the answer correct?	`accuracy` (100%)
Discoverability	Was the right skill loaded when needed?	`skill_execution` (100%)
Effectiveness	Did the skill help complete the task?	`goal_accuracy` (50%) + `behavior_check` (50%)
Efficiency	Did it avoid wasted tool calls and token usage?	`skill_efficiency` (50%) + `token_efficiency` (50%)

Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
Effectiveness is the equal-weight mean of goal completion (goal_accuracy) and expected workflow adherence (behavior_check).
Efficiency is 50% tool-call productivity (the backward-compatible skill_efficiency wire id) and 50% token_efficiency. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.

Signals present in this run:

security (Security): unsafe operations, secret leakage, and unauthorized access.
skill_execution (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.
skill_efficiency (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).
accuracy (Accuracy): final-answer correctness against the reference answer.
goal_accuracy (Goal Accuracy): whether the user's goal was achieved.
behavior_check (Behavior Check): whether the expected workflow behavior was followed.
token_efficiency (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).

</details>

Freshness

Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Isaac for Healthcare Team

Publisher

NVIDIA (Official)

Distributed by

NVIDIA

Source

NVIDIA/skills · skills/i4h-workflow-dataset-teleop · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by Isaac for Healthcare Team or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download i4h-workflow-dataset-teleop.zip