Physical AI Neural Reconstruction

NVIDIAOfficialCreated by NVIDIA Physical AI

Skill
Meta (authoring, orchestration, planning, memory)
Agent tooling
Apache-2.0
Medium risk

From the author

Router for NVIDIA NuRec/NRE: USDZ rendering, NCore conversion, 3DGS, gRPC sensor sim, carline adaptation, PhysicalAI HF datasets. Do NOT use for SimReady or infra setup.

— the description by NVIDIA Physical AI

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Physical AI Neural Reconstruction (NuRec) Router acts as a central navigation hub for NVIDIA's suite of neural reconstruction tools. It does not perform reconstruction tasks itself but identifies the correct sibling skill to handle specific stages of the pipeline, such as data conversion, training, rendering, asset harvesting, or frame harmonization. By providing a structured taxonomy and workflow ordering, it ensures users follow the correct sequence of operations—such as converting raw sensor data to NCore V4 before training or using the Asset Harvester before packaging objects into a USDZ.

What it does

Identifies the appropriate sibling skill for specific NuRec-related tasks.
Provides a structured workflow guide for multi-stage reconstruction pipelines.
Maps user requests to canonical product repositories for fetching required skills.
Maintains a registry of sibling skills including datasets, conversion tools, and rendering engines.
Enforces best practices for secret management and gated asset access.
Prevents duplication of logic by routing tasks to specialized upstream repositories.

When to use it

When you need to determine which NVIDIA NuRec skill to use for a specific task.
When starting a new neural reconstruction project and needing to understand the pipeline order.
When you need to find or download published NVIDIA Physical AI datasets.
When you are unsure whether to use the built-in NRE fixer or the standalone DiffusionHarmonizer.
When you need to coordinate complex workflows involving multiple sibling skills.

Advantages

Centralizes navigation for a complex, multi-repo ecosystem.
Ensures consistent workflow ordering across different reconstruction tasks.
Reduces user error by clearly defining the scope of each sibling skill.
Provides a single point of reference for troubleshooting and setup prerequisites.
Maintains clear boundaries between different pipelines like NuRec and SimReady.

Usage examples

“I want to convert my LiDAR recording into NCore V4 format, what should I do?”: The router identifies the ncore-data-conversion skill as the correct tool and provides the path to its repository.
“How do I train a 3D reconstruction from an NCore clip?”: The router directs you to use ncore-data-conversion first, followed by the nurec skill for training.
“Where can I find NVIDIA's published NuRec datasets?”: The router points you to the physical-ai-datasets skill for cataloging and downloading gated datasets.
“I need to clean up rendered frames with ghosting issues, which skill handles this?”: The router explains the difference between the built-in NRE fixer and the standalone harmonizer skill, guiding you to the latter for post-rendering cleanup.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/nvidia/physical-ai-neural-reconstruction/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip physical-ai-neural-reconstruction.zip -d .agents/skills

Documentation by NVIDIA Physical AI

Physical AI Neural Reconstruction (NuRec) Router

Purpose

This is a thin router for NVIDIA Neural Reconstruction (NuRec) requests. It points at sibling skills in their product repos (physical-ai-datasets, ncore-data-conversion, nurec, asset-harvester, harmonizer). Use this skill to:

Identify which sibling skill answers a NuRec question.
Name each sibling's product repo and path so it can be fetched with consent if it is not already installed.
Order multi-step NuRec workflows (data → conversion → train → render → cleanup) before opening the sibling recipe.

The canonical recipes (training, rendering, data conversion, dataset downloads, object harvesting, frame cleanup) live in the upstream sibling skills. Never copy or reconstruct their commands here.

Do NOT use this skill for:

SimReady packaging of CAD or source meshes → use omniverse-cad-to-simready.
Generic USD performance tuning unrelated to NuRec → use omniverse-usd-performance-tuning.
AKS / OSMO / NIM Operator infrastructure setup → use physical-ai-infrastructure-setup-and-resilient-scaling.

When to Use

Read this skill first whenever a user mentions any of:

nurec, nurec router, nurec index, neural reconstruction, neural reconstruction engine, NRE, 3DGUT, 3DGRT, USDZ, NCore V4, sensorsim, sensor sim, novel view synthesis, PhysicalAI-Autonomous-Vehicles-NuRec, PhysicalAI-Robotics-NuRec, PhysicalAI-NuRec-PPISP, Cosmos-Drive-Dreams, asset harvester, nurec fixer, DiffusionHarmonizer, harmonizer, difix, difix3d, carline adaptation, serve-grpc, render-grpc, warm serve-grpc, nre thin client, batch_render_rgb, nurec teardown, "where do I start with NuRec", "which NuRec skill should I use for X?".

Decide which upstream sibling skill answers the question, fetch it (see Locate and fetch the upstream skills), then follow that skill's body.

Prerequisites

The router itself has no runtime prerequisites beyond git for fetching the upstream. Downstream sibling skills need Linux x86_64, an NVIDIA GPU (Ampere+, CUDA 12.8, >= 24 GB VRAM), Docker plus the NVIDIA Container Toolkit, an NGC API key, a Hugging Face token with the relevant gated licenses already accepted, and Python 3.10+.

Full per-skill detail — driver floors, container names, key resolution order, which Hugging Face assets are gated, and how to verify secrets without echoing them — is in references/prerequisites.md. Prefer each sibling's scripts/validate_setup.py over hand-written checks.

What is NuRec?

NuRec (NVIDIA Omniverse Neural Reconstruction) turns camera, LiDAR, radar, or stereo recordings — typically from a self-driving car or a robot — into a 3D scene that can be re-rendered from any viewpoint. A typical project runs in three stages: get the input (convert a recording with ncore-data-conversion, or download a ready-made dataset with physical-ai-datasets), train the reconstruction (nurec, which emits a USDZ), then render new views (nurec). Projects that only want to use a scene NVIDIA already published skip the training stage.

Background on the vocabulary — NRE vs NuRec, USDZ, NCore V4, 3DGUT / 3DGRT — is in references/what-is-nurec.md.

Pick a skill

Match the user's goal in the left column and open the named upstream skill on the right. Arrows mean "do these in order".

I want to…	Upstream skill
Find or download a NuRec dataset NVIDIA has published	`physical-ai-datasets`
Convert my own camera / LiDAR / radar / depth / stereo recording into NCore V4	`ncore-data-conversion`
Write a new converter for an unsupported sensor setup (drone, RGB-D, ROS 2 bag, COLMAP, ScanNet++)	`ncore-data-conversion`
Train a 3D reconstruction from an NCore clip	`ncore-data-conversion` → `nurec`
Generate the extra inputs NRE needs (segmentation masks, depth, ego mask, DINOv2, LiDAR-seg visibility)	`nurec` (uses the `nre-tools-ga` container)
Render a USDZ along the original camera positions	`nurec`
Render at full resolution / highest quality	`nurec` (see "Quality presets")
Render along a shifted trajectory (e.g. car moved 3 m left)	`nurec`
Adapt an existing USDZ to an augmented target-vehicle rig (carline adaptation)	`nurec` (`export-custom-rig-trajectory` → `render`) → `harmonizer`
Render through a server so CARLA / Isaac Sim / AlpaSim / a custom simulator can ask for frames	`nurec` (`serve-grpc`)
Render the same USDZ many times back-to-back from Python with minimal per-call latency	`nurec` (warm `serve-grpc` + thin Python client / `batch_render_rgb`)
Render LiDAR sweeps (point clouds) from a USDZ	`nurec` (`render-grpc --lidar`)
Skip training and just render a NuRec scene NVIDIA already built	`physical-ai-datasets` → `nurec`
Skip training and use a pre-built indoor robotics scene	`physical-ai-datasets` → `nurec` (then Isaac Sim 5.1)
Extract individual 3D objects (cars, pedestrians) from a driving clip	`asset-harvester`
Add, remove, or replace cars / pedestrians in a NuRec scene	`asset-harvester` → `nurec`
Clean up or harmonize rendered frames (ghosting, floaters, flicker, lighting/shadows)	`harmonizer`, or `--enable-difix` inside `nre` for inline rendering
Export the scene as a PLY, mesh, depth maps, ego mask, etc.	`nurec`
Upgrade an old USDZ so newer NRE versions load it faster	`nurec` (`upgrade-artifact`)
Open a USDZ or PLY in a browser viewer	`nurec` (`viewer` / `ply_viewer`)
Measure rendering quality (PSNR, SSIM, LPIPS) against ground truth	`nurec` (`eval-rendering-metrics`)
Benchmark different reconstruction methods on the same scenes	`physical-ai-datasets` (`PhysicalAI-NuRec-PPISP`) → `nurec`
Train on multiple GPUs or on SLURM	`nurec`

Common workflows

Seven end-to-end workflows are documented in references/workflows.md, lettered A–G by this router:

A. Make a NuRec scene from your own recording.
B. Use a NuRec scene NVIDIA has already trained.
C. Use NuRec for indoor robot simulation.
D. Add, remove, or replace 3D objects in a scene.
E. Clean up rendered frames.
F. Benchmark reconstruction quality.
G. Connect NuRec to a simulator.

Open that file when the user's task spans more than one sibling skill.

Sibling skills (upstream)

Refer to a sibling by its name — that is the portable identifier. The skill-source column is the product repo and path of that skill.

Name	Skill source	What it does
`physical-ai-datasets`	`NVIDIA/nurec` → `skills/physical-ai-datasets/`	Catalog and download recipes for every NVIDIA Physical AI dataset on Hugging Face (driving, robotics, manipulation, NuRec scenes, benchmarks).
`ncore-data-conversion`	`NVIDIA/ncore` → `skills/ncore-data-conversion/`	Converts any sensor recording to NCore V4 (the format NRE needs). Also covers writing a new converter.
`nurec`	`NVIDIA/nurec` → `skills/nurec/`	The Neural Reconstruction Engine itself (`nvcr.io/nvidia/nre/nre-ga`, `nvcr.io/nvidia/nre/nre-tools-ga`, NRE 26.04 — image tags `26.04.01` / `26.04` / `latest`). Trains, performs carline adaptation, renders (locally, via warm `serve-grpc` + thin Python client / `batch_render_rgb`, or to an external simulator), exports meshes / point clouds / depth, edits actors, evaluates quality.
`asset-harvester`	`NVIDIA/asset-harvester` → `skills/asset-harvester/`	Open-source Apache-2.0 pipeline (SparseViewDiT + TokenGS) that extracts individual 3D objects from sparse views in a driving clip and saves them as `.ply` Gaussian splats, optionally emitting `metadata.yaml` for the NuRec handoff.
`harmonizer`	`NVIDIA/harmonizer` → `skills/harmonizer/`	Standalone NVIDIA DiffusionHarmonizer workflow — public successor to the older Fixer / Difix3D+ recipes — that cleans rendered frames, harmonizes inserted actors, evaluates PSNR/LPIPS, and optionally fine-tunes the model.

For naming overlaps (NRE vs Fixer, ncore-data-conversion vs nurec, AV-NuRec vs Cosmos-Drive-Dreams, NuRec vs SimReady) see references/mix-ups.md.

Locate and fetch the upstream skills

Select a sibling by its current registered name and the repo/path in the sibling table. Full lookup, consent, and stale-copy rules are in references/upstream-fetch.md.

For ncore-data-conversion, nurec, and harmonizer, try the runtime-local lookup first:

.agents/skills/<name>/SKILL.md (Cursor, Codex, NemoClaw)
.claude/skills/<name>/SKILL.md (Claude Code)
.cursor/skills/<name>/SKILL.md (project-scoped)
~/.cursor/skills/<name>/SKILL.md (personal skills)

A matching installed definition can be read locally. If it is a symlink, follow it and use that definition's own companion files.

For physical-ai-datasets and asset-harvester, generic same-name discovery cannot distinguish the old bundled copy. Use their canonical product-repo source instead. A canonical definition already acquired for this task can be reused.

Do not read sibling skills from a nurec-skills checkout: those copies are retired and may still exist. Use each skill's product-repo home. In particular, physical-ai-datasets and asset-harvester kept their names, so a generic same-name local hit may be the old copy. Renamed skills may resolve locally under their current names; never fall back to the retired ncore, nre or nurec-fixer skill folders.

If the selected current definition is unavailable, follow Ask before cloning before obtaining it from the mapped product repo. Missing a renamed local skill never falls back to an old folder or clone. If retrieval is declined, report the missing skill and stop that handoff.

Read the selected definition before following its workflow. Open its own references on demand. Companion files (references/, scripts/, assets/) ship inside the sibling's own skill directory. This catalog router owns its picker and workflow letters; there is no upstream index to fetch.

Hard Rules

Router only — do not duplicate upstream NuRec recipes here. Read the upstream sibling skill body before running any mutating command.
Refer to sibling skills by their name: (e.g. nurec), not by repo path. Folder layouts can change; the name is portable.
Never git clone a product repo without explicit user consent. Exhaust the local lookup order first (and the same-name exception for physical-ai-datasets and asset-harvester), show the exact command, and clone only into a path the user agreed to — never silently into /tmp. Do not scan broad developer workspaces such as ~/Codes or reuse unrelated old clones. Never fall back to a nurec-skills checkout.
Use the GA container channel: nvcr.io/nvidia/nre/nre-ga and nvcr.io/nvidia/nre/nre-tools-ga. The un-suffixed nvcr.io/nvidia/nre/nre / nre-tools names are the legacy channel — still valid for cached version pins, but not what a new workflow should pull.
Resolve the NGC key as ${NGC_CLI_API_KEY:-${NGC_API_KEY:-}} and log in with docker login nvcr.io --username '$oauthtoken' --password-stdin. Never echo a key.
physical-ai-datasets covers gated Hugging Face datasets. Do not bypass dataset license terms; the user must accept the PhysicalAI-* gated licenses on Hugging Face and provide a token before downloading.
Asset Harvester runs before packaging into a USDZ. Do not call nre's export-external-assets on hand-rolled .ply files unless the user explicitly asks to skip Asset Harvester. The owning skill for that command is nurec.
For artifact cleanup, prefer the built-in --enable-difix path in nre. Route to the standalone harmonizer only when the user needs the public code/model card, paired evaluation, fine-tuning, or fixes on previously rendered frames.
Do not invent NRE / NCore / DiffusionHarmonizer commands from memory. Re-read the upstream sibling skill — versions move fast (NRE 26.04 — pull nvcr.io/nvidia/nre/nre-ga:26.04.01 or :26.04; the release name release_26.04 is not a valid image tag).
This router does not deploy infrastructure. Route AKS / OSMO / NIM Operator setup to physical-ai-infrastructure-setup-and-resilient-scaling.

Limitations

Router only. This skill never executes mutating NuRec commands. All training, rendering, conversion, and harmonization happens in upstream sibling skills.
Upstream-pinned. Sibling recipes live in their product repos: https://github.com/NVIDIA/nurec, https://github.com/NVIDIA/ncore, https://github.com/NVIDIA/harmonizer, and https://github.com/NVIDIA/asset-harvester. A renamed skill already installed locally may be used without a network refresh; if a fetch is needed, follow Ask before cloning.
Hand-curated catalogue. A newly-added upstream sibling is not discoverable here until someone edits the tables (see references/maintenance.md).
Gated content. nvidia/PhysicalAI-*, nvidia/Harmonizer, and nvidia/Cosmos-Predict2-0.6B-Text2Image require the user to accept license terms on Hugging Face first. For asset-harvester only its optional DINOv3, Llama Guard and SAM 3D Body models are gated. The router cannot bypass this.
Heavy footprint. A complete NuRec workflow can leave 150 GB+ on disk. See references/teardown.md.
NVIDIA-only stack. Requires Linux x86_64 plus an NVIDIA GPU and the NVIDIA Container Toolkit. aarch64 / AMD / Intel / Apple Silicon are not supported.
No Omniverse / Isaac Sim integration steps. Handing a USDZ to Isaac Sim 5.1 (workflow C) is documented in the Isaac Sim docs, not in the NuRec skill family.
Not a SimReady pipeline. NuRec produces a renderable USDZ from a recording; SimReady packaging of CAD or source meshes is a different pipeline (see omniverse-cad-to-simready).

Troubleshooting

Routing-level symptoms — a missing upstream clone, gated-asset 403s, NGC login failures, manifest unknown on an NRE image, stale cached skill names — are tabulated in the troubleshooting companion file that ships alongside this skill. Symptoms specific to a sibling's own commands belong to that sibling's skill.

Cross-skill teardown

A complete NuRec workflow can leave 150 GB+ on disk between container images, model weights, code clones, conda envs, and output directories. Each sibling skill has its own dedicated Teardown section — read them in the order documented in references/teardown.md when the user no longer needs the workflow. Do not revoke NGC_API_KEY / HF_TOKEN as part of teardown unless they were leaked.

Keeping this router up to date

Procedure for adding new sibling skills, renames, or upstream URL changes lives in references/maintenance.md. This catalog router is authoritative for the routing taxonomy and workflow ordering. Sibling recipes live in their product repos; it is not authoritative for asset-harvester, which is maintained in https://github.com/NVIDIA/asset-harvester.

Files

ATTRIBUTION.mdAdded by AutoGineer969 B
BENCHMARK.md10.4 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md17.5 KiB
evals/evals.json4.9 KiB
references/maintenance.md2.5 KiB
references/mix-ups.md2.6 KiB
references/prerequisites.md2.5 KiB
references/secrets-handling.md1.7 KiB
references/teardown.md2.2 KiB
references/troubleshooting.md3.0 KiB
references/upstream-fetch.md3.2 KiB
references/what-is-nurec.md1.3 KiB
references/workflows.md5.7 KiB
skill-card.md4.7 KiB
skill.oms.sig6.6 KiB

references/secrets-handling.md

Secrets Handling Across Sibling Skills

Every sibling skill ships a Verifying secrets safely block in its Prerequisites section. Always verify prerequisites by running scripts/validate_setup.py (where it exists) or, for skills without one (ncore-data-conversion, physical-ai-datasets, the router), use hf auth whoami or a length-only shell check. Never write ad-hoc bash that interpolates secret values.

In particular, do not use the bash anti-pattern:

echo "HF_TOKEN: ${HF_TOKEN:+yes}${HF_TOKEN:-no}"

This prints yes<token-value> because ${VAR:-no} only falls back to no when the variable is empty. If you suspect a token was echoed, rotate it (huggingface.co/settings/tokens, org.ngc.nvidia.com/setup/api-key) before continuing.

Safe verification patterns

# Hugging Face token
hf auth whoami

# Length-only check (does not echo the value)
[ -n "${HF_TOKEN:-}" ] && echo "HF_TOKEN length=${#HF_TOKEN}" || echo "HF_TOKEN unset"
[ -n "${NGC_CLI_API_KEY:-}" ] && echo "NGC_CLI_API_KEY length=${#NGC_CLI_API_KEY}" || echo "NGC_CLI_API_KEY unset"
[ -n "${NGC_API_KEY:-}" ] && echo "NGC_API_KEY length=${#NGC_API_KEY}" || echo "NGC_API_KEY unset"

NGC key resolution

NRE and the fixer resolve the NGC key in this order — do not prompt the user until both are exhausted:

$NGC_CLI_API_KEY (primary; many CI runners and cloud images export it automatically)
$NGC_API_KEY (fallback)
Ask the user, pointing them at https://org.ngc.nvidia.com/setup/api-key

NGC_KEY="${NGC_CLI_API_KEY:-${NGC_API_KEY:-}}"
printf '%s' "$NGC_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Always use --password-stdin so the key never lands in process tables or shell history.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key

The author says: Router skill; downstream sibling skills require Linux x86_64, an NVIDIA GPU (Ampere+, CUDA 12.8, >= 24 GB VRAM), Docker >= 23.0.1, NVIDIA Container Toolkit >= 1.13.5, an NGC API key, a Hugging Face token with the relevant gated licenses accepted, Python 3.10+, and `huggingface_hub`. Optional: CARLA / Isaac Sim 5.1 / AlpaSim for simulator integration over `serve-grpc`.

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA Physical AI

Publisher

NVIDIA (Official)

Source

NVIDIA/skills · skills/physical-ai-neural-reconstruction · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by NVIDIA Physical AI or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download physical-ai-neural-reconstruction.zip