Omniverse Realtime Viewer

OpenAIOfficialCreated by NVIDIA OmniverseDistributed by OpenAI

Skill
Meta (authoring, orchestration, planning, memory)
Software development
Apache-2.0
Medium risk

Download ZIP

From the author

Use as the top-level router for Omniverse Realtime Viewer USD app requests and focused viewer reference documents.

— the description by NVIDIA Omniverse

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Omniverse Realtime Viewer skill acts as a central orchestrator for developing USD-based viewer applications. It provides a structured approach to selecting implementation recipes, enforcing architectural conventions, and managing dependencies. By utilizing a set of focused reference documents, it guides developers through the lifecycle of building, configuring, and validating viewer applications, ensuring consistent behavior across different delivery paths such as local workstations or streaming environments.

What it does

Routes development requests to specific implementation recipes based on the target delivery path.
Enforces architectural rules for USD rendering, stage management, and UI integration.
Provides guidance on camera selection, input handling, and viewport interaction.
Standardizes the integration of AOV switching and render product configuration.
Facilitates the capture of validation evidence for viewer readiness.
Maintains separation between user-provided USD files and application-specific session state.

When to use it

When initiating the development of a new USD viewer application within the Omniverse ecosystem.
When determining the appropriate delivery path for a viewer, such as local native or streaming.
When needing to implement consistent camera, selection, or UI behavior across viewer projects.
When integrating advanced rendering features like AOV switching into an existing viewer.

Advantages

Ensures architectural consistency across diverse viewer implementations.
Provides a clear, modular path for adding specific capabilities like input routing or data display.
Promotes clean separation of concerns between stage data and viewer-owned state.
Standardizes the validation process for consistent delivery.

Usage examples

“I need to build a browser-based viewer for USD files.”: The agent will route you to the streaming viewer recipe, providing guidance on WebRTC stream setup, UI integration, and viewport interaction.
“How do I implement AOV switching in my local USD viewer?”: The agent will provide the AOV switching reference, detailing how to map render vars and update the display state on the server.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/openai/omniverse-realtime-viewer/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip omniverse-realtime-viewer.zip -d .agents/skills

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

This is the top-level entry point for the Omniverse Realtime Viewer skill package. It is self-contained: all required routing, conventions, and validation guidance live in the selected references.

Use the focused reference documents as implementation recipes. This file chooses the right recipes and preserves the architectural rules that must hold across all generated viewer apps.

Instructions

Start by classifying the requested viewer, then read only the references needed for that delivery path and feature set. Implement the render path first, layer interaction and UI behavior on top of it, and finish by capturing validation evidence from references/validation.md.

Read Order

Read references/routing.md to choose the delivery path and focused references.
Read references/conventions.md before implementing camera, input, selection, viewport, streaming protocol, scene loading, or environment behavior.
For broad viewer requests, read references/usd-viewer-app/README.md.
If the delivery path is unclear, read references/streaming-vs-local/README.md.
If the prompt includes layout, panels, controls, inspectors, status, or UX, read references/viewer-ux-workflow/README.md and then the focused viewer UI references. This applies to React/WebRTC, Tauri, Electron, ovui, ovwidgets, and Dear ImGui apps; "frontend" means user-facing UI, not only browser UI.
For viewport interaction, read references/viewer-input-routing/README.md before references/camera-controls/README.md, references/native-picking-selection/README.md, or references/object-selection/README.md.
Read only the focused capability references needed for the requested app.
Use references/validation.md to capture review evidence before handoff.

Non-Negotiables

Use ovrtx for all USD and 3D rendering.
Browser apps display an ovstream WebRTC video stream plus UI. The browser does not render USD geometry.
Do not substitute WebGL, Three.js, Babylon.js, PlayCanvas, A-Frame, model-viewer, react-three-fiber, glTF browser viewers, or other client-side 3D renderers.
If local validation cannot run because the GPU/runtime environment is absent, scaffold the ovrtx path and document the runtime requirement. Do not add a browser-renderer fallback.
Keep user USD files unmodified. Viewer cameras, render products, render vars, settings, selection metadata, and runtime state belong in session/composite layers or app state.
Keep one owner for renderer.step(), stage mutation, native picking, selection writes, and live attribute writes.
Keep dependency acquisition in references/dependencies/README.md and deployment choices in references/cloud-deployment/README.md; do not duplicate package locations or deployment setup.

Focused Reference Families

Entry points and recipes: references/usd-viewer-app/README.md, references/streaming-viewer-recipe/README.md, references/ovui-local-viewer-recipe/README.md, references/streaming-vs-local/README.md, references/electron-shm-viewer/README.md, references/ovwidgets-editor-shell/README.md.
Rendering and stage: references/ovrtx-rendering/README.md, references/stage-loading/README.md, references/stage-management/README.md, references/render-settings/README.md, references/aov-switching/README.md, references/stage-hierarchy/README.md, references/stage-queries/README.md, references/stage-attribute-reads/README.md, references/prim-transform-safety/README.md, references/usd-sample-data/README.md.
Delivery and runtime: references/streaming-server/README.md, references/streaming-client/README.md, references/streaming-messages/README.md, references/streaming-lifecycle/README.md, references/local-viewer/README.md, references/tauri-local-viewer/README.md, references/cpp-native-viewer/README.md, references/headless-shm-cli/README.md, references/viewer-backend-interface/README.md, references/webgl-shm-transport/README.md.
Viewer UI/UX: references/viewer-ux-workflow/README.md, references/viewer-layout-patterns/README.md, references/viewer-control-patterns/README.md, references/viewer-data-view-patterns/README.md, references/viewer-feedback-status/README.md.
Interaction: references/viewer-input-routing/README.md, references/camera-controls/README.md, references/object-selection/README.md, references/native-picking-selection/README.md, references/selection-feedback/README.md, references/selection-animation/README.md, references/transform-manipulator/README.md, references/gl-viewport-overlay/README.md, references/ovui-library/README.md, references/prim-pick-effects/README.md, references/prim-info-display/README.md, references/viewport-overlays/README.md.
Infrastructure: references/dependencies/README.md, references/windows-native-setup/README.md, references/cloud-assets/README.md, references/cloud-deployment/README.md, references/troubleshooting/README.md.

Build Workflow

Classify the prompt by delivery path, target user, required capabilities, runtime environment, validation needs, and explicit constraints.
Select a small reference set. Start with the recipe or routing reference, then add focused capabilities such as camera, picking, hierarchy, properties, render settings, transform tools, cloud assets, or deployment.
Read selected references before writing app code. Follow their build order, import order, data-channel contracts, and renderer ownership rules.
Implement the core render path first, then input routing and camera, then selection and data panels, then scene/settings features, then packaging or deployment.
Treat the selected references as the behavior contract for API shape, compatibility, and generated project structure.
Capture validation evidence before calling the viewer ready.

Examples

For a browser viewer request, use the streaming recipe references plus camera, picking, hierarchy, properties, render settings, and stream-status references.
For a local workstation viewer request, use the local or native delivery references plus renderer setup, stage loading, viewport input, and validation.

Completion Checklist

Selected references match the user's intent and delivery path.
No code path uses a browser-side 3D renderer for USD.
The generated app has one clear owner for render stepping and stage mutation.
User USD files remain untouched by viewer-owned session data.
Camera, input, selection, scene loading, and stream behavior follow references/conventions.md.
Setup/build/run results and visual interaction evidence are captured with references/validation.md.

Files

ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
BENCHMARK.md5.5 KiB
LICENSEAdded by AutoGineer10.0 KiB
SKILL.md7.4 KiB
agents/openai.yaml315 B
evals/evals.json8.1 KiB
references/aov-switching/README.md15.2 KiB
references/camera-auto-select/README.md7.4 KiB
references/camera-controls/README.md17.1 KiB
references/camera-picker/README.md9.7 KiB
references/cloud-assets/README.md10.4 KiB
references/cloud-deployment/README.md18.0 KiB
references/conventions.md5.3 KiB
references/cpp-native-viewer/README.md2.4 KiB
references/cpp-native-viewer/interaction-features.md19.2 KiB
references/cpp-native-viewer/presentation-loop.md9.0 KiB
references/cpp-native-viewer/project-build.md6.4 KiB
references/cpp-native-viewer/renderer-session.md4.5 KiB
references/cpp-native-viewer/validation.md3.3 KiB
references/dependencies/README.md3.4 KiB
references/dependencies/electron-shm.md1.5 KiB
references/dependencies/environment-validation.md4.4 KiB
references/dependencies/frontend.md2.3 KiB
references/dependencies/local-openusd-gpu.md4.4 KiB
references/dependencies/nvidia-runtime.md9.0 KiB
references/dependencies/ovrtx.md2.4 KiB
references/dependencies/ovstream.md2.5 KiB
references/dependencies/quick-setup.md5.3 KiB
references/electron-shm-viewer/README.md2.2 KiB
references/electron-shm-viewer/architecture-project.md6.3 KiB
references/electron-shm-viewer/electron-client.md5.6 KiB
references/electron-shm-viewer/protocol-interaction-lifecycle.md6.9 KiB
references/electron-shm-viewer/python-shm-server.md4.9 KiB
references/electron-shm-viewer/validation.md2.8 KiB
references/gl-viewport-overlay/README.md7.9 KiB
references/gl-viewport-overlay/architecture.md4.1 KiB
references/gl-viewport-overlay/example-gizmo.md5.5 KiB
references/gl-viewport-overlay/interaction-pattern.md4.0 KiB
references/gl-viewport-overlay/projection-alignment.md4.3 KiB
references/headless-shm-cli/README.md12.2 KiB
references/huggingface-usd/README.md7.7 KiB
references/local-viewer/README.md15.0 KiB
references/native-picking-selection/README.md10.4 KiB
references/object-selection/README.md9.4 KiB
references/ovrtx-rendering/README.md18.2 KiB
references/ovui-library/README.md3.3 KiB
references/ovui-library/api-reference.md4.3 KiB
references/ovui-library/extending.md4.8 KiB
references/ovui-local-viewer-recipe/README.md3.7 KiB
references/ovui-local-viewer-recipe/interaction-features.md16.1 KiB
references/ovui-local-viewer-recipe/project-structure.md6.2 KiB
references/ovui-local-viewer-recipe/setup-shell-renderer.md14.3 KiB
references/ovui-local-viewer-recipe/validation-build-order.md5.5 KiB
references/ovwidgets-editor-shell/README.md8.6 KiB
references/prim-info-display/README.md9.3 KiB
references/prim-pick-effects/README.md5.9 KiB
references/prim-transform-safety/README.md8.2 KiB
references/render-settings/README.md9.9 KiB
references/routing.md16.5 KiB
references/seg-outline-highlight/README.md1.4 KiB
references/selection-animation/README.md9.7 KiB
references/selection-feedback/README.md6.6 KiB
references/stage-attribute-reads/README.md5.5 KiB
references/stage-hierarchy/README.md17.1 KiB
references/stage-hierarchy/fallback-worker-protocol.md3.5 KiB
references/stage-loading/README.md19.3 KiB
references/stage-management/README.md10.1 KiB
references/stage-queries/README.md5.3 KiB
references/streaming-client/README.md17.4 KiB
references/streaming-lifecycle/README.md8.8 KiB
references/streaming-messages/README.md16.5 KiB
references/streaming-messages/server-handler-map.md3.1 KiB
references/streaming-server/README.md18.6 KiB
references/streaming-server/frame-loop-and-continuity.md3.3 KiB
references/streaming-viewer-recipe/README.md4.8 KiB
references/streaming-viewer-recipe/client-protocol.md11.0 KiB
references/streaming-viewer-recipe/interaction-features.md13.0 KiB
references/streaming-viewer-recipe/project-structure.md7.0 KiB
references/streaming-viewer-recipe/server-runtime.md16.8 KiB
references/streaming-viewer-recipe/validation-build-order.md5.8 KiB
references/streaming-vs-local/README.md8.1 KiB
references/tauri-local-viewer/README.md18.2 KiB
references/tauri-shm-transform-gizmo/README.md11.8 KiB
references/transform-manipulator/README.md15.0 KiB
references/troubleshooting/README.md7.2 KiB
references/troubleshooting/scenario-playbooks.md18.8 KiB
references/usd-sample-data/README.md11.7 KiB
references/usd-viewer-app/README.md7.5 KiB
references/validation.md3.0 KiB
references/viewer-backend-interface/README.md17.7 KiB
references/viewer-control-patterns/README.md4.8 KiB
references/viewer-data-view-patterns/README.md4.8 KiB
references/viewer-feedback-status/README.md4.9 KiB
references/viewer-input-routing/README.md7.8 KiB
references/viewer-layout-patterns/README.md4.7 KiB
references/viewer-ux-workflow/README.md4.6 KiB
references/viewport-overlays/README.md4.8 KiB
references/viewport-resize/README.md8.5 KiB
references/webgl-shm-transport/README.md14.5 KiB
references/windows-native-setup/README.md5.7 KiB
skill-card.md2.9 KiB
skill.oms.sig28.6 KiB

references/dependencies/environment-validation.md

Environment And Validation

Environment Variable Summary

Set before any ovrtx work:

export OVRTX_SKIP_USD_CHECK=1

Set when renderer plugins or MDL libraries do not resolve:

export OVRTX_BIN_PATH="$(python3 -c 'import ovrtx, os; print(os.path.join(os.path.dirname(ovrtx.__file__), "bin"))')"
export LD_LIBRARY_PATH="$OVRTX_BIN_PATH/plugins${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

Use nvidia-runtime.md for current NVIDIA runtime acquisition locations. Do not keep separate ovstream source URLs or cache controls here.

Set only when an ovstream install cannot locate SDK libraries automatically:

export OVSTREAM_LIB_PATH=/absolute/path/to/ovstream/native/libs

Set for local UI under a known display:

export DISPLAY=${DISPLAY:-:0}

Set when the default home cache is not writable:

export XDG_CACHE_HOME="$PWD/.cache"
export CUDA_CACHE_PATH="$PWD/.cache/cuda"
export __GL_SHADER_DISK_CACHE_PATH="$PWD/.cache/gl"
export npm_config_cache="$PWD/.cache/npm"

Verification Checklist

Run these checks after installation:

Confirm GPU and driver are visible with nvidia-smi.
Confirm Python is the expected interpreter inside the virtual environment.
Confirm OVRTX_SKIP_USD_CHECK=1 is set before renderer imports.
Confirm ovrtx imports and renderer construction succeeds on the target GPU.
Confirm OVRTX_BIN_PATH resolves to the installed ovrtx bin directory when plugin resolution fails.
Confirm pxr imports only in the selected query subprocess.
Confirm usd-core is exactly 24.11.
Confirm ovstream.initialize() and ovstream.shutdown() work for streaming Omniverse Realtime Viewers.
Confirm warp can see CUDA devices when using CUDA conversion paths.
Confirm @nvidia/ov-web-rtc appears in npm ls when building a browser streaming frontend.
If the app imports local viewer UI components, confirm frontend/src/viewer-ui/ exists and exports the referenced components and ViewerBackend types.
Confirm omni.ui imports only when building a local desktop Omniverse Realtime Viewer with prebuilt local UI packages.
Confirm .cache/ is writable when running in containers, CI, or shared environments.

For generated browser-streamed viewers, add these readiness checks before reporting completion:

Start the Python server from the generated run wrapper or equivalent command.
Wait for /healthz to return 200 ok; if it does not, capture the server log.
Confirm the log contains a first-frame message after render-var mapping and RGBA-to-BGRA conversion.
Run the frontend build and browser smoke check only after the server runtime proof above has passed or has produced a concrete failure report.

Failure Mode Index

Wrong Python package index: ovrtx install fails or pulls nothing. Use NVIDIA PyPI for ovrtx; use PyPI for numpy, warp-lang, and usd-core==24.11.
Incorrect ovstream acquisition: use the current PyPI package source in nvidia-runtime.md.
Platform mismatch: wheel is unavailable or native import fails. Confirm OS, architecture, Python version, GPU driver, and package wheel tags.
Import order issue: USD registry, _tf, duplicate debug symbol, or MDL resolver errors. Set OVRTX_SKIP_USD_CHECK, construct ovrtx.Renderer first, and isolate pxr in a subprocess.
Native library path issue: CRenderApi, MDL, ovstream native, or plugin load failures. Set OVRTX_BIN_PATH, dynamic library path, or OVSTREAM_LIB_PATH as appropriate.
GPU access issue: renderer or streaming initialization fails despite successful imports. Verify nvidia-smi, container GPU devices, driver support, RTX cores, and NVENC.
Display issue: local desktop UI cannot open a window. Verify DISPLAY and X server availability.
Cache permission issue: Warp, CUDA shader caching, GL shader caching, or npm fails under ~/.cache. Set project-local cache paths and ignore .cache/.
Frontend registry issue: npm cannot resolve the NVIDIA package. Check package spelling, the @nvidia registry in .npmrc, lockfile, and proxy configuration.
Local viewer UI issue: TypeScript cannot resolve ViewerBackend, StageTree, Inspector, or related local viewer UI imports. Generate frontend/src/viewer-ui/ from viewer-backend-interface, or update imports to the app's actual local module path.

See also: ovrtx-rendering, local-viewer, streaming-server, streaming-client, stage-hierarchy, and windows-native-setup.

Requirements and compatibility

Written for: OpenAI Codex

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

The author says: Orchestrator skill. Downstream focused references may require NVIDIA GPUs, ovrtx, ovstream, ovui, OpenUSD, Python, Node/React, Tauri, Electron, C++, or cloud GPU deployment access depending on the selected viewer path.

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA Omniverse

Publisher

OpenAI (Official)

Distributed by

OpenAI

Source

openai/plugins · plugins/nvidia/skills/omniverse-realtime-viewer · commit d420d13

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by NVIDIA Omniverse or OpenAI.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download omniverse-realtime-viewer.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

Instructions

Read Order

Non-Negotiables

Focused Reference Families

Build Workflow

Examples

Completion Checklist

Files

references/dependencies/environment-validation.md

Environment And Validation

Environment Variable Summary

Verification Checklist

Failure Mode Index

Requirements and compatibility

Security

Attribution and licence

Related skills

Setup TS Deep Modules

To Spec

Improve Codebase Architecture

Domain Modeling

Codebase Design

Prototype

Download ZIP