Omniverse Realtime Viewer

OpenAIOfficialCreated by NVIDIA OmniverseDistributed by OpenAI

Skill
Meta (authoring, orchestration, planning, memory)
Software development
Apache-2.0
Medium risk

Download ZIP

From the author

Use as the top-level router for Omniverse Realtime Viewer USD app requests and focused viewer reference documents.

— the description by NVIDIA Omniverse

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Omniverse Realtime Viewer skill acts as a central orchestrator for developing USD-based viewer applications. It provides a structured approach to selecting implementation recipes, enforcing architectural conventions, and managing dependencies. By utilizing a set of focused reference documents, it guides developers through the lifecycle of building, configuring, and validating viewer applications, ensuring consistent behavior across different delivery paths such as local workstations or streaming environments.

What it does

Routes development requests to specific implementation recipes based on the target delivery path.
Enforces architectural rules for USD rendering, stage management, and UI integration.
Provides guidance on camera selection, input handling, and viewport interaction.
Standardizes the integration of AOV switching and render product configuration.
Facilitates the capture of validation evidence for viewer readiness.
Maintains separation between user-provided USD files and application-specific session state.

When to use it

When initiating the development of a new USD viewer application within the Omniverse ecosystem.
When determining the appropriate delivery path for a viewer, such as local native or streaming.
When needing to implement consistent camera, selection, or UI behavior across viewer projects.
When integrating advanced rendering features like AOV switching into an existing viewer.

Advantages

Ensures architectural consistency across diverse viewer implementations.
Provides a clear, modular path for adding specific capabilities like input routing or data display.
Promotes clean separation of concerns between stage data and viewer-owned state.
Standardizes the validation process for consistent delivery.

Usage examples

“I need to build a browser-based viewer for USD files.”: The agent will route you to the streaming viewer recipe, providing guidance on WebRTC stream setup, UI integration, and viewport interaction.
“How do I implement AOV switching in my local USD viewer?”: The agent will provide the AOV switching reference, detailing how to map render vars and update the display state on the server.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/openai/omniverse-realtime-viewer/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip omniverse-realtime-viewer.zip -d .agents/skills

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

This is the top-level entry point for the Omniverse Realtime Viewer skill package. It is self-contained: all required routing, conventions, and validation guidance live in the selected references.

Use the focused reference documents as implementation recipes. This file chooses the right recipes and preserves the architectural rules that must hold across all generated viewer apps.

Instructions

Start by classifying the requested viewer, then read only the references needed for that delivery path and feature set. Implement the render path first, layer interaction and UI behavior on top of it, and finish by capturing validation evidence from references/validation.md.

Read Order

Read references/routing.md to choose the delivery path and focused references.
Read references/conventions.md before implementing camera, input, selection, viewport, streaming protocol, scene loading, or environment behavior.
For broad viewer requests, read references/usd-viewer-app/README.md.
If the delivery path is unclear, read references/streaming-vs-local/README.md.
If the prompt includes layout, panels, controls, inspectors, status, or UX, read references/viewer-ux-workflow/README.md and then the focused viewer UI references. This applies to React/WebRTC, Tauri, Electron, ovui, ovwidgets, and Dear ImGui apps; "frontend" means user-facing UI, not only browser UI.
For viewport interaction, read references/viewer-input-routing/README.md before references/camera-controls/README.md, references/native-picking-selection/README.md, or references/object-selection/README.md.
Read only the focused capability references needed for the requested app.
Use references/validation.md to capture review evidence before handoff.

Non-Negotiables

Use ovrtx for all USD and 3D rendering.
Browser apps display an ovstream WebRTC video stream plus UI. The browser does not render USD geometry.
Do not substitute WebGL, Three.js, Babylon.js, PlayCanvas, A-Frame, model-viewer, react-three-fiber, glTF browser viewers, or other client-side 3D renderers.
If local validation cannot run because the GPU/runtime environment is absent, scaffold the ovrtx path and document the runtime requirement. Do not add a browser-renderer fallback.
Keep user USD files unmodified. Viewer cameras, render products, render vars, settings, selection metadata, and runtime state belong in session/composite layers or app state.
Keep one owner for renderer.step(), stage mutation, native picking, selection writes, and live attribute writes.
Keep dependency acquisition in references/dependencies/README.md and deployment choices in references/cloud-deployment/README.md; do not duplicate package locations or deployment setup.

Focused Reference Families

Entry points and recipes: references/usd-viewer-app/README.md, references/streaming-viewer-recipe/README.md, references/ovui-local-viewer-recipe/README.md, references/streaming-vs-local/README.md, references/electron-shm-viewer/README.md, references/ovwidgets-editor-shell/README.md.
Rendering and stage: references/ovrtx-rendering/README.md, references/stage-loading/README.md, references/stage-management/README.md, references/render-settings/README.md, references/aov-switching/README.md, references/stage-hierarchy/README.md, references/stage-queries/README.md, references/stage-attribute-reads/README.md, references/prim-transform-safety/README.md, references/usd-sample-data/README.md.
Delivery and runtime: references/streaming-server/README.md, references/streaming-client/README.md, references/streaming-messages/README.md, references/streaming-lifecycle/README.md, references/local-viewer/README.md, references/tauri-local-viewer/README.md, references/cpp-native-viewer/README.md, references/headless-shm-cli/README.md, references/viewer-backend-interface/README.md, references/webgl-shm-transport/README.md.
Viewer UI/UX: references/viewer-ux-workflow/README.md, references/viewer-layout-patterns/README.md, references/viewer-control-patterns/README.md, references/viewer-data-view-patterns/README.md, references/viewer-feedback-status/README.md.
Interaction: references/viewer-input-routing/README.md, references/camera-controls/README.md, references/object-selection/README.md, references/native-picking-selection/README.md, references/selection-feedback/README.md, references/selection-animation/README.md, references/transform-manipulator/README.md, references/gl-viewport-overlay/README.md, references/ovui-library/README.md, references/prim-pick-effects/README.md, references/prim-info-display/README.md, references/viewport-overlays/README.md.
Infrastructure: references/dependencies/README.md, references/windows-native-setup/README.md, references/cloud-assets/README.md, references/cloud-deployment/README.md, references/troubleshooting/README.md.

Build Workflow

Classify the prompt by delivery path, target user, required capabilities, runtime environment, validation needs, and explicit constraints.
Select a small reference set. Start with the recipe or routing reference, then add focused capabilities such as camera, picking, hierarchy, properties, render settings, transform tools, cloud assets, or deployment.
Read selected references before writing app code. Follow their build order, import order, data-channel contracts, and renderer ownership rules.
Implement the core render path first, then input routing and camera, then selection and data panels, then scene/settings features, then packaging or deployment.
Treat the selected references as the behavior contract for API shape, compatibility, and generated project structure.
Capture validation evidence before calling the viewer ready.

Examples

For a browser viewer request, use the streaming recipe references plus camera, picking, hierarchy, properties, render settings, and stream-status references.
For a local workstation viewer request, use the local or native delivery references plus renderer setup, stage loading, viewport input, and validation.

Completion Checklist

Selected references match the user's intent and delivery path.
No code path uses a browser-side 3D renderer for USD.
The generated app has one clear owner for render stepping and stage mutation.
User USD files remain untouched by viewer-owned session data.
Camera, input, selection, scene loading, and stream behavior follow references/conventions.md.
Setup/build/run results and visual interaction evidence are captured with references/validation.md.

Files

ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
BENCHMARK.md5.5 KiB
LICENSEAdded by AutoGineer10.0 KiB
SKILL.md7.4 KiB
agents/openai.yaml315 B
evals/evals.json8.1 KiB
references/aov-switching/README.md15.2 KiB
references/camera-auto-select/README.md7.4 KiB
references/camera-controls/README.md17.1 KiB
references/camera-picker/README.md9.7 KiB
references/cloud-assets/README.md10.4 KiB
references/cloud-deployment/README.md18.0 KiB
references/conventions.md5.3 KiB
references/cpp-native-viewer/README.md2.4 KiB
references/cpp-native-viewer/interaction-features.md19.2 KiB
references/cpp-native-viewer/presentation-loop.md9.0 KiB
references/cpp-native-viewer/project-build.md6.4 KiB
references/cpp-native-viewer/renderer-session.md4.5 KiB
references/cpp-native-viewer/validation.md3.3 KiB
references/dependencies/README.md3.4 KiB
references/dependencies/electron-shm.md1.5 KiB
references/dependencies/environment-validation.md4.4 KiB
references/dependencies/frontend.md2.3 KiB
references/dependencies/local-openusd-gpu.md4.4 KiB
references/dependencies/nvidia-runtime.md9.0 KiB
references/dependencies/ovrtx.md2.4 KiB
references/dependencies/ovstream.md2.5 KiB
references/dependencies/quick-setup.md5.3 KiB
references/electron-shm-viewer/README.md2.2 KiB
references/electron-shm-viewer/architecture-project.md6.3 KiB
references/electron-shm-viewer/electron-client.md5.6 KiB
references/electron-shm-viewer/protocol-interaction-lifecycle.md6.9 KiB
references/electron-shm-viewer/python-shm-server.md4.9 KiB
references/electron-shm-viewer/validation.md2.8 KiB
references/gl-viewport-overlay/README.md7.9 KiB
references/gl-viewport-overlay/architecture.md4.1 KiB
references/gl-viewport-overlay/example-gizmo.md5.5 KiB
references/gl-viewport-overlay/interaction-pattern.md4.0 KiB
references/gl-viewport-overlay/projection-alignment.md4.3 KiB
references/headless-shm-cli/README.md12.2 KiB
references/huggingface-usd/README.md7.7 KiB
references/local-viewer/README.md15.0 KiB
references/native-picking-selection/README.md10.4 KiB
references/object-selection/README.md9.4 KiB
references/ovrtx-rendering/README.md18.2 KiB
references/ovui-library/README.md3.3 KiB
references/ovui-library/api-reference.md4.3 KiB
references/ovui-library/extending.md4.8 KiB
references/ovui-local-viewer-recipe/README.md3.7 KiB
references/ovui-local-viewer-recipe/interaction-features.md16.1 KiB
references/ovui-local-viewer-recipe/project-structure.md6.2 KiB
references/ovui-local-viewer-recipe/setup-shell-renderer.md14.3 KiB
references/ovui-local-viewer-recipe/validation-build-order.md5.5 KiB
references/ovwidgets-editor-shell/README.md8.6 KiB
references/prim-info-display/README.md9.3 KiB
references/prim-pick-effects/README.md5.9 KiB
references/prim-transform-safety/README.md8.2 KiB
references/render-settings/README.md9.9 KiB
references/routing.md16.5 KiB
references/seg-outline-highlight/README.md1.4 KiB
references/selection-animation/README.md9.7 KiB
references/selection-feedback/README.md6.6 KiB
references/stage-attribute-reads/README.md5.5 KiB
references/stage-hierarchy/README.md17.1 KiB
references/stage-hierarchy/fallback-worker-protocol.md3.5 KiB
references/stage-loading/README.md19.3 KiB
references/stage-management/README.md10.1 KiB
references/stage-queries/README.md5.3 KiB
references/streaming-client/README.md17.4 KiB
references/streaming-lifecycle/README.md8.8 KiB
references/streaming-messages/README.md16.5 KiB
references/streaming-messages/server-handler-map.md3.1 KiB
references/streaming-server/README.md18.6 KiB
references/streaming-server/frame-loop-and-continuity.md3.3 KiB
references/streaming-viewer-recipe/README.md4.8 KiB
references/streaming-viewer-recipe/client-protocol.md11.0 KiB
references/streaming-viewer-recipe/interaction-features.md13.0 KiB
references/streaming-viewer-recipe/project-structure.md7.0 KiB
references/streaming-viewer-recipe/server-runtime.md16.8 KiB
references/streaming-viewer-recipe/validation-build-order.md5.8 KiB
references/streaming-vs-local/README.md8.1 KiB
references/tauri-local-viewer/README.md18.2 KiB
references/tauri-shm-transform-gizmo/README.md11.8 KiB
references/transform-manipulator/README.md15.0 KiB
references/troubleshooting/README.md7.2 KiB
references/troubleshooting/scenario-playbooks.md18.8 KiB
references/usd-sample-data/README.md11.7 KiB
references/usd-viewer-app/README.md7.5 KiB
references/validation.md3.0 KiB
references/viewer-backend-interface/README.md17.7 KiB
references/viewer-control-patterns/README.md4.8 KiB
references/viewer-data-view-patterns/README.md4.8 KiB
references/viewer-feedback-status/README.md4.9 KiB
references/viewer-input-routing/README.md7.8 KiB
references/viewer-layout-patterns/README.md4.7 KiB
references/viewer-ux-workflow/README.md4.6 KiB
references/viewport-overlays/README.md4.8 KiB
references/viewport-resize/README.md8.5 KiB
references/webgl-shm-transport/README.md14.5 KiB
references/windows-native-setup/README.md5.7 KiB
skill-card.md2.9 KiB
skill.oms.sig28.6 KiB

references/windows-native-setup/README.md

Windows Native Setup

Triggers

Use this skill for Windows, native Windows, WSL2, ERROR_INCOMPATIBLE_DRIVER, NVML, DLL load failed, or usd-core 24.11.

Run natively on Windows 10/11. Do not use WSL2; ovrtx needs direct Vulkan/NVML GPU access and WSL2 commonly fails with ERROR_INCOMPATIBLE_DRIVER or NVML_ERROR_DRIVER_NOT_LOADED.

Prerequisites

NVIDIA RTX GPU, Turing or newer.
NVIDIA driver 535+ with CUDA 12.x.
Python version matching the latest selected runtime wheels. Check the current ovui package files from references/dependencies and create the virtual environment with a supported Python version unless the project manifest pins a different compatible package set.
Node.js 20+, npm 10+, Git.

Additional prerequisites may be required by the selected local desktop ovui package or dependency build instructions:

Visual Studio Build Tools with the MSVC C++ x64 toolchain.
vswhere.exe available from Visual Studio Installer.
Ninja installed in the active venv or visible to pip build isolation.
Vulkan SDK when required by the current ovui package or dependency instructions.

Install

Read references/dependencies first. Its references/nvidia-runtime.md file owns current acquisition details for ovrtx, ovstream, ovui, and the ov-web-rtc browser client; this Windows guide should not repeat release URLs, wheel names, or artifact locations.

Start from the root of the generated viewer project:

cd C:\path\to\generated-viewer
py -3.10 -m venv .venv
.\.venv\Scripts\Activate.ps1

Install NVIDIA runtimes using references/dependencies, then install supporting packages:

pip install warp-lang
pip install usd-core==24.11
if (Test-Path server\requirements.txt) { pip install -r server\requirements.txt }

Pin usd-core==24.11. Version 26.x can cause TfType::AddAlias schema conflicts. Even 24.11 conflicts with ovrtx in-process on Windows, so USD queries run in pxr_worker.py.

Local ovui Setup

Use references/dependencies for the current ovui PyPI package guidance. Keep the base ovui package and companion packages on one compatible package set.

The distribution may install omni.ui and omni.ui_scene import packages. Verify with:

python -c "import omni.ui as ui; import omni.ui_scene; print('ok')"

If ovui-data-adapters reports that no setup.py or pyproject.toml exists, use a package set that includes matching package metadata. Do not patch packaging metadata from this skill.

If PowerShell launches a .bat file that uses for /f "usebackq" ... in (\...`)aroundpython -c "...", quoting can be mangled before cmd.exereceives it. Use a small helper.py` script for Python probes inside batch loops.

Stage Syntax Check

If using samples/stage01.usda, clippingRange must be:

float2 clippingRange = (0.1, 10000)

Not separate float clippingRange.near and .far attributes.

Run

cd frontend
npm install
npm run dev

cd server
python ov_web_viewer_server.py --stage ..\samples\stage01.usda --port 49100

Open:

http://localhost:3000?server=127.0.0.1&signalingport=49100

First launch can spend 5 to 10 minutes compiling RTX shaders after Stage loaded successfully, with many UJITSO material warnings. Do not kill it; cached shaders make later launches faster.

Architecture

Windows keeps pxr and ovrtx in separate processes:

ov_web_viewer_server.py
  ovrtx.Renderer
  ovstream.Server
  PxrWorkerClient -> pxr_worker.py -> pxr.Usd

The main process never imports pxr; USD hierarchy, variants, and property queries use newline-delimited JSON over stdin/stdout.

ovrtx also needs inline root/session data with a camera, RenderProduct, RenderVar, and RenderSettings. Missing this causes Unable to find RenderProduct prim.

WebRTC Rules

Server:

config.webrtc_signal_port = 49100
config.webrtc_public_ip = "127.0.0.1"

Frontend:

return { server, signalingPort }; // no mediaServer/mediaPort

The client must use server, not signalingServer, and must not set media fields. Callback registration must happen before server.start(). Data-channel messages may arrive wrapped in {messageType,messageRecipient,data} and must be unwrapped. On connect, push openStageResult and root getChildrenResult after about 300 ms so the frontend sees already-loaded state.

Troubleshooting

Symptom	Fix
`ERROR_INCOMPATIBLE_DRIVER` / `NVML_ERROR_DRIVER_NOT_LOADED`	run native Windows, not WSL2
`_tf` DLL import failure	keep pxr in worker subprocess
`TfType::AddAlias` conflict	pin `usd-core==24.11`
`OSError: cannot load library ovstream`	remove wrong `OVSTREAM_LIB_PATH`; use the current `ovstream` package from `references/dependencies`
`cannot import name 'VIEWPORT_CAMERA_POSE_SOURCE'`	install local UI packages from the same package set
`Neither 'setup.py' nor 'pyproject.toml' found` under `ovui-data-adapters`	use an `ovui` package set that includes matching package metadata
Native UI package requires a compiler toolchain	follow the current `ovui` package/build instructions
`TypeError: a coroutine was expected` from `ui.run`	pass an async render loop coroutine, not a plain callback
stuck "Loading stage..."	remove `mediaServer`/`mediaPort`
`Previous session is already running`	reduce reconnects, add delay
`VideoEncoder was not deinitialized`	non-fatal shutdown-order warning
`Unable to find RenderProduct prim`	use `stage-loading` wrapper/session stage
red/blue swapped	apply RGBA-to-BGRA warp swap before streaming

See also: streaming-server, streaming-client, streaming-lifecycle, stage-hierarchy, stage-loading.

Requirements and compatibility

Written for: OpenAI Codex

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

The author says: Orchestrator skill. Downstream focused references may require NVIDIA GPUs, ovrtx, ovstream, ovui, OpenUSD, Python, Node/React, Tauri, Electron, C++, or cloud GPU deployment access depending on the selected viewer path.

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA Omniverse

Publisher

OpenAI (Official)

Distributed by

OpenAI

Source

openai/plugins · plugins/nvidia/skills/omniverse-realtime-viewer · commit d420d13

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by NVIDIA Omniverse or OpenAI.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download omniverse-realtime-viewer.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

Instructions

Read Order

Non-Negotiables

Focused Reference Families

Build Workflow

Examples

Completion Checklist

Files

references/windows-native-setup/README.md

Windows Native Setup

Triggers

Prerequisites

Install

Local ovui Setup

Stage Syntax Check

Run

Architecture

WebRTC Rules

Troubleshooting

Requirements and compatibility

Security

Attribution and licence

Related skills

Setup TS Deep Modules

To Spec

Improve Codebase Architecture

Domain Modeling

Codebase Design

Prototype

Download ZIP