Omniverse Realtime Viewer

OpenAIOfficialCreated by NVIDIA OmniverseDistributed by OpenAI

Skill
Meta (authoring, orchestration, planning, memory)
Software development
Apache-2.0
Medium risk

Download ZIP

From the author

Use as the top-level router for Omniverse Realtime Viewer USD app requests and focused viewer reference documents.

— the description by NVIDIA Omniverse

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Omniverse Realtime Viewer skill acts as a central orchestrator for developing USD-based viewer applications. It provides a structured approach to selecting implementation recipes, enforcing architectural conventions, and managing dependencies. By utilizing a set of focused reference documents, it guides developers through the lifecycle of building, configuring, and validating viewer applications, ensuring consistent behavior across different delivery paths such as local workstations or streaming environments.

What it does

Routes development requests to specific implementation recipes based on the target delivery path.
Enforces architectural rules for USD rendering, stage management, and UI integration.
Provides guidance on camera selection, input handling, and viewport interaction.
Standardizes the integration of AOV switching and render product configuration.
Facilitates the capture of validation evidence for viewer readiness.
Maintains separation between user-provided USD files and application-specific session state.

When to use it

When initiating the development of a new USD viewer application within the Omniverse ecosystem.
When determining the appropriate delivery path for a viewer, such as local native or streaming.
When needing to implement consistent camera, selection, or UI behavior across viewer projects.
When integrating advanced rendering features like AOV switching into an existing viewer.

Advantages

Ensures architectural consistency across diverse viewer implementations.
Provides a clear, modular path for adding specific capabilities like input routing or data display.
Promotes clean separation of concerns between stage data and viewer-owned state.
Standardizes the validation process for consistent delivery.

Usage examples

“I need to build a browser-based viewer for USD files.”: The agent will route you to the streaming viewer recipe, providing guidance on WebRTC stream setup, UI integration, and viewport interaction.
“How do I implement AOV switching in my local USD viewer?”: The agent will provide the AOV switching reference, detailing how to map render vars and update the display state on the server.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/openai/omniverse-realtime-viewer/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip omniverse-realtime-viewer.zip -d .agents/skills

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

This is the top-level entry point for the Omniverse Realtime Viewer skill package. It is self-contained: all required routing, conventions, and validation guidance live in the selected references.

Use the focused reference documents as implementation recipes. This file chooses the right recipes and preserves the architectural rules that must hold across all generated viewer apps.

Instructions

Start by classifying the requested viewer, then read only the references needed for that delivery path and feature set. Implement the render path first, layer interaction and UI behavior on top of it, and finish by capturing validation evidence from references/validation.md.

Read Order

Read references/routing.md to choose the delivery path and focused references.
Read references/conventions.md before implementing camera, input, selection, viewport, streaming protocol, scene loading, or environment behavior.
For broad viewer requests, read references/usd-viewer-app/README.md.
If the delivery path is unclear, read references/streaming-vs-local/README.md.
If the prompt includes layout, panels, controls, inspectors, status, or UX, read references/viewer-ux-workflow/README.md and then the focused viewer UI references. This applies to React/WebRTC, Tauri, Electron, ovui, ovwidgets, and Dear ImGui apps; "frontend" means user-facing UI, not only browser UI.
For viewport interaction, read references/viewer-input-routing/README.md before references/camera-controls/README.md, references/native-picking-selection/README.md, or references/object-selection/README.md.
Read only the focused capability references needed for the requested app.
Use references/validation.md to capture review evidence before handoff.

Non-Negotiables

Use ovrtx for all USD and 3D rendering.
Browser apps display an ovstream WebRTC video stream plus UI. The browser does not render USD geometry.
Do not substitute WebGL, Three.js, Babylon.js, PlayCanvas, A-Frame, model-viewer, react-three-fiber, glTF browser viewers, or other client-side 3D renderers.
If local validation cannot run because the GPU/runtime environment is absent, scaffold the ovrtx path and document the runtime requirement. Do not add a browser-renderer fallback.
Keep user USD files unmodified. Viewer cameras, render products, render vars, settings, selection metadata, and runtime state belong in session/composite layers or app state.
Keep one owner for renderer.step(), stage mutation, native picking, selection writes, and live attribute writes.
Keep dependency acquisition in references/dependencies/README.md and deployment choices in references/cloud-deployment/README.md; do not duplicate package locations or deployment setup.

Focused Reference Families

Entry points and recipes: references/usd-viewer-app/README.md, references/streaming-viewer-recipe/README.md, references/ovui-local-viewer-recipe/README.md, references/streaming-vs-local/README.md, references/electron-shm-viewer/README.md, references/ovwidgets-editor-shell/README.md.
Rendering and stage: references/ovrtx-rendering/README.md, references/stage-loading/README.md, references/stage-management/README.md, references/render-settings/README.md, references/aov-switching/README.md, references/stage-hierarchy/README.md, references/stage-queries/README.md, references/stage-attribute-reads/README.md, references/prim-transform-safety/README.md, references/usd-sample-data/README.md.
Delivery and runtime: references/streaming-server/README.md, references/streaming-client/README.md, references/streaming-messages/README.md, references/streaming-lifecycle/README.md, references/local-viewer/README.md, references/tauri-local-viewer/README.md, references/cpp-native-viewer/README.md, references/headless-shm-cli/README.md, references/viewer-backend-interface/README.md, references/webgl-shm-transport/README.md.
Viewer UI/UX: references/viewer-ux-workflow/README.md, references/viewer-layout-patterns/README.md, references/viewer-control-patterns/README.md, references/viewer-data-view-patterns/README.md, references/viewer-feedback-status/README.md.
Interaction: references/viewer-input-routing/README.md, references/camera-controls/README.md, references/object-selection/README.md, references/native-picking-selection/README.md, references/selection-feedback/README.md, references/selection-animation/README.md, references/transform-manipulator/README.md, references/gl-viewport-overlay/README.md, references/ovui-library/README.md, references/prim-pick-effects/README.md, references/prim-info-display/README.md, references/viewport-overlays/README.md.
Infrastructure: references/dependencies/README.md, references/windows-native-setup/README.md, references/cloud-assets/README.md, references/cloud-deployment/README.md, references/troubleshooting/README.md.

Build Workflow

Classify the prompt by delivery path, target user, required capabilities, runtime environment, validation needs, and explicit constraints.
Select a small reference set. Start with the recipe or routing reference, then add focused capabilities such as camera, picking, hierarchy, properties, render settings, transform tools, cloud assets, or deployment.
Read selected references before writing app code. Follow their build order, import order, data-channel contracts, and renderer ownership rules.
Implement the core render path first, then input routing and camera, then selection and data panels, then scene/settings features, then packaging or deployment.
Treat the selected references as the behavior contract for API shape, compatibility, and generated project structure.
Capture validation evidence before calling the viewer ready.

Examples

For a browser viewer request, use the streaming recipe references plus camera, picking, hierarchy, properties, render settings, and stream-status references.
For a local workstation viewer request, use the local or native delivery references plus renderer setup, stage loading, viewport input, and validation.

Completion Checklist

Selected references match the user's intent and delivery path.
No code path uses a browser-side 3D renderer for USD.
The generated app has one clear owner for render stepping and stage mutation.
User USD files remain untouched by viewer-owned session data.
Camera, input, selection, scene loading, and stream behavior follow references/conventions.md.
Setup/build/run results and visual interaction evidence are captured with references/validation.md.

Files

ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
BENCHMARK.md5.5 KiB
LICENSEAdded by AutoGineer10.0 KiB
SKILL.md7.4 KiB
agents/openai.yaml315 B
evals/evals.json8.1 KiB
references/aov-switching/README.md15.2 KiB
references/camera-auto-select/README.md7.4 KiB
references/camera-controls/README.md17.1 KiB
references/camera-picker/README.md9.7 KiB
references/cloud-assets/README.md10.4 KiB
references/cloud-deployment/README.md18.0 KiB
references/conventions.md5.3 KiB
references/cpp-native-viewer/README.md2.4 KiB
references/cpp-native-viewer/interaction-features.md19.2 KiB
references/cpp-native-viewer/presentation-loop.md9.0 KiB
references/cpp-native-viewer/project-build.md6.4 KiB
references/cpp-native-viewer/renderer-session.md4.5 KiB
references/cpp-native-viewer/validation.md3.3 KiB
references/dependencies/README.md3.4 KiB
references/dependencies/electron-shm.md1.5 KiB
references/dependencies/environment-validation.md4.4 KiB
references/dependencies/frontend.md2.3 KiB
references/dependencies/local-openusd-gpu.md4.4 KiB
references/dependencies/nvidia-runtime.md9.0 KiB
references/dependencies/ovrtx.md2.4 KiB
references/dependencies/ovstream.md2.5 KiB
references/dependencies/quick-setup.md5.3 KiB
references/electron-shm-viewer/README.md2.2 KiB
references/electron-shm-viewer/architecture-project.md6.3 KiB
references/electron-shm-viewer/electron-client.md5.6 KiB
references/electron-shm-viewer/protocol-interaction-lifecycle.md6.9 KiB
references/electron-shm-viewer/python-shm-server.md4.9 KiB
references/electron-shm-viewer/validation.md2.8 KiB
references/gl-viewport-overlay/README.md7.9 KiB
references/gl-viewport-overlay/architecture.md4.1 KiB
references/gl-viewport-overlay/example-gizmo.md5.5 KiB
references/gl-viewport-overlay/interaction-pattern.md4.0 KiB
references/gl-viewport-overlay/projection-alignment.md4.3 KiB
references/headless-shm-cli/README.md12.2 KiB
references/huggingface-usd/README.md7.7 KiB
references/local-viewer/README.md15.0 KiB
references/native-picking-selection/README.md10.4 KiB
references/object-selection/README.md9.4 KiB
references/ovrtx-rendering/README.md18.2 KiB
references/ovui-library/README.md3.3 KiB
references/ovui-library/api-reference.md4.3 KiB
references/ovui-library/extending.md4.8 KiB
references/ovui-local-viewer-recipe/README.md3.7 KiB
references/ovui-local-viewer-recipe/interaction-features.md16.1 KiB
references/ovui-local-viewer-recipe/project-structure.md6.2 KiB
references/ovui-local-viewer-recipe/setup-shell-renderer.md14.3 KiB
references/ovui-local-viewer-recipe/validation-build-order.md5.5 KiB
references/ovwidgets-editor-shell/README.md8.6 KiB
references/prim-info-display/README.md9.3 KiB
references/prim-pick-effects/README.md5.9 KiB
references/prim-transform-safety/README.md8.2 KiB
references/render-settings/README.md9.9 KiB
references/routing.md16.5 KiB
references/seg-outline-highlight/README.md1.4 KiB
references/selection-animation/README.md9.7 KiB
references/selection-feedback/README.md6.6 KiB
references/stage-attribute-reads/README.md5.5 KiB
references/stage-hierarchy/README.md17.1 KiB
references/stage-hierarchy/fallback-worker-protocol.md3.5 KiB
references/stage-loading/README.md19.3 KiB
references/stage-management/README.md10.1 KiB
references/stage-queries/README.md5.3 KiB
references/streaming-client/README.md17.4 KiB
references/streaming-lifecycle/README.md8.8 KiB
references/streaming-messages/README.md16.5 KiB
references/streaming-messages/server-handler-map.md3.1 KiB
references/streaming-server/README.md18.6 KiB
references/streaming-server/frame-loop-and-continuity.md3.3 KiB
references/streaming-viewer-recipe/README.md4.8 KiB
references/streaming-viewer-recipe/client-protocol.md11.0 KiB
references/streaming-viewer-recipe/interaction-features.md13.0 KiB
references/streaming-viewer-recipe/project-structure.md7.0 KiB
references/streaming-viewer-recipe/server-runtime.md16.8 KiB
references/streaming-viewer-recipe/validation-build-order.md5.8 KiB
references/streaming-vs-local/README.md8.1 KiB
references/tauri-local-viewer/README.md18.2 KiB
references/tauri-shm-transform-gizmo/README.md11.8 KiB
references/transform-manipulator/README.md15.0 KiB
references/troubleshooting/README.md7.2 KiB
references/troubleshooting/scenario-playbooks.md18.8 KiB
references/usd-sample-data/README.md11.7 KiB
references/usd-viewer-app/README.md7.5 KiB
references/validation.md3.0 KiB
references/viewer-backend-interface/README.md17.7 KiB
references/viewer-control-patterns/README.md4.8 KiB
references/viewer-data-view-patterns/README.md4.8 KiB
references/viewer-feedback-status/README.md4.9 KiB
references/viewer-input-routing/README.md7.8 KiB
references/viewer-layout-patterns/README.md4.7 KiB
references/viewer-ux-workflow/README.md4.6 KiB
references/viewport-overlays/README.md4.8 KiB
references/viewport-resize/README.md8.5 KiB
references/webgl-shm-transport/README.md14.5 KiB
references/windows-native-setup/README.md5.7 KiB
skill-card.md2.9 KiB
skill.oms.sig28.6 KiB

references/dependencies/nvidia-runtime.md

NVIDIA Runtime Dependency Source Of Truth

This file is the single source of truth for NVIDIA runtime dependency acquisition in this skill package. Downstream skills may name the runtime they use and document API behavior, but they should not repeat package URLs, release URLs, workflow artifact links, registry paths, or fallback install locations.

Primary NVIDIA Dependencies

Dependency	Acquisition path	Used by	Guidance
`ovrtx`	NVIDIA Python package index	Local and streaming RTX USD rendering	Resolve the latest available package from this location.
`ovui`	PyPI package	Native local UI and server-side/headless overlay UI	Resolve the latest available package from this location.
`ovstream`	PyPI package	WebRTC and SHM streaming server/runtime	Resolve the latest available package from this location.
`ov-web-rtc client` (`@nvidia/ov-web-rtc`)	NVIDIA npm package	Browser-side WebRTC client for standalone `ovstream` Direct connections	Use the package guidance below.

Version Selection Rule

For new generated viewer apps, install the latest available ovrtx, ovui, and ovstream packages from the acquisition locations in this file. Do not copy a resolved version number into downstream skills, templates, or setup recipes.

If the host project already has a manifest or lockfile with an explicit runtime pin, respect that pin unless the user asks to update it. If compatibility requires a pin, keep it in the project manifest with a short reason rather than in this dependency source of truth.

Supplemental Dependency Documentation

These links centralize dependency documentation and examples. Use them for dependency-specific API behavior that is not covered by the selected viewer skills.

Dependency	Current documentation pointer	Use for
`ovrtx`	https://github.com/nvidia-omniverse/ovrtx	Renderer API behavior, Python/C API notes, stage composition, render-var/AOV behavior, picking/selection behavior, and release notes.
`ovui`	https://github.com/NVIDIA-Omniverse/ovui	Widget behavior, `ovwidgets`, `omni.ui`, headless overlay behavior, and native UI conventions.
`ovstream`	https://github.com/NVIDIA-Omniverse/ovstream	Library-specific `skills/`, sample servers, WebRTC lifecycle, SHM/client behavior, native input, examples, and package release notes.

Use this table only as supplemental documentation when the selected references do not contain enough detail for dependency-specific API behavior.

For ovstream, always check the supplemental repository when the task needs library-specific behavior, newer transport examples, native input details, or implementation patterns beyond this viewer skill package. That repository owns additional skills/ and samples for the streaming library itself.

Package-Index Dependencies

ovrtx

Use the NVIDIA Python package index for ovrtx.

Current supplemental repository pointer: https://github.com/nvidia-omniverse/ovrtx

python3 -m pip install --upgrade ovrtx --index-url https://pypi.nvidia.com --extra-index-url https://pypi.org/simple

If a project provides server/requirements.txt, prefer that project manifest over an ad hoc direct install. Preserve existing pins in that manifest unless the user asks to update them.

For ovrtx API behavior, renderer configuration, render vars, picking, selection, stage composition, or release-specific behavior not covered in this skill package, use the supplemental documentation pointer above.

ov-web-rtc client

Use the released @nvidia/ov-web-rtc package for the browser client that connects to standalone ovstream WebRTC servers in Direct mode. This skill package targets ovstream plus ovrtx viewer services. Those services may be containerized and launched by OKAS, Kubernetes, or another GPU session orchestrator. Do not use Kit, OVC, NVCF, or GFN client connection profiles as the browser WebRTC configuration; after orchestration resolves an endpoint, the frontend still uses Direct mode against the exposed ovstream signaling host and port.

Use the current released package. Do not copy resolved client version numbers into skills, templates, or setup recipes:

registry=https://registry.npmjs.org/
@nvidia:registry=https://edge.urm.nvidia.com/artifactory/api/npm/omniverse-client-npm/

npm install @nvidia/ov-web-rtc

For the ovstream-compatible Direct connection shape, use the current ovstream WebRTC browser client example as the reference pattern: https://github.com/NVIDIA-Omniverse/ovstream/tree/main/examples/webrtc_client

Use @nvidia/ov-web-rtc for new browser clients. Do not document alternate browser streaming package names, legacy package names, or Kit/OVC/NVCF/GFN client connection profiles for generated Omniverse Realtime Viewer apps.

Centralized Dependencies

GitHub Asset Retrieval

Use the package URLs and release selectors listed below. If direct browser, curl, or GitHub API access cannot retrieve a listed release or artifact, check whether GitHub CLI is authenticated and use gh for access:

gh auth status

For release assets, use gh release view and gh release download. For Actions artifacts, list artifacts through the API and download the named artifact:

gh api repos/OWNER/REPO/actions/runs/RUN_ID/artifacts \
  --jq '.artifacts[] | [.name, .expired, .archive_download_url] | @tsv'

gh run download RUN_ID \
  -R OWNER/REPO \
  -n ARTIFACT_NAME \
  -D vendor/ARTIFACT_NAME

If gh auth status is not authenticated or the token cannot access the listed repository, report the dependency retrieval failure. Do not use alternate wheel or tarball locations.

ovstream

Keep the current ovstream package source here rather than in streaming skills, templates, or setup recipes.

Current supplemental repository pointer: https://github.com/NVIDIA-Omniverse/ovstream

Current Python package: https://pypi.org/project/ovstream/

NVIDIA Python package index mirror: https://pypi.nvidia.com/ovstream/

Install the latest available wheel into the app virtual environment:

python3 -m pip install --upgrade ovstream

The current Python wheels bundle the native ovstream library, StreamSDK, GStreamer, the bundled gstnvenc plugin, CUDA runtime pieces, and ovstream_utils; no separate runtime zip is needed for normal Python apps.

If an environment must route NVIDIA packages through NVIDIA's package index, use the mirror with PyPI as the fallback:

python3 -m pip install --upgrade ovstream \
  --index-url https://pypi.nvidia.com \
  --extra-index-url https://pypi.org/simple

Use the C/CMake platform zips from the same release only for native C/C++ integrations. Set OVSTREAM_LIB_PATH only when running from an extracted runtime artifact layout, or when explicitly debugging native library discovery.

Rules:

Use the PyPI package and install instructions from this section for Python apps.
Install the latest available ovstream version unless the project manifest already pins a compatible version.
Do not repeat wheel filenames, direct wheel URLs, or alternate package locations in app-specific setup notes.
Do not point app-specific setup notes at unrelated local cache paths.
Runtime guidance may still document API usage such as ovstream.Server, callback ordering, OVSTREAM_LIB_PATH, and video frame submission.
For ovstream API or SHM behavior not covered in this skill package, downstream skills should ask agents to inspect the current supplemental repository pointer's skills/, samples, and release notes.

ovui

Keep the current ovui package source here rather than in local-viewer, overlay, or Windows setup skills.

Current supplemental repository pointer: https://github.com/NVIDIA-Omniverse/ovui

Current Python package: https://pypi.org/project/ovui/

NVIDIA Python package index mirror: https://pypi.nvidia.com/ovui/

Install the latest available wheel into the app virtual environment:

python3 -m pip install --upgrade ovui

Use the wheel matching the selected Python version, OS, and architecture.

If an environment must route NVIDIA packages through NVIDIA's package index, use the mirror with PyPI as the fallback:

python3 -m pip install --upgrade ovui \
  --index-url https://pypi.nvidia.com \
  --extra-index-url https://pypi.org/simple

Rules:

Use the PyPI package and install instructions from this section for Python apps.
Install the latest available ovui version unless the project manifest already pins a compatible version.
Keep ovui, ovui-data-adapters, ovwidgets, and related local UI companion packages on one compatible package set.
Keep direct wheel URLs, wheel filenames, and alternate install commands out of app-specific setup notes.
Runtime guidance may still document API usage such as omni.ui, omni.ui_scene, headless overlay contracts, PYTHONPATH, and display setup.
For ovui widget, ovwidgets, editor shell, or headless overlay behavior not covered in this skill package, use the supplemental documentation pointer above.

Requirements and compatibility

Written for: OpenAI Codex

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

The author says: Orchestrator skill. Downstream focused references may require NVIDIA GPUs, ovrtx, ovstream, ovui, OpenUSD, Python, Node/React, Tauri, Electron, C++, or cloud GPU deployment access depending on the selected viewer path.

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA Omniverse

Publisher

OpenAI (Official)

Distributed by

OpenAI

Source

openai/plugins · plugins/nvidia/skills/omniverse-realtime-viewer · commit d420d13

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by NVIDIA Omniverse or OpenAI.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download omniverse-realtime-viewer.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

Instructions

Read Order

Non-Negotiables

Focused Reference Families

Build Workflow

Examples

Completion Checklist

Files

references/dependencies/nvidia-runtime.md

NVIDIA Runtime Dependency Source Of Truth

Primary NVIDIA Dependencies

Version Selection Rule

Supplemental Dependency Documentation

Package-Index Dependencies

ovrtx

ov-web-rtc client

Centralized Dependencies

GitHub Asset Retrieval

ovstream

ovui

Requirements and compatibility

Security

Attribution and licence

Related skills

Setup TS Deep Modules

To Spec

Improve Codebase Architecture

Domain Modeling

Codebase Design

Prototype

Download ZIP