Nemotron Speech

Riva TTS NIM

Two modes: cloud-hosted (no GPU, uses build.nvidia.com) or self-hosted (your own GPU + Docker).

Agent: When walking the user through a multi-step workflow, announce each step before presenting it: Step N/M — Step Title (e.g., "Step 1/4 — Deploy the Container").

Source of truth. This skill describes deployment mechanics, which are stable across releases. For anything that varies per release — model catalog, container IDs, function IDs, voice lists, supported languages, feature support per model, VRAM minimums — fetch or open the canonical doc page and answer from that, not from this skill's text. See Looking up current information below.

Purpose

Deploy and run NVIDIA Riva TTS (text-to-speech) NIMs for speech synthesis. Supports cloud-hosted inference via build.nvidia.com (no GPU required) and self-hosted deployment. Covers offline synthesis, streaming, and Helm deployment.

Looking up current information

This skill is orientation, not catalog. When a question depends on data that changes per release, fetch or open the relevant page and answer from that page:

Do not infer from this skill's text: which models exist, which voices they expose, which languages are supported, what NIM_TAGS_SELECTOR values are valid, or what VRAM is required. The docs are the contract.

Naming caveat. The same model can appear under different slugs across NVIDIA's catalogs: support-matrix label (e.g., "Magpie TTS Multilingual"), CONTAINER_ID, NVCF function name (ai-magpie-tts-multilingual), and build.nvidia.com URL slug. Do not assume they match — cross-reference each from its own catalog. The NVCF Functions API is the only catalog you can hit programmatically; use it to resolve function-ids at runtime rather than hardcoding.

Workflow

Choose Option A (cloud) for quick testing without a GPU, or Option B (self-hosted) for production. Self-hosted follows a 4-step process: deploy container → verify health → list voices → synthesize speech.

Prerequisites

• Complete setup.md before self-hosted deployment: NVIDIA Container Toolkit, NGC_API_KEY exported, Docker logged in to nvcr.io

• Cloud-hosted inference: pip install -U nvidia-riva-client and a valid NVIDIA_API_KEY

• Not sure which TTS model to use? Run model-selection.md first

Instructions

For cloud synthesis: install nvidia-riva-client, set NVIDIA_API_KEY, fetch the model's function-id from its build.nvidia.com API page, and run talk.py against grpc.nvcf.nvidia.com:443 with --use-ssl.

For self-hosted: fetch the current CONTAINER_ID and NIM_TAGS_SELECTOR from the support matrix, then follow Steps 1–4 below.

For runtime feature questions (voice list, SSML, streaming format): fetch or open the customization page from the routing table above before answering — feature support is per-model and changes per release.

Option A — Cloud-Hosted Inference (build.nvidia.com)

Setup: pip install -U nvidia-riva-client, then clone https://github.com/nvidia-riva/python-clients and cd into it.

Auth: Set NVIDIA_API_KEY — either a build.nvidia.com personal key, or an NGC personal key with the Cloud Functions scope enabled (the same NGC key you use for docker login nvcr.io). Most users export the same value to both NVIDIA_API_KEY and NGC_API_KEY.

Server: grpc.nvcf.nvidia.com:443 — always pass --use-ssl.

Function ID lookup (JSON, scriptable, no hardcoding):

NVCF_FUNCTIONS_JSON=$(mktemp)
trap 'rm -f "$NVCF_FUNCTIONS_JSON"' EXIT

curl -fsS -H "Authorization: Bearer $NVIDIA_API_KEY" \
  --output "$NVCF_FUNCTIONS_JSON" \
  "https://api.nvcf.nvidia.com/v2/nvcf/functions?visibility=public,authorized"

python3 - "$NVCF_FUNCTIONS_JSON" <<'PY'
import json
import re
import sys

pat = re.compile(r'magpie|tts', re.I)
with open(sys.argv[1], encoding="utf-8") as response:
    functions = json.load(response).get('functions', [])
for f in functions:
    if f.get('status') == 'ACTIVE' and pat.search(f.get('name','')):
        print(f['id'], f['name'])
PY

Pick the id of the function whose name matches your model.

Function IDs and versionId rotate per release — never hardcode them; always resolve fresh via this API.

For interactive browsing only: https://build.nvidia.com/<org>/<model>/api. That page is JS-rendered and not suitable for non-browser fetch tools.

Synthesize speech:

python python-clients/scripts/tts/talk.py \
    --server grpc.nvcf.nvidia.com:443 --use-ssl \
    --metadata function-id "<FUNCTION_ID>" \
    --metadata authorization "Bearer $NVIDIA_API_KEY" \
    --language-code <LANG_CODE> \
    --text "Hello from NVIDIA TTS." \
    --voice "<VOICE_NAME>" \
    --output audio.wav

Security note: $NVIDIA_API_KEY passed as a command-line argument is visible in process listings and shell history. Prefix the command with a space (HISTCONTROL=ignorespace) or store the key in a file with chmod 600 and reference it at runtime.

List available voices: add --list-voices (drop --text, --voice, --output).

Option B — Self-Hosted TTS NIM Deployment

For TTS self-hosting, use the TTS support matrix to confirm the voice-capable model, deployment profile, and current GPU requirement before Step 1.

Step 1 — Deploy the Container

Fetch the current CONTAINER_ID and NIM_TAGS_SELECTOR for your chosen model from the support matrix.

export CONTAINER_ID=<container-id-from-support-matrix>
export NIM_TAGS_SELECTOR="<selector-from-support-matrix>"
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
sudo chown 1000:1000 $LOCAL_NIM_CACHE

docker run -it --rm --name=$CONTAINER_ID \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY \
  -e NIM_TAGS_SELECTOR \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
  nvcr.io/nim/nvidia/$CONTAINER_ID:latest

For a specific batch size, append it to NIM_TAGS_SELECTOR: name=<model>,batch_size=32.

Security note: NGC_API_KEY passed via -e NGC_API_KEY inherits from the shell environment. For production, use Docker secrets or a secrets manager.

Step 2 — Verify Readiness

If you started the container yourself, the HTTP probe is enough:

curl -fsS http://localhost:9000/v1/health/ready    # expect {"status":"ready"}

If a container was already running when you arrived (shared dev box, mystery process), the HTTP check is not sufficient — a host-mapped gRPC port can route to a container with nothing bound inside, and connections silently drop mid-RPC. Confirm a TTS model is actually being served with this inline probe (needs only pip install nvidia-riva-client):

python3 - <<'PY'
import sys, riva.client
from riva.client.proto.riva_tts_pb2 import RivaSynthesisConfigRequest
auth = riva.client.Auth(uri="0.0.0.0:50051")
try:
    cfg = riva.client.SpeechSynthesisService(auth).stub.GetRivaSynthesisConfig(
        RivaSynthesisConfigRequest(), metadata=auth.get_auth_metadata())
except Exception as e:
    print(f"UNHEALTHY: {e}"); sys.exit(2)
if not cfg.model_config:
    print("UNHEALTHY: server responded but exposes no TTS models"); sys.exit(2)
print(f"OK: {len(cfg.model_config)} model(s)")
for m in cfg.model_config:
    voices = m.parameters.get("voice_name", "")
    langs  = m.parameters.get("language_code", "")
    print(f"  - {m.model_name}  [{langs}]  voices={voices}")
PY

An empty model list or UNAVAILABLE: Socket closed means the server is not actually running TTS — restart the NIM rather than continuing.

Step 3 — List Available Voices

Voice names and pattern conventions vary per model — always discover at runtime:

gRPC:

python3 python-clients/scripts/tts/talk.py \
  --server 0.0.0.0:50051 \
  --list-voices

HTTP:

curl -sS http://localhost:9000/v1/audio/list_voices | python3 -m json.tool

Use the voice names returned by the running NIM rather than memorized strings — they may change between model versions.

Step 4 — Run Speech Synthesis

Quick path — inline (no separate scripts, no upstream coupling)

This recipe uses only the nvidia-riva-client pip package — no python-clients clone, no docker exec, no vendored scripts. It travels with this SKILL.md, so any update to the skill includes the latest recipe.

Cloud — discover function-id, then synthesize:

NVCF_FUNCTIONS_JSON=$(mktemp)
trap 'rm -f "$NVCF_FUNCTIONS_JSON"' EXIT

curl -fsS -H "Authorization: Bearer $NVIDIA_API_KEY" \
  --output "$NVCF_FUNCTIONS_JSON" \
  "https://api.nvcf.nvidia.com/v2/nvcf/functions?visibility=public,authorized"

FID=$(python3 - "$NVCF_FUNCTIONS_JSON" <<'PY'
import json
import sys

with open(sys.argv[1], encoding="utf-8") as response:
    functions = json.load(response).get('functions', [])
for f in functions:
    if f.get('status') == 'ACTIVE' and f.get('name','').removeprefix('ai-') == 'magpie-tts-multilingual':
        print(f['id']); break
PY
)

test -n "$FID" || { echo "No matching active NVCF function found" >&2; exit 1; }

# Replace VOICE with a value returned by --list-voices.
TEXT="Hello from NVIDIA TTS." OUT=out.wav SERVER=grpc.nvcf.nvidia.com:443 \
VOICE="<voice-name-from-list-voices>" FID=$FID python3 - <<'PY'
import os, wave, riva.client
server = os.environ["SERVER"]
is_cloud = "nvcf" in server
md = None
if is_cloud:
    md = [["function-id", os.environ["FID"]],
          ["authorization", f"Bearer {os.environ['NVIDIA_API_KEY']}"]]
auth = riva.client.Auth(uri=server, use_ssl=is_cloud, metadata_args=md)
tts = riva.client.SpeechSynthesisService(auth)
sr = 44100
resp = tts.synthesize(
    text=os.environ["TEXT"],
    voice_name=os.environ["VOICE"],
    language_code="en-US",
    encoding=riva.client.AudioEncoding.LINEAR_PCM,
    sample_rate_hz=sr,
)
with wave.open(os.environ["OUT"], "wb") as w:
    w.setnchannels(1); w.setsampwidth(2); w.setframerate(sr)
    w.writeframes(resp.audio)
print(f"wrote {os.environ['OUT']} ({len(resp.audio)} bytes @ {sr} Hz)")
PY

Self-hosted: drop FID=... and set SERVER=0.0.0.0:50051 — the heredoc auto-skips the cloud metadata. To discover voices, see Step 3's probe output or use the list_voices HTTP endpoint.

Alternative — upstream python-clients CLI

https://github.com/nvidia-riva/python-clients ships canonical talk.py and realtime_tts_client.py. Useful for richer CLI flags or interactive exploration:

PY_CLIENTS=~/.cache/riva-skills/python-clients
[ -d "$PY_CLIENTS" ] || git clone --depth 1 https://github.com/nvidia-riva/python-clients "$PY_CLIENTS"

python3 "$PY_CLIENTS/scripts/tts/talk.py" \
  --server grpc.nvcf.nvidia.com:443 --use-ssl \
  --metadata function-id "$FID" \
  --metadata authorization "Bearer $NVIDIA_API_KEY" \
  --text "Hello." --voice "<voice-name-from-list-voices>" \
  --output out.wav

Note. python-clients tags are stale (last tag is r2.19.0 while pip ships much newer) — always use main, which git clone --depth 1 pulls by default. If main briefly outpaces your installed nvidia-riva-client and a script fails with ImportError, fall back to the inline Quick path above (it depends only on the pip package).

Offline Synthesis (Full Audio in One Response)

gRPC:

python3 python-clients/scripts/tts/talk.py \
  --server 0.0.0.0:50051 \
  --language-code <LANG_CODE> \
  --text "Deploy and run speech synthesis with NVIDIA TTS NIM." \
  --voice <VOICE_NAME> \
  --output output.wav

HTTP:

curl -sS http://localhost:9000/v1/audio/synthesize --fail-with-body \
  -F language=<LANG_CODE> \
  -F text="Deploy and run speech synthesis with NVIDIA TTS NIM." \
  -F voice=<VOICE_NAME> \
  --output output.wav

Streaming Synthesis (Lower Latency, Audio Chunks)

gRPC:

python3 python-clients/scripts/tts/talk.py \
  --server 0.0.0.0:50051 \
  --language-code <LANG_CODE> \
  --text "..." \
  --voice <VOICE_NAME> \
  --stream \
  --output output.wav

HTTP (returns raw LPCM, not WAV — wrap with sox):

curl -sS http://localhost:9000/v1/audio/synthesize_online --fail-with-body \
  -F language=<LANG_CODE> \
  -F text="..." \
  -F voice=<VOICE_NAME> \
  -F sample_rate_hz=22050 \
  --output output.raw

sox -b 16 -e signed -c 1 -r 22050 output.raw output.wav

WebSocket / Realtime API

For OpenAI-realtime-compatible WebSocket TTS sessions, the realtime API has its own request / response shape and custom_configuration keys. Fetch the realtime API reference cited in the routing table for current event names, payload schemas, and supported keys.

python3 python-clients/scripts/tts/realtime_tts_client.py \
  --server localhost:9000 \
  --language-code <LANG_CODE> \
  --text "..." \
  --voice <VOICE_NAME> \
  --output output.wav

Helm Deployment (Kubernetes)

# custom-values.yaml
image:
  repository: nvcr.io/nim/nvidia/<container-id>
  pullPolicy: IfNotPresent
  tag: latest
nim:
  ngcAPISecret: ngc-api
imagePullSecrets:
  - name: ngc-secret
envVars:
  NIM_TAGS_SELECTOR: "<selector-from-support-matrix>"

helm install riva-tts <chart> -f custom-values.yaml

Key Parameters for talk.py

For the full flag list (and any per-model behavior), see the customization page or run talk.py --help.

Examples

Cloud synthesis (replace <FUNCTION_ID> and <VOICE_NAME> with values fetched from the model's build.nvidia.com page and --list-voices):

python python-clients/scripts/tts/talk.py \
    --server grpc.nvcf.nvidia.com:443 --use-ssl \
    --metadata function-id "<FUNCTION_ID>" \
    --metadata authorization "Bearer $NVIDIA_API_KEY" \
    --text "Hello from NVIDIA TTS." \
    --voice "<VOICE_NAME>" \
    --output audio.wav

Self-hosted offline synthesis:

python3 python-clients/scripts/tts/talk.py \
  --server 0.0.0.0:50051 \
  --text "Deploy speech synthesis with NVIDIA TTS NIM." \
  --voice <VOICE_NAME> \
  --output output.wav

Runtime feature lookup — agent flow: When a user asks "does Magpie support SSML?" or "what voices are available for English?", the agent should:

1. Fetch or open the customization page (or the support matrix for voice lists)

2. Answer based on the fetched content

Do not answer feature/voice questions from this skill's text alone.

Troubleshooting

• gRPC 4 MB limit — if synthesized audio exceeds 4 MB, switch to --stream or use the WebSocket client.

• HTTP streaming returns raw LPCM — not a WAV file; use sox to convert.

• Voice name not recognized — voice strings are case-sensitive and per-model. Always run --list-voices against the running NIM rather than copying from documentation.

• Function-id rejected by cloud — fetch the model's current API page on build.nvidia.com; function IDs rotate.

Limitations

• x86_64 architecture only — ARM is not supported

• Self-hosted deployment requires an NVIDIA AI Enterprise license

• Cloud-hosted inference requires an active NVIDIA_API_KEY and internet access

• gRPC responses are limited to 4 MB — long synthesis requests must use streaming or be chunked

• HTTP streaming returns raw LPCM (not WAV) — requires client-side wrapping

• Voice names are case-sensitive