Nemotron Speech

Riva NMT NIM

Agent: When walking the user through a multi-step workflow, announce each step before presenting it: Step N/M — Step Title (e.g., "Step 1/4 — Deploy the Container").

Source of truth. This skill describes deployment mechanics, which are stable across releases. For anything that varies per release — model catalog, container IDs, supported language pairs, feature support, VRAM minimums — fetch or open the canonical doc page and answer from that, not from this skill's text. See Looking up current information below.

Purpose

Deploy and run NVIDIA Riva NMT (neural machine translation) NIMs for bidirectional text translation. Covers container deployment, inference modes (basic, batch, do-not-translate tags), and Helm deployment.

Looking up current information

This skill is orientation, not catalog. When a question depends on data that changes per release, fetch or open the relevant page and answer from that page:

Do not infer from this skill's text: which models exist, which language pairs are supported, what CONTAINER_ID to use, which decoders honor <dnt> tags, or what VRAM is required. The docs are the contract. Always run --list-models against a running NIM to see the exact language codes that server accepts.

Naming caveat. The same model can appear under different slugs across NVIDIA's catalogs: support-matrix label (e.g., "Megatron 1B NMT", "Riva Translate 1.6B"), CONTAINER_ID, NVCF function name (ai-megatron-1b-nmt, ai-riva-translate-1_6b), and build.nvidia.com URL slug. Do not assume they match — cross-reference each from its own catalog. The NVCF Functions API is the only catalog you can hit programmatically; use it to resolve function-ids at runtime rather than hardcoding.

Workflow

Use this reference after model selection for NMT-specific commands: local deployment, readiness checks, language-pair discovery, translation, and optional Helm deployment.

Prerequisites

• Self-hosted: complete setup.md first — NVIDIA Container Toolkit, NGC_API_KEY exported, Docker logged in to nvcr.io. Driver, GPU, and VRAM minimums change per release — fetch the support matrix and prerequisites pages cited above before deploying.

• Cloud (build.nvidia.com): pip install -U nvidia-riva-client and a valid NVIDIA_API_KEY. The same NGC personal key works for both planes if it was issued with the Cloud Functions scope; most users export the same value to both NVIDIA_API_KEY and NGC_API_KEY. No GPU needed.

Instructions

• Deploy the NMT container when using a self-hosted NIM.

• Verify server readiness before running clients.

• List available language pairs from the running server.

• Run translation with the selected source and target language codes.

• Use the Helm section only for production Kubernetes deployments.

For runtime feature questions (<dnt> tags, custom dictionaries, max-length variation, supported language pairs): fetch or open the customization page from the routing table above before answering.

Step 1 — Deploy the Container

Fetch the current CONTAINER_ID from the support matrix.

export CONTAINER_ID=<container-id-from-support-matrix>
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
sudo chown 1000:1000 $LOCAL_NIM_CACHE

docker run -it --rm --name=$CONTAINER_ID \
  --runtime=nvidia \
  --gpus '"device=0"' \
  --shm-size=8GB \
  -e NGC_API_KEY \
  -e NIM_HTTP_API_PORT=9000 \
  -e NIM_GRPC_API_PORT=50051 \
  -p 9000:9000 \
  -p 50051:50051 \
  -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
  nvcr.io/nim/nvidia/$CONTAINER_ID:latest

NIM_TAGS_SELECTOR is optional for NMT — the container selects the best profile for your GPU automatically.

Security note: Environment variables passed via -e to Docker are visible in docker inspect output and process listings. For production, use Docker secrets or a secrets manager instead of passing credentials as env vars.

Step 2 — Verify Readiness

If you started the container yourself, the HTTP probe is enough:

curl -fsS http://localhost:9000/v1/health/ready    # expect {"status":"ready"}

If a container was already running when you arrived (shared dev box, mystery process), the HTTP check is not sufficient — a host-mapped gRPC port can route to a container with nothing bound inside, and connections silently drop mid-RPC. Confirm an NMT model is actually being served with this inline probe (needs only pip install nvidia-riva-client):

python3 - <<'PY'
import sys, riva.client
auth = riva.client.Auth(uri="0.0.0.0:50051")
nmt = riva.client.NeuralMachineTranslationClient(auth)
try:
    cfg = nmt.get_config("")    # empty model name returns all loaded models
except Exception as e:
    print(f"UNHEALTHY: {e}"); sys.exit(2)
if not cfg.languages:
    print("UNHEALTHY: server responded but exposes no NMT models"); sys.exit(2)
print(f"OK: {len(cfg.languages)} model(s)")
for name, langs in cfg.languages.items():
    s = ",".join(list(langs.src_lang)[:5])
    t = ",".join(list(langs.tgt_lang)[:5])
    print(f"  - {name}  src=[{s}...]  tgt=[{t}...]")
PY

An empty model list or UNAVAILABLE: Socket closed means the server is not actually running NMT — restart the NIM rather than continuing.

Step 3 — List Available Models and Language Pairs

The set of supported language pairs is per-model and discovered at runtime:

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --list-models

Use these codes verbatim in --source-language-code / --target-language-code — the documented codes may use slightly different casing or hyphenation than what the server returns.

Step 4 — Run Translation

Quick path — inline (no separate scripts, no upstream coupling)

This recipe uses only the nvidia-riva-client pip package — no python-clients clone, no docker exec, no vendored scripts. It travels with this SKILL.md, so any update to the skill includes the latest recipe.

Cloud — discover function-id, then translate:

First, discover the function-id. Pick a specific model rather than relying on a broad regex — multiple NMT functions are typically active and some may be paused or returning 502 at any given time. To list everything currently active:

NVCF_FUNCTIONS_JSON=$(mktemp)
trap 'rm -f "$NVCF_FUNCTIONS_JSON"' EXIT

curl -fsS -H "Authorization: Bearer $NVIDIA_API_KEY" \
  --output "$NVCF_FUNCTIONS_JSON" \
  "https://api.nvcf.nvidia.com/v2/nvcf/functions?visibility=public,authorized"

python3 - "$NVCF_FUNCTIONS_JSON" <<'PY'
import json
import re
import sys

pat = re.compile(r'nmt|translate|megatron-nmt|seamless', re.I)
with open(sys.argv[1], encoding="utf-8") as response:
    functions = json.load(response).get('functions', [])
for f in functions:
    if f.get('status') == 'ACTIVE' and pat.search(f.get('name','')):
        print(f['id'], f['name'])
PY

Pick the id of the function whose name matches your model. Function IDs rotate per release — never hardcode them; always resolve fresh via this API.

For interactive browsing only: https://build.nvidia.com/<org>/<model>/api. That page is JS-rendered and not suitable for non-browser fetch tools.

Then anchor on a specific name (replace riva-translate-1_6b with whichever you picked):

NVCF_FUNCTIONS_JSON=$(mktemp)
trap 'rm -f "$NVCF_FUNCTIONS_JSON"' EXIT

curl -fsS -H "Authorization: Bearer $NVIDIA_API_KEY" \
  --output "$NVCF_FUNCTIONS_JSON" \
  "https://api.nvcf.nvidia.com/v2/nvcf/functions?visibility=public,authorized"

FID=$(python3 - "$NVCF_FUNCTIONS_JSON" <<'PY'
import json
import sys

with open(sys.argv[1], encoding="utf-8") as response:
    functions = json.load(response).get('functions', [])
for f in functions:
    if f.get('status') == 'ACTIVE' and f.get('name','').removeprefix('ai-') == 'riva-translate-1_6b':
        print(f['id']); break
PY
)

test -n "$FID" || { echo "No matching active NVCF function found" >&2; exit 1; }

TEXT="Hello, how are you today?" SRC=en TGT=de SERVER=grpc.nvcf.nvidia.com:443 FID=$FID python3 - <<'PY'
import os, riva.client
server = os.environ["SERVER"]
is_cloud = "nvcf" in server
md = None
if is_cloud:
    md = [["function-id", os.environ["FID"]],
          ["authorization", f"Bearer {os.environ['NVIDIA_API_KEY']}"]]
auth = riva.client.Auth(uri=server, use_ssl=is_cloud, metadata_args=md)
nmt = riva.client.NeuralMachineTranslationClient(auth)
resp = nmt.translate(
    texts=[os.environ["TEXT"]],
    model="",
    source_language=os.environ["SRC"],
    target_language=os.environ["TGT"],
)
for t in resp.translations:
    print(t.text)
PY

Self-hosted: drop FID=... and set SERVER=0.0.0.0:50051 — the heredoc auto-skips the cloud metadata. Pass model="<model-name-from-probe>" if the running NIM serves more than one NMT model.

For language codes accepted by a specific server, use the inline probe from Step 2 — codes are server-defined and may differ from documented values.

Alternative — upstream python-clients CLI

https://github.com/nvidia-riva/python-clients ships a canonical nmt.py with richer CLI flags (batch translation from file, --dnt-phrases-file, --max-len-variation, --list-models). Useful for one-off interactive exploration:

PY_CLIENTS=~/.cache/riva-skills/python-clients
[ -d "$PY_CLIENTS" ] || git clone --depth 1 https://github.com/nvidia-riva/python-clients "$PY_CLIENTS"

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text "This will become German words." \
  --source-language-code en-US \
  --target-language-code de-DE

Output has a ## prefix added by the client script (not the model).

Note. python-clients tags are stale (last tag is r2.19.0 while pip ships much newer) — always use main, which git clone --depth 1 pulls by default. If main briefly outpaces your installed nvidia-riva-client and a script fails with ImportError, fall back to the inline Quick path above (it depends only on the pip package).

Protect Terms from Translation (<dnt> tags)

Wrap terms in <dnt>...</dnt> to prevent them from being translated. <dnt> tag support is per-model — verify on the customization page before relying on it.

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text "<dnt>NVIDIA NIM</dnt> provides optimized inference." \
  --source-language-code en-US \
  --target-language-code fr-FR

For a list of phrases to protect, use a custom dictionary file with --dnt-phrases-file.

Batch Translation from File

Translate multiple lines (one input per line in the file):

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text-file input_text.txt \
  --source-language-code en \
  --target-language-code de \
  --batch-size 8

Morphologically Complex Languages

For target languages that produce longer output than the source (Arabic, Turkish, Finnish, Hungarian, etc.), increase --max-len-variation to prevent truncation. The exact recommended values per language live on the customization page; tune empirically:

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text "Despite numerous challenges, several countries committed to net-zero by 2050." \
  --source-language-code en-US \
  --target-language-code ar-AR \
  --max-len-variation 150

Default is 20; range is 0–256. Higher values allow longer output but can increase latency.

Key Parameters

For the full flag list, run nmt.py --help.

Helm Deployment (Kubernetes)

# custom-values.yaml
image:
  repository: nvcr.io/nim/nvidia/<container-id-from-support-matrix>
  pullPolicy: IfNotPresent
  tag: latest
nim:
  ngcAPISecret: ngc-api
imagePullSecrets:
  - name: ngc-secret
envVars:
  NIM_TAGS_SELECTOR: "name=<container-id-from-support-matrix>"

helm install riva-nmt <chart> -f custom-values.yaml

Examples

Translate English to German:

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text "Hello, world." \
  --source-language-code en-US \
  --target-language-code de-DE

Protect a brand name from translation:

python3 "$PY_CLIENTS/scripts/nmt/nmt.py" \
  --server 0.0.0.0:50051 \
  --text "<dnt>NVIDIA NIM</dnt> provides optimized inference." \
  --source-language-code en-US \
  --target-language-code fr-FR

Runtime feature lookup — agent flow: When a user asks "does Riva NMT support Hindi?" or "can I use a DNT dictionary?", the agent should:

1. Fetch or open the support matrix (for language pairs) or the customization page (for feature behavior)

2. Answer based on the fetched content

Do not answer language-pair or feature questions from this skill's text alone.

Troubleshooting

• --text and --text-file are mutually exclusive — use one or the other; they cannot be combined.

• ## prefix in output — added by the client script, not the model; strip programmatically if needed.

• Truncation on long output — increase --max-len-variation (try 100–200 for Arabic, Turkish, Finnish).

• Language code rejected — codes are server-defined; run --list-models and use the values it returns verbatim. Documented codes may differ from server-accepted codes between releases.

• Container ID not recognized — fetch the current value from the support matrix. Names rotate between releases.

Limitations

• x86_64 architecture only; NVIDIA AI Enterprise license required for self-hosting

• Morphologically complex languages may require a higher --max-len-variation value (see Troubleshooting)

• Language pair availability and DNT support are per-model — verify on the support matrix and customization page before assuming a pair / feature is available