Doca Urom

DOCA UROM capabilities, version overlay, errors, observability, safety

Where to start: Pick the H2 anchor that matches your question (paired-contract model / Service + Worker context model / enqueue-side operation surface / plugin discovery / env preconditions / errors) and read that section end-to-end. The tables in each section are the load-bearing content; the prose around them is interpretation.

Read this file when the loader sent you here from SKILL.md. For the how of executing each pattern (the verbs configure / build / modify / run / test / debug), jump to TASKS.md. For the canonical DOCA version-handling rules that this skill layers a UROM overlay on top of, see doca-version/SKILL.md. For the RDMA transport substrate UROM rides on top of, see doca-rdma/SKILL.md.

Pattern overview

Every UROM question this skill teaches resolves into one of SIX patterns. The patterns are CLASSES — they apply across every DOCA UROM release and every host + BlueField + UROM Service combination.

Two cross-cutting rules that apply to every pattern above:

• The paired-contract model is non-negotiable. Every UROM deployment is two components: the host-side library (this skill, doca-urom) that ENQUEUES offloaded operations AND the DPU-side DOCA UROM Service that EXECUTES them. An agent that treats the host side as self-sufficient — for example by walking the user from doca_urom_* calls directly into on-wire behavior without checking the DPU-side service is even running — has the model wrong for every version of UROM. A library call that succeeds at the host-side API surface but has no service to offload to fails at the first enqueue, often with a DOCA_ERROR_NOT_PERMITTED that masks the real cause.

• Discover the version-installed + service-installed surface, do not assume. Every pattern above gates on pkg-config --modversion doca-urom (host-side build-time), the doca_urom_service_get_plugins_list query against a started Service (host-side runtime — UROM has no doca_urom_cap_* devinfo capability family; the supported plugins discovered on the DPU side ARE the capability surface), AND the DPU-side UROM Service's version reachable from the host. Quoting an operation type, an atomic op, or a collective primitive without checking those three is the most common hallucination failure mode for UROM. Exact symbol names for the operation surface are plugin- and install-bound and must be read from the headers under $(pkg-config --variable=includedir doca-common) and from the shipped samples under /opt/mellanox/doca/samples/doca_urom/, not quoted from memory.

Capabilities and modes

The two orthogonal selection axes for any UROM design from the host side are which BlueField the host is offloading to (a doca_urom_service bound, via doca_urom_service_set_dev, to the doca_dev that maps to a BlueField running the DOCA UROM Service) and which operation kind the HPC / UCX / MPI stack on top wants offloaded (one-sided data movement, atomic, active message, collective primitive — each delivered by a plugin the Worker loads). Choose both before writing any host-side enqueue code, then drill into the plugin-discovery step.

Paired-contract model — the host library and the DPU service.

The agent's rule: when the user asks "how do I deploy / start / stop / scale the DOCA UROM Service on my BlueField", that is the DPU-side service question and the right artifact is the public DOCA UROM Service guide via doca-public-knowledge-map ## DOCA services (doca-public-knowledge-map/SKILL.md#doca-services). When the user asks "how do I enqueue an operation from my host code and observe its completion", that is this skill's scope. Two distinct artifacts; two distinct surfaces.

Path selection — doca-urom vs raw doca-rdma vs neither. Before any doca-urom setup, the agent must confirm UROM is even the right path:

The two host-side context types — a doca_urom_service per BlueField, and the doca_urom_worker contexts attached to it. There is NO single doca_urom Core context and no doca_urom_create; the host-side model is two distinct DOCA Core contexts.

A host driving UROM toward more than one BlueField needs one doca_urom_service per target BlueField — there is no "global UROM context". The agent must ask which BlueField (which doca_dev, mapping to which physical BlueField actually running the UROM Service) the user intends to offload to before recommending any doca_urom_service_* / doca_urom_worker_* call. (An optional doca_urom_domain coordinates multiple Workers for plugins that implement a parallel communication model.)

Enqueue-side operation surface — plugin-defined Command tasks; exact symbols are plugin- and install-bound. The host enqueues work to a Worker as a Command task (doca_urom_worker_cmd_task_*) whose payload is interpreted by the plugin the Worker loaded; the loaded plugin running under the UROM Service on the DPU side actually executes it. The operation families below are therefore plugin families, not a fixed library call set. The agent must NAME the operation families and route to the plugin headers + samples for exact symbol names — quoting specific operation function names from memory is the most common hallucination failure mode for UROM.

The agent's rule: when the user asks "what is the exact function name for a UROM put / atomic / collective", the authoritative answer is the header under $(pkg-config --variable=includedir doca-common) and the shipped sample under /opt/mellanox/doca/samples/doca_urom/ on the user's install, NOT this skill or agent memory.

Plugin discovery — the only rule. UROM has NO doca_urom_cap_* / doca_urom_cap_get_* devinfo capability family. The capability surface is the list of plugins the DPU side supports, discovered by calling doca_urom_service_get_plugins_list on a STARTED Service (DOCA_ERROR_BAD_STATE if the Service is not running):

Configuration shape. Mandatory preconditions before doca_ctx_start() on the Service and then on each Worker: the DPU-side UROM Service is deployed and running on the BlueField the host intends to offload to; the doca_urom_service is bound via doca_urom_service_set_dev to a doca_dev that maps to that BlueField; the user / process can open that doca_dev; doca_urom_service_get_plugins_list (on the started Service) shows the plugin for the operation family the user intends to enqueue, and the Worker selected it via doca_urom_worker_set_plugins. Optional configurations (doca_urom_worker_set_max_inflight_tasks, doca_urom_service_set_max_workers, transport-type knobs inherited from the RDMA substrate) ride on top of the same plugin-discovery rule.

Version compatibility

For the canonical DOCA version-detection chain, the four-way match rule, NGC container semantics, and the headers-win-over-docs rule, see doca-version/SKILL.md. The body lives there; this skill does not duplicate it.

The UROM-specific overlay is:

• The host-side library version and the DPU-side UROM Service version must agree per the DOCA Compatibility Policy. This is the paired-contract version axis the agent must ALWAYS surface: a host-side doca-urom upgraded without the DPU-side UROM Service being upgraded (or vice versa) fails at the first enqueue, often with DOCA_ERROR_NOT_SUPPORTED for an operation family that DOES exist on one side but not the other. Surface BOTH pkg-config --modversion doca-urom on the host AND the DPU-side service version (per the public DOCA UROM Service guide via doca-public-knowledge-map ## DOCA services (doca-public-knowledge-map/SKILL.md#doca-services)); cross-check them against the DOCA Compatibility Policy. This skill provides no host-side command for obtaining the DPU-side version: obtain it through the service guide, and stop rather than claim a compatible pair if it cannot be obtained. Per the cross-cutting discovery rule in doca-version CAPABILITIES.md ## Observability (doca-version/CAPABILITIES.md#observability), the doca_urom_service_get_plugins_list result on a started Service is the runtime authority for "is this operation family's plugin available on this hardware + this DOCA install + this UROM Service", and the four-way-match check (doca-urom.pc + doca-common.pc + doca_caps --version) per doca-version CAPABILITIES.md ## Version compatibility (doca-version/CAPABILITIES.md#version-compatibility) validates only the host-side install; it does not prove that the DPU-side service version matches.

Error taxonomy

UROM-specific overlays on the cross-library DOCA_ERROR_* taxonomy. The cross-library taxonomy itself lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy); the rows below are the UROM surface meaning that the agent must disambiguate before falling back to the cross-library response.

The agent's rule: never recommend a retry loop on DOCA_ERROR_* without first identifying which of the rows above is the cause. _AGAIN is the only one that wants a retry (after doca_pe_progress()); the others want investigation — service-side, lifecycle, version-axis, RDMA substrate, or memory descriptor — not retry.

Observability

UROM observability surface is two-sided: there is a host-side observability surface (operation completions on the DOCA progress engine, the DOCA logger, capability snapshots from doca_urom_service_get_plugins_list) AND an infrastructure-side observability surface (the DPU-side UROM Service's own observability documented in the public DOCA UROM Service guide via doca-public-knowledge-map ## DOCA services (doca-public-knowledge-map/SKILL.md#doca-services), plus the underlying RDMA substrate counters documented in doca-rdma CAPABILITIES.md ## Observability (doca-rdma/CAPABILITIES.md#observability)). The agent must reach for both, not just one — a UROM enqueue that returns DOCA_SUCCESS on the host side but produces no visible remote-side effect is almost always visible on one of the infrastructure-side surfaces.

Three primary signals the agent should reach for:

1. Per-operation completion events on the DOCA progress engine. Every enqueued operation produces a completion event when it finishes (or errors) — the completion carries the doca_error_t if it failed. The host must inspect the per-operation completion, not just the return value of the enqueue call. Submitted but no completion is almost always either a missing doca_pe_progress() call in the host loop OR a DPU-side service that has stopped processing enqueues (route to the public service guide).

2. Plugin snapshot at configure time. The output of doca_urom_service_get_plugins_list on the started Service, together with pkg-config --modversion doca-urom on the host and the DPU-side UROM Service version, is the baseline of "what the library + the hardware + the DPU service said was possible" before any enqueue. Save it; if an enqueue later returns DOCA_ERROR_NOT_SUPPORTED the diff against this baseline is the bug.

3. Infrastructure-side: DPU service observability + RDMA substrate counters. When the host-side surface shows operations completing cleanly but the user reports the remote-side memory does not have the expected bytes, the right diagnostic is the DPU-side UROM Service's own observability (reach via the public service guide through doca-public-knowledge-map ## DOCA services (doca-public-knowledge-map/SKILL.md#doca-services)) PLUS the underlying RDMA counters (per doca-rdma CAPABILITIES.md ## Observability (doca-rdma/CAPABILITIES.md#observability)). The agent must NAME the existence of both surfaces and route the user there; the per-surface details belong in their owning skills / guides.

For cross-cutting observability primitives (--sdk-log-level, the DOCA_LOG_LEVEL env var, the trace build flavor) see doca-debug CAPABILITIES.md ## Observability (doca-debug/CAPABILITIES.md#observability). For the install-tree observability (logger names, package layout, sample tree) defer to doca-public-knowledge-map/SKILL.md.

Safety policy

Overlay on the bundle-wide hardware-safety meta-policy. The rules below are this skill's per-artifact overlay on the cross-cutting rules in doca-hardware-safety CAPABILITIES.md ## Safety policy (doca-hardware-safety/CAPABILITIES.md#safety-policy) (specifically ### Per-artifact overlay pattern (doca-hardware-safety/CAPABILITIES.md#per-artifact-overlay-pattern)). When the two layers disagree, the stricter wins; when either layer says STOP, the agent stops.

UROM's safety surface is env-precondition-driven AND paired-contract-driven AND RDMA-substrate-driven. The three most common UROM first-app failures are (1) the DPU-side UROM Service is not deployed and running on the BlueField the host is enqueueing to; (2) the host-side library and the DPU-side service are at versions the DOCA Compatibility Policy does not support pairing; (3) the underlying RDMA fabric between host and BlueField (or between BlueFields, for multi-node HPC) is not healthy. The agent's job is to verify all three before any host-side doca_urom_* enqueue, not after the first DOCA_ERROR_*.

The env-precondition matrix the agent must walk for any new host-side UROM setup:

Do not invent a "host-CPU fallback" for UROM. UROM's entire reason to exist is to remove host-CPU work from the communication path. If the DPU-side UROM Service is not running or is at the wrong version, the right answer is to fix that — NOT to silently fall back to raw doca-rdma posted from the host CPU. A fallback that masks the missing service produces an HPC stack whose offload claims are silently wrong; the agent must surface the service-side gap, not paper over it.

Lifecycle ordering is UROM-aware. The Worker contexts must be stopped and destroyed (doca_urom_worker_destroy) before the Service they are attached to is stopped and destroyed (doca_urom_service_destroy); both follow the universal DOCA Core teardown order on top of the doca_dev. Out-of-order teardown surfaces as DOCA_ERROR_BAD_STATE (or DOCA_ERROR_IN_USE when Workers are still attached) on subsequent calls and may leave DPU-side resources (memory registrations, worker processes, completion queues) half-released that the next doca_urom_service_create on the same device then has to recover from. The agent must surface this ordering explicitly. If a later create reports DOCA_ERROR_BAD_STATE or DOCA_ERROR_IN_USE after an out-of-order teardown, stop blind retries and route recovery to the public DOCA UROM Service guide; this host-library skill does not define a service-side cleanup command.

This skill does not define an HPC algorithm. doca-urom offloads remote memory operations; it does not implement an MPI collective, a UCX active message handler, or any particular HPC stack algorithm. When the user asks "what all-reduce algorithm should I use" or "how do I implement my collective", the agent must refuse to invent an algorithm and must route the user to the upstream MPI / UCX documentation and to the user's own HPC domain expertise.

Deferred topic boundaries

This skill scopes itself to the host-side doca-urom library for HPC / UCX / MPI workloads offloading remote memory operations to the BlueField DPU. Adjacent topics the agent will get asked but should route elsewhere:

• DOCA UROM Service deployment and operation on the DPU side — a SEPARATE artifact with its own public guide; route via doca-public-knowledge-map ## DOCA services (doca-public-knowledge-map/SKILL.md#doca-services). Conflating the library and the service is the single most common UROM first-app design error: the agent must NAME the split explicitly whenever the question straddles the two.

• MPI / UCX / HPC collective algorithm design itself — out of scope. Route to the upstream MPI / UCX documentation; this skill assumes the user has the algorithmic shape they want and is asking how to express the underlying remote memory operations through UROM.

• Setting up the RDMA / RoCE / IB transport substrate UROM rides on top of — owned by doca-rdma/SKILL.md. UROM uses the RDMA substrate; it does not stand it up. A UROM deployment over a broken RDMA fabric is a substrate problem, not a UROM bug.

• DOCA Core context and progress engine internals — owned by doca-programming-guide/SKILL.md. This skill uses the Core lifecycle; it does not redefine it.

• Cross-library DOCA_ERROR_* taxonomy — owned by doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy). This skill adds the UROM overlay, not the taxonomy itself.

• Cross-cutting debug ladder (install / version / build / link / runtime / program / driver) — owned by doca-debug ## debug (doca-debug/TASKS.md#debug). This skill's ## debug redirects there for layer 1-4 and layer 7; layers 5-6 carry the UROM-specific overlay (including the DPU-side-service-not-running route and the RDMA-substrate route).

• Cross-library doca_caps invocation patterns — owned by doca-public-knowledge-map/SKILL.md. This skill references the UROM plugin-discovery surface (doca_urom_service_get_plugins_list), which is per-library; the cross-library capability snapshot tool (doca_caps --list-devs) is a separate surface routed via the public knowledge map.