Doca Pcc

DOCA PCC capabilities, version overlay, errors, observability, safety

Where to start: Pick the H2 anchor that matches your question (host-side doca_pcc context / loaded algorithm image / attach-to-port semantics / triple-axis capability discovery / env preconditions / errors) and read that section end-to-end. The tables in each section are the load-bearing content; the prose around them is interpretation.

Read this file when the loader sent you here from SKILL.md. For the how of executing each pattern (the verbs configure / build / modify / run / test / debug), jump to TASKS.md. For the canonical DOCA version-handling rules that this skill layers a PCC overlay on top of, see doca-version/SKILL.md. For the host-side DPA lifecycle this skill conceptually inherits (custom PCC algorithms are DPA-side code), see doca-dpa/SKILL.md.

Pattern overview

Every PCC question this skill teaches resolves into one of SIX patterns. The patterns are CLASSES — they apply across every DOCA PCC release and every host + BlueField + DPACC + firmware combination.

Two cross-cutting rules apply to every pattern above, both defined in ## Capabilities and modes: the two-side-program model (host loads, parameterizes and starts; the DPA-side algorithm executes — a proposal that the host “compute the rate-update directly” has the model wrong) and triple-axis capability discovery (DOCA cap-query + DPA exposure + firmware custom-PCC slot — quoting one or two axes misses the “the doc page says custom PCC exists but doca_pcc_* returns DOCA_ERROR_NOT_PERMITTED” cases).

Capabilities and modes

The two orthogonal selection axes for any custom-PCC design from the host side are which BlueField port whose RDMA / RoCE traffic the algorithm controls (doca_pcc per doca_dev that maps to a DPA-capable BlueField with the firmware custom-PCC slot enabled) and which PCC algorithm image (the doca_pcc_app produced by dpacc from the user's DPA-side algorithm source) the host is going to load. Choose both before writing any host-side load code, then drill into the relevant capability-query.

Two-side-program model — the host side and the DPA side.

The agent's rule: when the user asks "what should my algorithm compute", that is a domain question (research / workload tuning), NOT an API question — route via doca-public-knowledge-map/SKILL.md to the public DOCA PCC programming guide and to the user's own congestion-control expertise. When the user asks "how do I load my algorithm from the host and attach it to a port", that is this skill's scope. Two distinct questions; two distinct surfaces.

Path selection — custom PCC vs default firmware PCC vs counter CLI. Before any doca-pcc setup, the agent must confirm doca-pcc is even the right artifact:

The per-PCC-instance doca_pcc context — one per BlueField port the host is driving.

A host driving custom PCC on more than one BlueField port needs one doca_pcc per port — there is no "global PCC context". The agent must ask which BlueField port (which doca_dev, mapping to which physical port handling the user's RDMA / RoCE traffic) the user intends to control before recommending any doca_pcc_* call.

The loaded PCC algorithm image (doca_pcc_app) — the output of dpacc made visible to the host.

The agent's rule: the host's view of "which custom PCC algorithm is running on this port" is exactly the set of entry points in the loaded doca_pcc_app. If the user is "trying to attach an algorithm whose entry points aren't in the image", that is a build-side question — go back to ## build and to the DPACC guide via doca-public-knowledge-map/SKILL.md, not a host-side load-call fix.

Attach-to-port semantics — the algorithm binds to a BlueField port carrying RDMA / RoCE.

Triple-axis capability discovery — the only rule. Before deploying any custom PCC algorithm, run ALL THREE of: a DOCA cap-query against the active doca_devinfo, confirm the BlueField generation actually exposes the DPA processor (the algorithm runs on the DPA), AND confirm the BlueField firmware has the custom-PCC slot enabled. Any axis missing the support fails the deployment.

Device-side declaration workflow (DOCA 3.5). The public PCC header adds a declaration stage that must be completed before doca_pcc_start(). The host declares the device-side resource layout with the doca_pcc_dev_declare_* family: number of counter groups, number of histograms, threads per counter set, per-algorithm parameter and counter counts, and slot-to-algorithm bindings. Histogram IDs are registered per algorithm with doca_pcc_dev_algo_register_hist_ids, counter types are set with doca_pcc_dev_set_counter_type, and the declared user memory is committed with doca_pcc_dev_user_declare_memory. The public PCC header is the authoritative source for the exact call order and required arguments; do not guess the sequence from function names alone.

Histograms are declared with a mode from the DOCA_PCC_DEV_HISTOGRAM_MODE_* family in the public PCC header. The family selects linear, exponential, or free histogram binning; the histogram init entry point takes the mode, edges, bin count, port, target app, counter group ID, histogram ID, and description. The same header defines the doca_pcc_dev_histogram_* lifecycle entry points for enable, disable, clear, update, and description retrieval.

Configuration shape. Mandatory preconditions before any doca_pcc_start(): the doca_pcc context must be created (via doca_pcc_create) against a doca_dev that maps to a DPA-capable BlueField port whose firmware has the custom-PCC slot enabled; the PCC algorithm image (doca_pcc_app) compiled by dpacc must be loaded into the doca_pcc; the device-side declaration workflow from the public PCC header must be completed (declare counter groups, histograms, threads per counter set, per-algorithm parameter/counter counts, histogram IDs, counter types, slot bindings, and user-declared memory); the algorithm's host-side parameters (whatever knobs the algorithm exposes) must be set; the doca_pcc_cap_* queries and the doca_pcc_dev_get_* limit getters must agree the deployment is supported on this combo. Optional configurations (algorithm parameter retunes, observability queue sizing) are program-side tunables that ride on top of the same cap-query rule.

Version compatibility

For the canonical DOCA version-detection chain, the four-way match rule, NGC container semantics, and the headers-win-over-docs rule, see doca-version/SKILL.md. The body lives there; this skill does not duplicate it.

The PCC-specific overlay is:

• DOCA must match the DPACC compiler per the DOCA Compatibility Policy. The DPACC compiler is the build-time component that turns the user's DPA-side PCC algorithm source into the binary embedded in the host executable as a doca_pcc_app; the host-side runtime that loads it is doca-pcc. Mismatched DOCA + DPACC versions fail at link time (missing symbols on either side) or at load time (DOCA_ERROR_DRIVER or DOCA_ERROR_INVALID_VALUE from the host-side load call) in ways that look like hardware bugs but are version-skew bugs. This is the same overlay doca-dpa CAPABILITIES.md ## Version compatibility (doca-dpa/CAPABILITIES.md#version-compatibility) documents for DPA itself, because the custom PCC algorithm is DPA-side code. The agent must surface BOTH pkg-config --modversion doca-pcc AND the installed dpacc version, cross-check them against the DOCA Compatibility Policy at https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html, and route any disagreement to doca-version TASKS.md ## debug (doca-version/TASKS.md#debug) before any PCC-layer diagnosis. Per the cross-cutting cap-query rule in doca-version CAPABILITIES.md ## Observability (doca-version/CAPABILITIES.md#observability), the doca_pcc_cap_* query against the active doca_devinfo is the runtime authority for "is this custom PCC deployment supported on this hardware + this DOCA install + this firmware", and the four-way-match check (doca-pcc.pc plus doca-common.pc plus the matching DPACC) per doca-version CAPABILITIES.md ## Version compatibility (doca-version/CAPABILITIES.md#version-compatibility) catches the DOCA upgraded but DPACC didn't partial-install pattern before it surfaces as a load failure.

• DOCA 3.5 expanded the public API surface. The 3.5 release added the doca_pcc_dev_* declaration, histogram, and counter families, the doca_pcc_* thread and log families, and the DOCA_PCC_DEV_HISTOGRAM_MODE_* constants. Code written against the pre-3.5 API must be recompiled against the 3.5 public headers; code that uses the new declaration/histogram/counter calls requires DOCA 3.5 or later and will not link or load on older DOCA. Treat the presence or absence of these families as a version signal: a build that references the new families is a 3.5+ build, and a 3.5 library paired with pre-3.5 headers (or vice versa) is the same version-skew failure class as the DPACC mismatch above. The authoritative declarations are in the doca-pcc public headers shipped with the component; do not infer the API from older documentation.

Error taxonomy

PCC-specific overlays on the cross-library DOCA_ERROR_* taxonomy. The cross-library taxonomy itself lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy); the rows below are the PCC surface meaning that the agent must disambiguate before falling back to the cross-library response.

The agent's rule: never recommend a retry loop on DOCA_ERROR_* without first identifying which of the rows above is the cause. None of the PCC errors above want a retry — every row points at investigation (env / firmware / version / lifecycle / two-side-program signature mismatch), not at retry.

Observability

PCC observability surface is two-sided: there is a host-side observability surface (what the running algorithm reports back to the host through the host-side doca-pcc API, the DOCA logger, cap-query snapshots, and the PCC log file path get/set APIs doca_pcc_log_file_get / doca_pcc_log_file_set) AND an infrastructure-side observability surface (the pcc_counters diagnostic CLI documented in the public DOCA Tools umbrella reachable via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools) plus the DPA-side developer tools inherited from doca-dpa CAPABILITIES.md ## Observability (doca-dpa/CAPABILITIES.md#observability)). The agent must reach for both, not just one — a custom PCC algorithm that loads cleanly but produces no visible change in RDMA / RoCE traffic is almost always visible on one of the infrastructure-side tools.

Three primary signals the agent should reach for:

1. Host-side reports surfaced by the running algorithm. Whatever observability the host-side doca-pcc API exposes (algorithm-defined report stream, per-event callbacks, status counters) is the first place to look when the user reports "the algorithm loaded but I'm not sure it's doing anything". Missing expected reports can result from the host PE not being progressed, no DPA-side report-emission path, or a host report queue that is not drained and becomes full. Check those in that order before attributing the issue to the algorithm body. As of DOCA 3.5, the host-side API also exposes explicit PCC log file path control through doca_pcc_log_file_get and doca_pcc_log_file_set; the developer must use these to direct PCC logs to a known file and to retrieve the current path before collecting logs, rather than relying only on the generic DOCA logger settings.

2. Capability snapshot at configure time. The output of doca_pcc_cap_* against the active doca_devinfo together with the installed pkg-config --modversion doca-pcc and the installed dpacc version is the baseline of "what the library + the hardware + the firmware + the DPACC compiler said was possible" before any algorithm was loaded. Save it; if a runtime call later returns DOCA_ERROR_NOT_SUPPORTED or DOCA_ERROR_NOT_PERMITTED the diff against this baseline is the bug.

3. Infrastructure-side counter tool + DPA-side developer tools (route via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools)). When the host-side surface shows the algorithm is loaded and reports look fine but the user's RDMA / RoCE traffic shows no change in congestion behavior, the pcc_counters CLI is the diagnostic the agent should route to for read-only inspection of PCC counters at the port. For DPA-side hangs (the algorithm body is running but stuck), the DPA-side developer tools named in doca-dpa CAPABILITIES.md ## Observability (doca-dpa/CAPABILITIES.md#observability) apply. The agent must NAME the existence of these tools and route the user there; the per-tool surface is out of scope for this skill.

For cross-cutting observability primitives (--sdk-log-level, the DOCA_LOG_LEVEL env var, the trace build flavor) see doca-debug CAPABILITIES.md ## Observability (doca-debug/CAPABILITIES.md#observability). For the install-tree observability (logger names, package layout, sample tree) defer to doca-public-knowledge-map/SKILL.md.

Safety policy

Overlay on the bundle-wide hardware-safety meta-policy. The rules below are this skill's per-artifact overlay on the cross-cutting rules in doca-hardware-safety CAPABILITIES.md ## Safety policy (doca-hardware-safety/CAPABILITIES.md#safety-policy) (specifically ### Per-artifact overlay pattern (doca-hardware-safety/CAPABILITIES.md#per-artifact-overlay-pattern)). When the two layers disagree, the stricter wins; when either layer says STOP, the agent stops.

PCC's safety surface is env-precondition-driven AND two-side-program-driven AND firmware-configuration-driven. The three most common PCC first-app failures are (1) the host's BlueField generation does not expose a DPA processor to the host at all; (2) the BlueField firmware does not have the custom-PCC slot enabled; and (3) the host-side load call and the DPA-side algorithm image disagree because one side was rebuilt while the other was not. The agent's job is to verify all three before any host-side load call, not after the first DOCA_ERROR_DRIVER.

The env-precondition matrix the agent must walk for any new host-side custom PCC setup:

Do not partial-rebuild one side. A host-side rebuild against a new DOCA install with the PCC algorithm image still built by an old dpacc, or a DPA-side rebuild without rebuilding the host that embeds it, fails PCC in non-obvious ways: the host load call may succeed and the algorithm may silently misbehave on the wire or return DOCA_ERROR_DRIVER at load. The fix is to rebuild both sides against the matched DOCA + DPACC versions per the DOCA Compatibility Policy, not to silence the error.

Lifecycle ordering is PCC-aware. The loaded doca_pcc_app must be released BEFORE the parent doca_pcc is destroyed; stop the instance with doca_pcc_stop, release the loaded app, destroy the instance with doca_pcc_destroy, then close the doca_dev. Do not use doca_ctx_stop or doca_ctx_destroy for this object. Out-of-order teardown surfaces as DOCA_ERROR_BAD_STATE on subsequent calls and may also leave the BlueField port with a half-configured custom PCC state that the firmware has to recover from on the next start — the agent must surface this ordering explicitly.

This skill does not define an algorithm. doca-pcc loads a congestion control algorithm the user supplies; it does not implement one. When the user asks "what algorithm should I write", the agent must refuse to invent an algorithm body and must route the user to the public DOCA PCC programming guide via doca-public-knowledge-map/SKILL.md and to the user's own congestion-control domain expertise.

Deferred topic boundaries

This skill scopes itself to the host-side doca-pcc library for custom PCC algorithms. Adjacent topics the agent will get asked but should route elsewhere:

• DPA-side congestion-control algorithm design itself (the algorithm body that runs on the DPA processor, the per-event entry-point set, the DPA-side memory model the algorithm uses, congestion-control theory) — outside this skill. Route to the public DOCA PCC programming guide and the DPACC compiler guide via doca-public-knowledge-map/SKILL.md; this skill assumes the user has the DPA-side algorithm and is asking how to load and attach it from the host.

• pcc_counters diagnostic CLI (the real artifact is the pcc_counters.sh script installed at /opt/mellanox/doca/tools/pcc_counters.sh) — a separate artifact with its own public page; route via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools). It is read-only inspection; this skill is control + load. Conflating the tool with the library is the single most common PCC first-app design error.

• Default factory PCC algorithm shipped in ConnectX firmware — does not need this library. Configuration of the factory PCC is a firmware-side concern; route via doca-public-knowledge-map/SKILL.md.

• Generic DPA workloads that are not congestion control — doca-dpa/SKILL.md (see SKILL.md ## Related skills).

• DPA developer tools (the DPA debugger, the DPA process-state inspector, the DPA statistics tool) — named in ## Observability for routing, but the per-tool surface lives in the public DPA Tools umbrella via doca-public-knowledge-map/SKILL.md.

• DPACC compiler internals (flags, target options, how the host + DPA split-build is wired) — out of scope. Route to the public DOCA DPACC Compiler guide via doca-public-knowledge-map/SKILL.md.

• RDMA / RoCE traffic bring-up, Core context internals, the cross-cutting DOCA_ERROR_* taxonomy and the cross-cutting debug ladder — owned by doca-rdma, doca-programming-guide and doca-debug; see SKILL.md ## Related skills. A doca-pcc deployment with no RDMA / RoCE traffic on the attached port is a pre-flight gap, not a doca-pcc bug. This skill's ## debug carries the PCC overlay for layers 5-7 (firmware custom-PCC slot, two-side-program parameter-shape mismatch).