Doca Pcc

DOCA PCC workflows

Where to start: The verbs run configure → build → modify → run → test → debug. Skip ahead only when the user is already past a verb. The ## test verb is an iterative loop (single small-algorithm smoke → traffic-affecting smoke → sustained run → loop back if a precondition changes), not a one-shot pass — see the eval-loop overlay in ## test below.

Read this file when the loader sent you here from SKILL.md. For the PCC capability surface, the two-side-program model, the doca_pcc per-instance context, the loaded PCC algorithm image, the attach-to-port semantics, the triple-axis capability-discovery rule, the env-precondition matrix, the error taxonomy, the observability surface, and the safety policy, see CAPABILITIES.md. For the cross-library DOCA patterns layered under everything below (the universal Core lifecycle, the cross-library DOCA_ERROR_* taxonomy, the modify-a-shipped-sample workflow), see doca-programming-guide/SKILL.md.

Each verb below describes the shape of the workflow, not a copy-paste recipe. The agent's job is to walk the user through the steps in order, verifying preconditions before recommending the next call.

Loads fine but no on-wire change

A load that returned success has already cleared the install, version, build, link and custom-PCC-slot layers, so do not open with the install → driver walk or with the algorithm body. Check the host side in this order:

1. doca_pcc_start() never called. The algorithm is loaded but inert; a load without a start produces exactly this symptom.

2. Wrong doca_dev. The doca_dev the doca_pcc was created against is not the port carrying the user's RoCE traffic.

3. No RDMA / RoCE traffic on that port to modulate (route to doca-rdma).

Reports firing does not distinguish these: the body runs in all three. The DPA-side algorithm body is a later, conditional cause owned by doca-dpa and must not lead. Layer 5 below has the detail.

configure

Goal: load, parameterize and start the user's PCC algorithm on the port.

Steps the agent should walk the user through:

1. Confirm the triple-axis env preconditions. Walk the env-precondition matrix in CAPABILITIES.md ## Safety policy for the four environment rows (DPA visible to the host, firmware custom-PCC slot, DOCA + DPACC matched, doca_dev access); the parameter-shape and smoke rows are checked later, in ## build and ## test. Surface the firmware custom-PCC slot explicitly: it is the precondition agents most often miss, and a disabled slot looks like a permission bug at the API surface. If the slot was just enabled, confirm the BlueField has completed the required reset; otherwise stop and route to doca-setup for the hardware-safe reset workflow. Any failing environment row is an env / firmware / version fix via doca-setup TASKS.md ## configure (doca-setup/TASKS.md#configure)

   • doca-version TASKS.md ## configure (doca-version/TASKS.md#configure), NOT a code change in the host-side PCC program.

2. Walk the two-side-program model BEFORE writing any host code. Per the two-side-program rule in CAPABILITIES.md ## Capabilities and modes, the user is writing TWO translation units: a host-side program that calls doca_pcc_* (load / parameterize / start / observe), and a DPA-side program (the congestion-control algorithm body) that dpacc compiles into a binary the host executable embeds as a doca_pcc_app. The agent must surface this model EXPLICITLY before any host-side load call is drafted. If the user is asking “how do I design the algorithm itself”, that is the public DOCA PCC programming guide + the user's domain expertise — route via doca-public-knowledge-map/SKILL.md to the public guide; do not invent an algorithm body.

3. Run triple-axis capability discovery against the target BlueField per the triple-axis rule in CAPABILITIES.md ## Capabilities and modes, and quote ALL THREE results back to the user. If any says not supported / not enabled, that axis is the answer — do not proceed.

4. Create the doca_pcc Core context against the chosen doca_dev. Pick the doca_dev deliberately: it must map to the BlueField port whose RDMA / RoCE traffic the custom algorithm is meant to control. A host driving custom PCC on more than one port needs one doca_pcc per port, not a “global” one, per the per-instance rule in CAPABILITIES.md ## Capabilities and modes. Use the PCC-specific lifecycle: doca_pcc_create → configure / load / parameterize → doca_pcc_start → use → doca_pcc_stop → doca_pcc_destroy. A doca_pcc is not a doca_ctx; do not substitute doca_ctx_* calls.

5. Load the PCC algorithm image and parameterize the algorithm. Before loading the algorithm image, declare the PCC device configuration using the doca_pcc_dev_declare_* family from the public PCC header: number of counter groups, number of histograms, threads per counter set, each algorithm, each slot algorithm, counter types, and user memory. Associate histograms with algorithms via the doca_pcc_dev_algo_register_hist_ids call. Initialize each histogram with doca_pcc_dev_histogram_init (mode from the DOCA_PCC_DEV_HISTOGRAM_MODE_* family) and manage it with the doca_pcc_dev_histogram_* family. Validate limits with the doca_pcc_dev_get_* family before declaring. These declarations are mandatory before runtime; skipping them leaves the configure flow incomplete. Then load the doca_pcc_app produced by dpacc from the user's DPA-side algorithm source into the doca_pcc; then set whatever host-side parameters the algorithm exposes. Start it with doca_pcc_start() (there is no doca_pcc_as_ctx) — this is the moment the algorithm becomes live on the port. Record the order so teardown follows the reverse lifecycle in CAPABILITIES.md ## Safety policy.

6. Confirm the attached port actually has RDMA / RoCE traffic to control. A custom PCC algorithm on a port with no RDMA / RoCE flows has nothing to do — “my algorithm loaded but I see no effect” is most often “there is no traffic to affect”. If the user has not stood up RDMA / RoCE on this port, route to doca-rdma/SKILL.md BEFORE expecting visible PCC behavior; PCC modulates existing traffic.

7. Sanity check before the first run. Confirm with the user: which BlueField port (which doca_dev) the custom PCC algorithm is attached to; which PCC algorithm image (doca_pcc_app) is loaded and which entry points it exposes; which host-side parameters were set and what the DPA-side algorithm advertises as their valid ranges; how the agent will know the algorithm is having a visible effect on RDMA / RoCE traffic (which counter / which report). If any of those are unclear, stop and ask — do not invent.

For shared device and progress-engine patterns around this PCC-specific lifecycle, see doca-programming-guide TASKS.md ## configure (doca-programming-guide/TASKS.md#configure).

build

Goal: compile a PCC-using consumer (host-side C / C++ + at least one DPA-side translation unit containing the custom algorithm) against the user's installed DOCA + DPACC compiler, with pkg-config + dpacc as the joint sources of truth for include + link flags + DPA binary embedding.

The build pattern for any DOCA C / C++ consumer is fully documented in doca-programming-guide TASKS.md ## build (doca-programming-guide/TASKS.md#build). PCC adds a DPACC layer on top — the same DPACC layer doca-dpa TASKS.md ## build (doca-dpa/TASKS.md#build) documents for DOCA DPA — because the custom PCC algorithm is DPA-side code. The DPA-side translation unit is compiled by dpacc into a binary embedded into the host executable, and the link line on the host side pulls in the DOCA PCC library. This skill carries only the PCC-specific overlay:

For non-C host-side consumers (Rust, Go, Python) that drive host-side PCC setup and embed a custom PCC algorithm image built separately by dpacc, the host-side link line and version rules above still apply; the DPA-side build is a separate compilation unit and is out of scope for this skill, but the dpacc version check is the load-bearing input the wrapper still needs.

modify

Goal: take the closest-fitting shipped DOCA PCC sample as the verified starting point and apply a minimum diff to make it match the user's intent, without rewriting from scratch and without inventing algorithm bodies.

The universal modify-a-shipped-sample workflow is in doca-programming-guide TASKS.md ## modify (doca-programming-guide/TASKS.md#modify); this skill provides the PCC-specific slot fill.

The agent emits an intent description + the seven filled slots; the actual unified diff against the sample source is produced the way every other library skill in this bundle handles modify — the agent walks the user through the diff line-by-line against the sample source they read on disk, and has the user paste back the result for validation. The agent's anti-pattern alert: a “clean rewrite” from scratch is almost always slower to first green than a minimum-diff modify on a shipped PCC sample, and removes the user's ability to bisect against a known-good baseline.

run

Goal: actually start the loaded custom PCC algorithm on the attached BlueField port and confirm the host observes the algorithm reporting and the RDMA / RoCE traffic on the port showing the expected congestion behavior.

Steps the agent should walk the user through:

1. Confirm the loaded image exposes the algorithm entry points the host configure step expects. The host's view of “which algorithm is running on this port” is exactly the entry-point set in the loaded doca_pcc_app. If the host-side configure step assumes entry points dpacc did not compile in, that is a build-side bug, not a runtime bug; route back to ## build.

2. Start the doca_pcc via doca_pcc_start(). Per the attach-to-port section in CAPABILITIES.md ## Capabilities and modes, doca_pcc_start() is the moment the loaded algorithm becomes live on the port; before this point the algorithm is loaded but inert. A successful start does NOT yet imply the algorithm is having a visible effect — that needs traffic on the port AND the host observing the algorithm's reports.

3. Drive the host-side doca_pe_progress loop in parallel with the running PCC algorithm. The host-side reports the algorithm emits flow through the progress engine; a host that starts the doca_pcc and then blocks without progressing the PE will see no reports and conclude “the algorithm is broken” incorrectly. This is the cross-library “PE not progressed” failure mode applied to PCC.

4. Confirm there is RDMA / RoCE traffic on the attached port to observe. If the port is idle, the algorithm will run but have no visible effect; route to doca-rdma/SKILL.md for bringing up the traffic the algorithm is meant to modulate.

5. Capture the structured log on first failure. Set DOCA_LOG_LEVEL=trace for the first run (see doca-debug CAPABILITIES.md ## Observability (doca-debug/CAPABILITIES.md#observability)); if the host-side log is silent but the algorithm seems inactive, reach for the pcc_counters CLI named in CAPABILITIES.md ## Observability and routed via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools) to inspect counters at the port — this catches the “algorithm loaded fine but is not actually affecting traffic” case the host-side surface alone may not show.

For the runtime version + LD_LIBRARY_PATH cross-checks that underlie “the program built but does nothing”, see doca-version TASKS.md ## run (doca-version/TASKS.md#run).

test

Goal: prove the configured custom PCC deployment actually loads the algorithm onto the BlueField, attaches it to the intended port, and produces visible, expected changes in the RDMA / RoCE traffic the port carries on the user's hardware, and that the env-precondition + capability set was sized right.

This is a loop, not a one-shot pass. Each iteration narrows either the env-precondition set, the capability set, the two-side-program parameter shape, the algorithm-body behavior, or the observability hook-up. The loop terminates when either (a) the host observes the custom algorithm affecting RDMA / RoCE traffic on the attached port the way the user intended, or (b) the agent has narrowed the failure cause to a layer outside PCC itself (algorithm-body design, DPA-side hang, DPACC build, BlueField firmware configuration, DOCA install) and escalated to the matching skill.

Iteration shape:

1. Single small-algorithm smoke. Deploy ONE trivial custom PCC algorithm (a no-op rate-update, or a pass-through that emits one report) and confirm: the host-side load + start succeed; the cap-query baseline matches what was captured at configure; the host observes at least one report from the running algorithm. If yes, advance. If no — and the host-side log shows no error — the algorithm is loaded but not running, or the observability hook-up is wrong; route to ## debug layer 5 and consult the pcc_counters tool. Per the safety-policy rule in CAPABILITIES.md ## Safety policy, DO NOT scale a broken smoke into a complex algorithm design.

2. Traffic-affecting smoke. Start the smoke algorithm with active RDMA / RoCE traffic on the attached port and confirm the algorithm's intended effect (even if trivial — e.g. a fixed rate cap) is visible in the PCC counters exposed by the pcc_counters CLI. Catches the “the algorithm loaded but is not actually modulating traffic” case; if it fails, the algorithm body has no effect path, or it is not actually attached to the traffic-carrying port — re-verify the doca_dev selection at configure time. Before starting, confirm the selected port and workload are an approved, isolated test scope; otherwise do not run a traffic-affecting algorithm. Afterward, stop PCC and verify that the port returned to its intended baseline behavior.

3. Sustained-run loop. Let the smoke algorithm run for a sustained period (minutes, not seconds) under RDMA / RoCE load and confirm the host continues to observe reports without loss and the counter tool shows continuous modulation. Catches observability-queue-sizing bugs and DPA-side algorithm slow leaks.

4. Two-side-program parameter negative test. Intentionally pass a host-side parameter outside the range the DPA-side algorithm advertises and confirm the host gets DOCA_ERROR_INVALID_VALUE cleanly — validates that the agent's earlier two-side-program parameter check is real, not just notional. Then restore the in-range parameter. Run this only on the approved, isolated test scope; after restoring the parameter, verify the PCC context remains healthy.

5. Cap-query negative test. Intentionally call a doca_pcc_cap_* axis the agent expects to be not supported on this BlueField + DOCA + firmware combo and confirm the reported DOCA_ERROR_NOT_SUPPORTED matches the triple capability discovery from ## configure step 3 — validates the agent's capability-discovery itself is correct. Keep the same approved, isolated test scope and verify the PCC context remains healthy before continuing.

Eval-loop overlay — why this is a loop, not a one-shot pass:

Loop identity is the same trigger or DOCA_ERROR_* on the same port, algorithm image, parameter set, and traffic profile, with unchanged host logs, PCC counters, algorithm reports, and traffic evidence. Stop after two consecutive iterations with that identity produce no evidence change — that means the cause is below PCC (BlueField firmware slot, algorithm design bug, DPACC bug, NIC firmware). Escalate to doca-debug TASKS.md ## debug (doca-debug/TASKS.md#debug) with the captured layer-1-through-5 evidence including BOTH the host-side DOCA log and the pcc_counters output.

debug

Goal: when a doca_pcc_* call (or a custom PCC algorithm running on the BlueField) returns a DOCA_ERROR_* or does not make forward progress, narrow the cause to a specific layer and act on it.

The cross-library debug ladder lives in doca-debug TASKS.md ## debug (doca-debug/TASKS.md#debug). Walk through it in order — install → version → build → link → runtime → program → driver — before recommending PCC-specific fixes. This skill's overlay names the PCC-specific manifestation at layers 5 (runtime), 6 (program), and 7 (driver):

Layer 5 (runtime) — PCC overlay.

• A loaded custom PCC algorithm that produces no host-side report is almost always one of three things: the host is not progressing the PE; the DPA-side algorithm body has no emit path; or the observability queue attached to the doca_pcc is full and being silently dropped at the host's failure to drain. Confirm the env-side preconditions per ## configure step 1 and the host-side progress per ## run step 3 before assuming the PCC algorithm itself is broken.

• A loaded custom PCC algorithm that shows no on-wire change has three candidate causes, and the agent must rule them out in this order — the first two are host-side and cheap to check, the third means auditing DPA code:

  1. The algorithm is loaded but inert. A successful load is not a live algorithm: doca_pcc_start() is the moment it becomes live on the port, per the attach-to-port table in CAPABILITIES.md ## Capabilities and modes. A load that succeeded without a start produces exactly this symptom and reports nothing wrong.

  2. No traffic on the attached port. PCC only modulates existing RDMA / RoCE traffic; the doca_dev the doca_pcc was created against is the port that matters. If RDMA / RoCE is not stood up there, route to doca-rdma/SKILL.md.

  3. The algorithm body takes no rate-update action. Only once 1 and 2 are excluded is the DPA-side body the suspect.

  Use the pcc_counters CLI (route via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools)) to inspect the port-level counters and confirm which case applies. Do not open the algorithm body first: reports firing proves the body runs, not that it is bound and live.

• A “DPA-side PCC algorithm hung; cannot stop the program” pattern is almost always a missing termination signal in the DPA-side algorithm body, per the doca-dpa TASKS.md ## modify (doca-dpa/TASKS.md#modify) DPA-kernel termination rule that PCC inherits.

Layer 6 (program) — PCC overlay.

• Lifecycle ordering: the loaded doca_pcc_app must be released BEFORE the doca_pcc itself is destroyed. Out-of-order returns DOCA_ERROR_BAD_STATE per CAPABILITIES.md ## Safety policy.

• Two-side-program parameter mismatch: a host-side parameter call whose shape does not match the DPA-side algorithm's parameter shape returns DOCA_ERROR_INVALID_VALUE. The fix is to rebuild the DPA-side image via dpacc after re-aligning the parameter shapes, then rebuild the host executable; do not patch only one side. See the do not partial-rebuild one side rule in CAPABILITIES.md ## Safety policy.

• Attached-port mismatch: if the user reports “the algorithm runs but does not affect my flows” and the doca_pcc was created against a doca_dev for a different BlueField port than the one carrying the user's RDMA / RoCE traffic, that is the bug. Re-verify the doca_dev selection at configure time against the user’s traffic-carrying port.

Layer 7 (driver) — PCC overlay.

• DOCA_ERROR_DRIVER from doca_pcc_* is most often the PCC driver layer reporting failure to DOCA, and most often because DOCA + DPACC are skewed. Capture pkg-config --modversion doca-pcc, the installed dpacc version, and doca_caps --version; cross-check against the DOCA Compatibility Policy at https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html.

• DOCA_ERROR_NOT_PERMITTED from doca_pcc create / load, with doca_dev access otherwise fine, points at the BlueField firmware not having the custom-PCC slot enabled — this is a firmware-side fix, NOT a host-OS permission fix. Route to doca-setup TASKS.md ## debug (doca-setup/TASKS.md#debug) layer 5 (driver) for the firmware-side enable; the BlueField typically needs a reset after the slot is flipped before the new state takes effect.

• A BlueField in the wrong mode that does NOT expose the DPA surfaces as DOCA_ERROR_NOT_SUPPORTED at doca_pcc create time. Route to doca-setup TASKS.md ## debug (doca-setup/TASKS.md#debug) layer 5 (driver) for the env-side BlueField mode fix; this is NOT a code change in the program.

Once the layer is identified, route to the matching debug verb on the matching skill: install / build / link / driver to doca-setup ## debug (doca-setup/TASKS.md#debug); version to doca-version TASKS.md ## debug (doca-version/TASKS.md#debug); cross-cutting runtime to doca-debug ## debug (doca-debug/TASKS.md#debug); program-layer Core-context patterns to doca-programming-guide TASKS.md ## debug (doca-programming-guide/TASKS.md#debug); DPA-specific kernel-side hangs to doca-dpa TASKS.md ## debug (doca-dpa/TASKS.md#debug).

Deferred task verbs

The following verbs are out of scope for this skill but are commonly asked in the same conversations. Route them as follows so the agent does not invent guidance:

• install. Installing DOCA, installing the DPACC compiler, choosing matched versions, enabling the BlueField firmware custom-PCC slot, post-install verification, pkg-config wiring — defer to doca-setup/SKILL.md and to the install-tree layout in doca-public-knowledge-map ## Layout of an installed DOCA package (doca-public-knowledge-map/SKILL.md#layout-of-an-installed-doca-package). This skill assumes DOCA + DPACC are installed and matched and the firmware custom-PCC slot is enabled.

• deploy. Deploying custom-PCC-using applications at scale (multi-BlueField clusters, Kubernetes operator workflows for DPU workloads, multi-tenant PCC sharing) — out of scope for this skill. For single-host first-run testing, the right verb in this skill is ## run; do not invent a "deploy" workflow.

• algorithm design. Designing the congestion-control algorithm body itself — out of scope. Route via doca-public-knowledge-map/SKILL.md to the public DOCA PCC programming guide and to the user's own congestion-control domain expertise. This skill prescribes how to load and attach an algorithm from the host; it does not design the algorithm.

• DPA-side algorithm programming and DPACC usage. Writing the DPA-side algorithm body, DPA-side memory layout used inside the algorithm, DPACC compile flags, DPA-side debugging from inside the algorithm — out of scope. Route via doca-public-knowledge-map/SKILL.md to the public DOCA PCC, DOCA DPACC Compiler, and DOCA DPA guides plus the DPA Tools umbrella. This skill prescribes how to use the DPACC-produced image from the host; it does not redefine how to produce it.

• pcc_counters diagnostic CLI — a separate artifact for read-only PCC counter inspection; route via doca-public-knowledge-map ## DOCA tools (doca-public-knowledge-map/SKILL.md#doca-tools). This skill is for custom algorithm load + control, not for inspection.

• Default factory PCC algorithm in ConnectX firmware — not a doca-pcc topic. Configuration is firmware-side; route via doca-public-knowledge-map/SKILL.md.

Command appendix

Every command below is cross-cutting on DOCA PCC — it answers a recurring class of question that comes up in the verbs above. The agent should treat the class as load-bearing; the worked example is a single instance. Run-as user is the unprivileged user unless noted. Rows that need elevated privileges call that out explicitly.

Infra-aware preamble (every row below). Per the bundle's detect → prefer → fall back → report contract documented in doca-structured-tools-contract ## The agent behavior contract (doca-structured-tools-contract/SKILL.md#the-agent-behavior-contract), the agent should:

1. Probe for the matching structured helper FIRST (doca-env --json for version + devices + libraries + drivers + hugepages in one shot; doca-capability-snapshot for per-device capability flags; version-matrix.json for “available since” lookups).

2. If the probe succeeds, the structured tool's output is the authoritative answer and the agent SHOULD NOT also run the manual command in the row below. Report “using structured <tool>”.

3. If the probe fails, fall back to the manual command in the row. Report “falling back to manual chain”.

4. The schemas the structured tools emit are defined in doca-structured-tools-contract ## Schemas (doca-structured-tools-contract/SKILL.md#schemas); the version-handling semantics (four-way match, NGC, headers-win) are owned by doca-version/SKILL.md.

For commands shared across libraries (pkg-config --modversion, doca_caps, cat /opt/mellanox/doca/applications/VERSION, DOCA_LOG_LEVEL) the cross-library overlay is in doca-debug TASKS.md ## Command appendix (doca-debug/TASKS.md#command-appendix); this table adds the PCC-specific rows on top.