Matlab Set Up Worker State

Set Up Worker State for Parallel Pools

By default, process workers in a parallel pool inherit MATLAB path state from their controlling client, but they do not inherit loaded libraries, open connections, or expensive pre-computed objects. Code that relies on any of these needs explicit setup. This skill teaches the correct APIs for each scenario. Most patterns target process-based pools; thread pool applicability is noted where relevant.

When to Use

• Code needs a non-serializable resource on workers (database connections, COM objects, loaded shared libraries, file handles)

• Code needs expensive one-time setup per worker (large object construction, data loading) that should not repeat every parfor iteration

• Code needs paths or environment variables set on workers

• User has existing code using spmd for side-effect setup before parfor (anti-pattern — help them modernise)

• User sees errors about objects not being serializable when passed to parfor

When NOT to Use

• Data already in client memory used in a single parfor loop (MATLAB broadcasts it automatically — Constant adds complexity for no benefit in this case)

• Choosing between process and thread pools (out of scope — this skill assumes a pool type is already chosen)

Decision Framework

Thread pool notes: parallel.pool.Constant and parfevalOnAll also work on thread pools. However, thread workers share the client's process, so path and environment variable changes on the client are visible to threads automatically — AdditionalPaths and EnvironmentVariables do not apply. To modify a thread worker's environment, alter the client environment before the parfor.

The spmd Anti-Pattern

This applies equally to process pools and thread pools — prefer parallel.pool.Constant over spmd for worker state setup regardless of pool type.

What it looks like

% ANTI-PATTERN: Do not do this
spmd
    loadlibrary("mylib", "mylib.h");
end
parfor i = 1:100
    result(i) = calllib("mylib", "compute", data(i));
end
spmd
    unloadlibrary("mylib");
end

Why it's fragile

1. No cleanup guarantee — if the parfor errors, the second spmd never runs

2. Makes the pool fragile to worker disconnection — once a pool has run an spmd block, a single worker losing connection tears down the entire pool. Pools that never use spmd continue operating with fewer workers if one disconnects (no replacement occurs — the pool simply shrinks).

3. Composite variables cannot be used inside parfor — values created in spmd are Composite objects that require indexing on the client; they cannot be referenced directly from within a parfor body

How to modernise

Replace with parallel.pool.Constant:

C = parallel.pool.Constant( ...
    @() loadAndReturn(), ...
    @(~) unloadlibrary("mylib")); %#ok<NASGU> — kept alive for cleanup

result = zeros(1, 100);
parfor i = 1:100
    result(i) = calllib("mylib", "compute", data(i));
end

% Cleanup happens automatically when C goes out of scope

Or with parfevalOnAll for simple side effects:

f = parfevalOnAll(pool, @setupWorker, 0);
fetchOutputs(f);

result = zeros(1, 100);
parfor i = 1:100
    result(i) = doWork(data(i));
end
% No automatic cleanup

Prefer parallel.pool.Constant when cleanup is needed. Use parfevalOnAll only for operations where cleanup is not required.

Patterns

Pattern 1: Loading data from a file for use in parfor

Always use the build-function form when the source is a file. This avoids loading the data into client memory entirely — each worker loads directly from disk. This is critical for large files but is also the correct default for any file-based data because it scales without code changes as file size grows.

c = parallel.pool.Constant(@() load("costSurface.mat").costSurface);

results = zeros(1, 10000);
parfor i = 1:10000
    results(i) = processWithLookup(input(i), c.Value);
end

Each worker calls load() once. The client never holds the full dataset.

Do NOT do this — loading on the client then wrapping defeats the purpose:

% WRONG: loads entire file into client memory, then copies to each worker
data = load("costSurface.mat").costSurface;
c = parallel.pool.Constant(data);  % client already paid the memory cost

The only time parallel.pool.Constant(data) (without a build function) is appropriate is when the data is already in client memory and used across multiple parfor loops (avoids re-broadcasting each loop). For a single parfor loop, just let MATLAB broadcast — Constant adds no benefit.

Pattern 2: Shared library (non-serializable, needs cleanup)

c = parallel.pool.Constant( ...
    @() loadLibraryAndReturn("mylib", "mylib.h"), ...
    @unloadlibrary);

results = zeros(1, 100);
parfor i = 1:100
    results(i) = calllib(c.Value, "compute", data(i));
end
% Cleanup runs automatically when c goes out of scope

function libName = loadLibraryAndReturn(libName, headerFile)
    loadlibrary(libName, headerFile);
end

The build function returns the library name so that c.Value is naturally referenced in the parfor body — this triggers lazy initialisation. The cleanup function receives the same value and calls unloadlibrary on it.

Pattern 3: One-shot setup with parfevalOnAll (no cleanup needed)

pool = gcp;
f = parfevalOnAll(pool, @setupWorker, 0);
fetchOutputs(f);

Pattern 4: Database connection (use createConnectionForPool)

Database connections are a common specific case that requires Database Toolbox. Prefer the purpose-built helper createConnectionForPool over a hand-rolled parallel.pool.Constant. It returns a parallel.pool.Constant whose value is a per-worker connection to the configured data source, and it handles worker-side initialisation that a naive database() call inside a Constant build function does not.

pool = gcp;
c = createConnectionForPool(pool, "MyDataSource", "", "");

results = cell(1, 1000);
parfor i = 1:1000
    conn = c.Value;
    results{i} = fetch(conn, sprintf("SELECT val FROM t WHERE id=%d", i));
end

The returned Constant is the standard parallel.pool.Constant type, so it participates in the normal lifetime model: connections are torn down when the Constant goes out of scope or the pool shuts down.

Recognising When pool.Constant Is Needed

Look for these signals in user code:

Common Mistakes

Key Functions

Conventions

• Prefer parallel.pool.Constant over spmd for any worker state that persists across parfor iterations

• The build-function form (parallel.pool.Constant(@buildFcn)) is lazily initialised — the build function does not run until c.Value is first accessed on a worker. Always reference c.Value inside the parfor body to ensure initialisation occurs.

• Always provide a cleanup function when the resource needs explicit teardown

• Use parfevalOnAll only when no handle is needed back and no cleanup is required

• Use parpool name-value pairs when the setup is needed for the entire pool lifetime (paths, env vars)

• When modernising spmd-based setup code, preserve the original's intent (what resource, what cleanup) while switching to Constant

Copyright 2026 The MathWorks, Inc.