Matlab Run Tests
MathWorksOfficialCreated by MathWorks
- Skill
- Tool integration
- Software development
- BSD-2-Clause
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a structured workflow for running and managing MATLAB test suites. It covers test execution, filtering, parallel processing, and diagnostic reporting. Additionally, it details how to collect and analyze code coverage metrics, including statement, decision, condition, and MC/DC levels, as well as data type and size coverage. It also supports CI/CD integration using MATLAB buildtool.
What it does
- Executes MATLAB test suites with support for filtering by name, tag, or folder.
- Provides diagnostic reporting for failed or incomplete test results.
- Configures code coverage collection using plugins to measure statement, decision, condition, and MC/DC metrics.
- Extracts and analyzes data type and size coverage for function inputs.
- Integrates with CI/CD pipelines using buildfile.m and buildtool.
- Supports parallel test execution to improve performance.
When to use it
- When you need to run an existing MATLAB test suite.
- When you need to verify code coverage metrics for your source code.
- When you are setting up or configuring CI/CD pipelines for MATLAB projects.
- When you need to diagnose why specific tests are failing or incomplete.
- When you need to filter large test suites to run only relevant tests.
Advantages
- Enables programmatic control over test execution and reporting.
- Provides granular visibility into code coverage and testing gaps.
- Facilitates automated testing workflows within CI/CD environments.
- Supports advanced coverage metrics like MC/DC and data type/size analysis.
Usage examples
- “Run all tests in the 'tests' folder using parallel execution.”
- The agent will execute runtests('tests', UseParallel=true) and report the pass/fail results.
- “Collect MC/DC coverage for the 'src' folder.”
- The agent will set up a TestRunner with a CodeCoveragePlugin, run the tests, and output the coverage summary.
- “Show me which tests failed in my last run.”
- The agent will iterate through the results array, identify failed tests, and display their diagnostic records.
- “How do I configure a buildfile for my CI pipeline?”
- The agent will provide a template for buildfile.m including a TestTask and instructions for CI integration.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/matlab/matlab-run-tests/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip matlab-run-tests.zip -d .agents/skills
Documentation by MathWorks
Run Tests
Run MATLAB test suites, analyze results, collect code coverage metrics, and configure CI/CD pipelines using matlab.unittest infrastructure.
When to Use
User wants to run an existing test suite
User needs to filter tests by name, tag, or folder
User wants to analyze test failures or incomplete results
User needs code coverage collection or analysis
User wants CI/CD pipeline configuration with buildtool
User asks about parallel test execution
When NOT to Use
Writing tests, creating test classes, parameterization, fixtures, mocking — use
matlab-write-testsBaseline/regression tests against stored reference data — use
matlab-write-testsTesting Simulink models — use Simulink test skills
Performance benchmarking — use profiling workflows
Workflow
Run and report
Run tests — Execute via
run_matlab_test_fileMCP tool orruntestswith filteringAnalyze results — Report pass/fail counts, iterate failed tests for diagnostics
Verify — Confirm all tests pass before proceeding to coverage or CI
Coverage analysis (when requested)
Collect — Set up
TestRunnerwithCodeCoveragePluginandCoverageResultReport summary — Call
coverageSummaryand present percentages to userAct on results — Generate reports, analyze gaps, or justify — as user directs
CI/CD setup
Create buildfile.m — Define
TestTaskwith source and report optionsAdd CI config — GitHub Actions, Azure DevOps, or GitLab CI template
Running Tests
Via MCP
Use the run_matlab_test_file MCP tool to run test files directly. When you need filtering, parallel execution, or other options, use evaluate_matlab_code with runtests:
results = runtests('tests'); % all tests in folder
results = runtests('tests', Tag='Unit'); % by tag
results = runtests('tests', Name='*Calculator*'); % by name pattern
results = runtests('tests', UseParallel=true); % parallel execution
results = runtests('tests', Strict=true); % warnings = failuresAnalyzing Results
disp(results);
for r = results([results.Failed])
fprintf('\nFAILED: %s\n', r.Name);
disp(r.Details.DiagnosticRecord.Report);
end
for r = results([results.Incomplete])
fprintf('\nINCOMPLETE: %s\n', r.Name);
disp(r.Details.DiagnosticRecord.Report);
endCoverage Analysis
import matlab.unittest.TestRunner
import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult
runner = TestRunner.withTextOutput;
covFormat = CoverageResult;
runner.addPlugin(CodeCoveragePlugin.forFolder('src', ...
IncludingSubfolders=true, Producing=covFormat, MetricLevel='mcdc'));
results = runner.run(testsuite('tests'));
covResults = covFormat.Result;
summary = coverageSummary(covResults, "mcdc");
fprintf('Coverage: %d/%d (%.1f%%)\n', sum(summary(:,1)), sum(summary(:,2)), ...
100*sum(summary(:,1))/sum(summary(:,2)));For detailed coverage workflows — gap analysis, justifications, generated C code coverage, and data type/size coverage — see references/code-coverage-guidance.md.
Key Functions
| Category | Functions | Purpose |
|---|---|---|
| Execution | runtests, TestSuite, TestRunner | Run and organize tests |
| Coverage | CodeCoveragePlugin, CoverageResult, coverageSummary | Measure test coverage |
| Reports | CoverageReport, generateStandaloneReport, generateCoberturaReport | Coverage output formats |
| CI/CD | buildtool, TestTask, addCodeCoverage | Pipeline automation |
| Scripts | printCoverageGaps | Print all uncovered items across metric levels — see references/code-coverage-guidance.md for interface |
CI/CD Integration
Use buildtool with a buildfile.m for CI pipelines. See references/test-execution-guidance.md for buildfile.m templates and CI configs (GitHub Actions, Azure DevOps, GitLab CI).
For toolbox projects, matlab-package-toolbox generates a buildfile.m in which the packaging task depends on the test task.
References
Load these on demand — most test runs only need what's in this file.
| Load when... | Reference |
|---|---|
| Running tests in CI, buildtool config, filtering options, parallel execution details | references/test-execution-guidance.md |
| Collecting code coverage (MC/DC, statement, decision, condition), analyzing gaps, justifying uncovered code, generated C code coverage, type-size coverage | references/code-coverage-guidance.md |
| Creating or modifying filter rule XML for coverage justifications | references/justification-filter-rules.md |
| Extracting data type and size coverage results from hidden property | references/datatype-size-extraction.md |
Conventions
Always: run tests via the
run_matlab_test_fileMCP tool when possibleAlways: report pass/fail summary before taking further action
Always: check for Failed/Incomplete tests before analyzing coverage
Prefer:
forFolderwithIncludingSubfolders=trueas the default source specification for coveragePrefer: MC/DC as the default metric level because each level includes all lower levels
Never: generate coverage reports (HTML, Cobertura) unless the user explicitly requests them
Never: use
CoverageResultas the only format — always use it as the primary programmatic format
Copyright 2026 The MathWorks, Inc.
Files
- ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
- LICENSELicence file added by AutoGineer1.4 KiB
- SKILL.md6.3 KiB
- manifest.yaml472 B
- references/code-coverage-guidance.md14.6 KiB
- references/datatype-size-extraction.md4.8 KiB
- references/justification-filter-rules.md4.7 KiB
- references/test-execution-guidance.md2.8 KiB
- scripts/printCoverageGaps.m3.0 KiB
references/code-coverage-guidance.md
Code Coverage Collection and Analysis
Collect coverage metrics for MATLAB source code and generated C code, identify gaps, justify unreachable code, and write targeted tests to close gaps.
Workflow
1. Check environment
Before collecting coverage, verify what's available:
hasMLTest = ~isempty(ver('matlabtest'));
v = version('-release');
fprintf('MATLAB %s | MATLAB Test installed: %d\n', v, hasMLTest);Advanced metrics (decision, condition, MC/DC, justifications) require MATLAB Test. Default to MetricLevel="mcdc" when MATLAB Test is installed; fall back to "statement" otherwise.
2. Collect coverage
Coverage collection requires programmatic TestRunner setup with plugins. Run all coverage code via mcp__matlab__evaluate_matlab_code — the mcp__matlab__run_matlab_test_file tool does not support coverage plugins.
Use CodeCoveragePlugin.forFolder with CoverageResult as the single output format:
import matlab.unittest.TestRunner
import matlab.unittest.TestSuite
import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult
suite = TestSuite.fromFolder("tests");
runner = TestRunner.withTextOutput;
format = CoverageResult;
runner.addPlugin(CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, Producing=format, MetricLevel="mcdc"));
results = runner.run(suite);
coverageResult = format.Result; % Object array — one element per source fileVerify results has no Failed or Incomplete outcomes before analyzing coverage — failed tests produce incomplete coverage data. If tests fail, fix them first and re-run.
3. Report summary
coverageResult is an object array with one element per instrumented source file. coverageSummary accepts the full array and returns an Nx2 matrix (one row per file: [achieved, total]).
Report all metric levels up to the collected level — call coverageSummary once per level. Present the results in a single combined table with one row per file and one column per metric level.
| Collected MetricLevel | Report these levels |
|---|---|
"statement" | statement |
"decision" | statement, decision |
"condition" | statement, decision, condition |
"mcdc" | statement, decision, condition, mcdc |
% 1-output form: Nx2 matrix [achieved, total] per file
summary = coverageSummary(coverageResult, "mcdc");
fprintf('Coverage: %d/%d (%.1f%%)\n', sum(summary(:,1)), sum(summary(:,2)), ...
100*sum(summary(:,1))/sum(summary(:,2)));After reporting the summary, tell the user the coverage percentage and ask what they'd like to do next: analyze gaps, generate a report, or write tests.
4. Act on results (user-directed)
Based on user request:
Generate interactive report —
generateHTMLReport(coverageResult, "coverage-report")(folder-based, browseable per-file view)Generate portable report —
generateStandaloneReport(coverageResult, "report.html")(single file, shareable)Generate CI artifact —
generateCoberturaReport(coverageResult, "coverage.xml")Analyze gaps — see Gap Analysis pattern below
Justify unreachable code — see Justification pattern (only when user asks)
Write tests for gaps — see Test Generation pattern
Alternatively, produce the interactive report during collection using CoverageReport:
import matlab.unittest.plugins.codecoverage.CoverageReport
runner.addPlugin(CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, ...
Producing=[format, CoverageReport("coverage-report")], ...
MetricLevel="mcdc"));Key Functions
| Function | Purpose | Requires |
|---|---|---|
CodeCoveragePlugin.forFolder | Instrument source folder for coverage | (core) |
CodeCoveragePlugin.forFile | Instrument specific files | (core) |
CodeCoveragePlugin.forNamespace | Instrument source code in a specified namespace | (core) |
coverageSummary(result, level) | Get coverage counts + gap details | (core) |
generateHTMLReport | Interactive HTML report (folder-based, browseable) | (core) |
generateStandaloneReport | Single-file HTML report (portable, shareable) | (core) |
generateCoberturaReport | Cobertura XML for CI | (core) |
CodeCoveragePlugin(..., Filter=file) | Apply filter rules during collection | MATLAB Test |
result.applyFilter(file) | Apply filter rules to existing results | MATLAB Test |
result.resetFilter | Remove applied filters, return unfiltered results | MATLAB Test |
GeneratedCodeCoveragePlugin | Coverage for generated C code | MATLAB Test + MATLAB Coder |
Patterns
Basic Coverage Collection
Always use forFolder with IncludingSubfolders=true for projects with nested source directories. Use forFile only when measuring specific individual files. Use forNamespace when source code is organized in namespaces (+packageName folders).
% CORRECT: covers all .m files in src/ and its subfolders
plugin = CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, Producing=format, MetricLevel="mcdc");
% For namespace-organized code: covers all files in +myPackage and inner namespaces
plugin = CodeCoveragePlugin.forNamespace("myPackage", ...
IncludingInnerNamespaces=true, Producing=format, MetricLevel="mcdc");
% WRONG: verbose manual enumeration — don't do this
% srcFiles = dir(fullfile("src", '**', '*.m'));
% plugin = CodeCoveragePlugin.forFile(filePaths, ...);MetricLevel values: "statement", "decision", "condition", "mcdc". Default to "mcdc" when MATLAB Test is installed, "statement" otherwise.
Each level includes all levels below it — "mcdc" gives you statement + decision + condition + MC/DC.
Release compatibility: On R2026b+, MetricLevel is deprecated in CodeCoveragePlugin — use Metrics instead. Metrics also supports "type-size" (see Data Type and Size Coverage below). When unsure of the release, use MetricLevel — it works on all supported versions.
% R2026b+ only: Metrics replaces MetricLevel for CodeCoveragePlugin
plugin = CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, Producing=format, Metrics="mcdc");Generated C Code Coverage
For generated code from MATLAB Coder, use GeneratedCodeCoveragePlugin (not CodeCoveragePlugin). Always pass Producing=format to get programmatic access to results — without it, coverage is collected but not accessible.
import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult
suite = TestSuite.fromClass(?myCoderEquivalenceTest);
runner = TestRunner.withTextOutput;
format = CoverageResult;
runner.addPlugin(GeneratedCodeCoveragePlugin( ...
MetricLevel="condition", Producing=format));
results = runner.run(suite);
coverageResult = format.Result; % Object array, same as CodeCoveragePluginPreserveInFolder is only required for Cobertura XML output. When producing CoberturaFormat, the test must use PreserveInFolder in its build() call so build artifacts (C source files) remain on disk for report generation:
import matlab.unittest.plugins.codecoverage.CoberturaFormat
% Only needed when producing Cobertura XML:
runner.addPlugin(GeneratedCodeCoveragePlugin( ...
MetricLevel="condition", Producing=CoberturaFormat("coverage.xml")));
% The test class must call:
% testCase.build(fcn, Inputs=args, Configuration=cfg, PreserveInFolder=folder)Data Type and Size Coverage
Tracks which (type, size) combinations of input arguments are exercised. Uses the Metrics parameter with "type-size" as an additional metric alongside a structural metric.
Available from R2026b. Check availability before using:
plugin = CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, Producing=format, ...
Metrics=["statement", "type-size"]);Note: "type-size" cannot be used alone — it must be combined with a structural metric ("statement", "decision", "condition", or "mcdc").
For extracting and interpreting the results, see datatype-size-extraction.md — consult when the user asks about type-size coverage data or input diversity.
Gap Analysis
Use the 2-output form of coverageSummary to get detailed gap information:
[summary, description] = coverageSummary(coverageResult, "mcdc");
% description.mcdc contains per-decision detail:
mcdcDetails = description.mcdc;
for i = 1:numel(mcdcDetails)
conditions = mcdcDetails(i).Condition;
for j = 1:numel(conditions)
if ~conditions(j).Achieved
fprintf('Decision %d, Cond %d: "%s" — needs: %s\n', ...
i, j, conditions(j).Text, conditions(j).FalseResult);
end
end
endFor decision-level gaps:
[summary, description] = coverageSummary(coverageResult, "decision");
decisions = description.decision;
for i = 1:numel(decisions)
for j = 1:numel(decisions(i).Outcome)
if decisions(i).Outcome(j).ExecutionCount == 0
fprintf('Decision %d: %s outcome uncovered\n', ...
i, decisions(i).Outcome(j).Text);
end
end
endprintCoverageGaps
Prints all uncovered items across all metric levels (statement, function, decision, condition, MC/DC) present in the coverage result.
Script: ../scripts/printCoverageGaps.m
| Input | covResults — matlab.coverage.Result array from CoverageResult format |
| Output | Prints to stdout; no return value |
printCoverageGaps(covResults);Programmatic Justification
Justify structurally unreachable code when the user requests it. Justifications can only be applied up to decision level — condition and MC/DC outcomes cannot be justified. This is a human-in-the-loop workflow: highlight the gaps, then justify when the user confirms.
The approach: create a filter rule XML file that can be reused across runs. This is preferable to constructing justification objects programmatically because the XML file is version-controllable, shareable, and can be supplied at collection time or applied after.
For the full XML schema, attribute reference, and usage patterns, see justification-filter-rules.md — consult whenever applying justifications.
Quick example:
% 1. Create the filter rule XML file
filterXml = fullfile(projectRoot, "resources", "CodeCoverageFilter.xml");
xmlContent = [ ...
'<?xml version="1.0" encoding="utf-8"?>', newline, ...
'<filter>', newline, ...
' <rule type="DECISION" fileName="shortest_path.m" ', ...
'functionName="shortest_path" ', ...
'expr="if ~isNodeValid(endIdx)" index="3" ', ...
'mode="JUSTIFIED" ', ...
'rationale="Structurally unreachable: early return prevents reaching this branch"/>', newline, ...
'</filter>'];
writelines(string(xmlContent), filterXml);
% 2a. Supply at collection time (justifications applied during run)
plugin = CodeCoveragePlugin.forFolder("src", ...
IncludingSubfolders=true, Producing=format, ...
MetricLevel="decision", Filter=filterXml);
% 2b. Or apply after collection
filteredResult = coverageResult.applyFilter(filterXml);
generateStandaloneReport(filteredResult, "report.html");Coverage-Driven Test Generation
When the user wants to close coverage gaps, follow this pattern:
Collect coverage and extract gaps using
coverageSummary2-output formFor each uncovered condition/decision, analyze the source code to determine what input would trigger it
Write targeted test methods that exercise the specific uncovered path
Re-run coverage to verify improvement
Do not use internal matlabtest.internal.testcreation.* APIs to generate tests (they produce empty stubs only). Write test methods directly instead.
% After identifying: condition "windowSize < 1" is uncovered (false branch)
% Write a test that makes windowSize < 1 to trigger that branch:
function testWindowSizeLessThanOne(testCase)
data = randn(100, 1);
testCase.verifyError(@() movingAverageFilter(data, 0), ...
'movingAverageFilter:invalidWindow');
endCommon Mistakes
| Mistake | Why It's Wrong | Correct Approach |
|---|---|---|
dir+arrayfun+forFile for multi-folder | Verbose, fragile with package dirs | forFolder(path, IncludingSubfolders=true) |
CodeCoveragePlugin for generated C code | Only instruments MATLAB code | matlabtest.coder.plugins.GeneratedCodeCoveragePlugin |
Missing PreserveInFolder when using CoberturaFormat with generated code | Build artifacts cleaned up, empty Cobertura report | Add PreserveInFolder=folder to testCase.build() (only needed for Cobertura XML) |
Using Metrics= without checking release | Metrics only works on R2026b+; errors on older releases | Use MetricLevel by default (works on all versions); use Metrics only when you confirm R2026b+ |
Metrics="type-size" alone | Must combine with a structural metric | Metrics=["statement","type-size"] |
Treating coverageSummary 2nd output as numeric | 2-output form returns [summary, description] — description is a struct, not a matrix | Use 1-output summary = coverageSummary(result, level) for totals; 2-output only for gap analysis |
Using coverageResult.Files | Property does not exist | Use coverageResult(i).Filename in a loop, or [coverageResult.Filename] to get all paths |
| Generating multiple report formats unrequested | Wastes tokens and confuses user | Use CoverageResult only; generate reports when asked |
Using matlabtest.coverage.Justification constructor | Error-prone 8-arg API, hard to reuse | Write a filter rule XML file and pass via Filter= or applyFilter |
Lowercase type in filter rule XML | type attribute requires uppercase | Use type="DECISION", type="MCDC_OUTCOME" |
Conventions
Always: use
CoverageResultas the primary output — produce reports only when the user asksAlways: report coverage numbers to the user before taking further action
Always: when MATLAB Test is not installed, fall back to statement-level coverage only
Prefer:
forFolderwithIncludingSubfolders=trueas the default source specificationPrefer: advanced metrics (decision, MC/DC) — MATLAB Test provides capabilities beyond what most coverage tools offer
Prefer: MC/DC as the default metric level because each level includes all lower levels
Never: justify coverage proactively — wait for user to request justification
Never: use internal
matlabtest.internal.*APIs
Copyright 2026 The MathWorks, Inc.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- MathWorks
- Publisher
- MathWorks (Official)
- Source
- matlab/matlab-agentic-toolkit · skills-catalog/matlab-software-development/matlab-run-tests · commit 2a7f215
- Licence
- BSD 2-Clause License (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE.md)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by MathWorks.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download matlab-run-tests.zip