Generated banner for Matlab Run Tests
Generated logo for Matlab Run Tests

Matlab Run Tests

MathWorksOfficialCreated by MathWorks

  • Skill
  • Tool integration
  • Software development
  • BSD-2-Clause
  • No issues found

From the author

Run MATLAB test suites and analyze results, including test filtering, parallel execution, result diagnostics, code coverage collection, gap analysis, and CI/CD with buildtool. Use when running tests, filtering test suites, analyzing test failures, checking, collecting, or analyzing coverage, justifying uncovered code, or configuring CI pipelines. Do NOT use for writing, generating, or structuring tests. Do NOT use for Simulink coverage or Simulink testing workflows.
— the description by MathWorks

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured workflow for running and managing MATLAB test suites. It covers test execution, filtering, parallel processing, and diagnostic reporting. Additionally, it details how to collect and analyze code coverage metrics, including statement, decision, condition, and MC/DC levels, as well as data type and size coverage. It also supports CI/CD integration using MATLAB buildtool.

What it does

  • Executes MATLAB test suites with support for filtering by name, tag, or folder.
  • Provides diagnostic reporting for failed or incomplete test results.
  • Configures code coverage collection using plugins to measure statement, decision, condition, and MC/DC metrics.
  • Extracts and analyzes data type and size coverage for function inputs.
  • Integrates with CI/CD pipelines using buildfile.m and buildtool.
  • Supports parallel test execution to improve performance.

When to use it

  • When you need to run an existing MATLAB test suite.
  • When you need to verify code coverage metrics for your source code.
  • When you are setting up or configuring CI/CD pipelines for MATLAB projects.
  • When you need to diagnose why specific tests are failing or incomplete.
  • When you need to filter large test suites to run only relevant tests.

Advantages

  • Enables programmatic control over test execution and reporting.
  • Provides granular visibility into code coverage and testing gaps.
  • Facilitates automated testing workflows within CI/CD environments.
  • Supports advanced coverage metrics like MC/DC and data type/size analysis.

Usage examples

“Run all tests in the 'tests' folder using parallel execution.”
The agent will execute runtests('tests', UseParallel=true) and report the pass/fail results.
“Collect MC/DC coverage for the 'src' folder.”
The agent will set up a TestRunner with a CodeCoveragePlugin, run the tests, and output the coverage summary.
“Show me which tests failed in my last run.”
The agent will iterate through the results array, identify failed tests, and display their diagnostic records.
“How do I configure a buildfile for my CI pipeline?”
The agent will provide a template for buildfile.m including a TestTask and instructions for CI integration.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/matlab/matlab-run-tests/download
  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip matlab-run-tests.zip -d .agents/skills

Documentation by MathWorks

Run Tests

Run MATLAB test suites, analyze results, collect code coverage metrics, and configure CI/CD pipelines using matlab.unittest infrastructure.

When to Use

  • User wants to run an existing test suite

  • User needs to filter tests by name, tag, or folder

  • User wants to analyze test failures or incomplete results

  • User needs code coverage collection or analysis

  • User wants CI/CD pipeline configuration with buildtool

  • User asks about parallel test execution

When NOT to Use

  • Writing tests, creating test classes, parameterization, fixtures, mocking — use matlab-write-tests

  • Baseline/regression tests against stored reference data — use matlab-write-tests

  • Testing Simulink models — use Simulink test skills

  • Performance benchmarking — use profiling workflows

Workflow

Run and report
  1. Run tests — Execute via run_matlab_test_file MCP tool or runtests with filtering

  2. Analyze results — Report pass/fail counts, iterate failed tests for diagnostics

  3. Verify — Confirm all tests pass before proceeding to coverage or CI

Coverage analysis (when requested)
  1. Collect — Set up TestRunner with CodeCoveragePlugin and CoverageResult

  2. Report summary — Call coverageSummary and present percentages to user

  3. Act on results — Generate reports, analyze gaps, or justify — as user directs

CI/CD setup
  1. Create buildfile.m — Define TestTask with source and report options

  2. Add CI config — GitHub Actions, Azure DevOps, or GitLab CI template

Running Tests

Via MCP

Use the run_matlab_test_file MCP tool to run test files directly. When you need filtering, parallel execution, or other options, use evaluate_matlab_code with runtests:

results = runtests('tests');                            % all tests in folder
results = runtests('tests', Tag='Unit');                % by tag
results = runtests('tests', Name='*Calculator*');       % by name pattern
results = runtests('tests', UseParallel=true);          % parallel execution
results = runtests('tests', Strict=true);               % warnings = failures
Analyzing Results
disp(results);

for r = results([results.Failed])
    fprintf('\nFAILED: %s\n', r.Name);
    disp(r.Details.DiagnosticRecord.Report);
end

for r = results([results.Incomplete])
    fprintf('\nINCOMPLETE: %s\n', r.Name);
    disp(r.Details.DiagnosticRecord.Report);
end

Coverage Analysis

import matlab.unittest.TestRunner
import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult

runner = TestRunner.withTextOutput;
covFormat = CoverageResult;
runner.addPlugin(CodeCoveragePlugin.forFolder('src', ...
    IncludingSubfolders=true, Producing=covFormat, MetricLevel='mcdc'));
results = runner.run(testsuite('tests'));

covResults = covFormat.Result;
summary = coverageSummary(covResults, "mcdc");
fprintf('Coverage: %d/%d (%.1f%%)\n', sum(summary(:,1)), sum(summary(:,2)), ...
    100*sum(summary(:,1))/sum(summary(:,2)));

For detailed coverage workflows — gap analysis, justifications, generated C code coverage, and data type/size coverage — see references/code-coverage-guidance.md.

Key Functions

CategoryFunctionsPurpose
Executionruntests, TestSuite, TestRunnerRun and organize tests
CoverageCodeCoveragePlugin, CoverageResult, coverageSummaryMeasure test coverage
ReportsCoverageReport, generateStandaloneReport, generateCoberturaReportCoverage output formats
CI/CDbuildtool, TestTask, addCodeCoveragePipeline automation
ScriptsprintCoverageGapsPrint all uncovered items across metric levels — see references/code-coverage-guidance.md for interface

CI/CD Integration

Use buildtool with a buildfile.m for CI pipelines. See references/test-execution-guidance.md for buildfile.m templates and CI configs (GitHub Actions, Azure DevOps, GitLab CI).

For toolbox projects, matlab-package-toolbox generates a buildfile.m in which the packaging task depends on the test task.

References

Load these on demand — most test runs only need what's in this file.

Load when...Reference
Running tests in CI, buildtool config, filtering options, parallel execution detailsreferences/test-execution-guidance.md
Collecting code coverage (MC/DC, statement, decision, condition), analyzing gaps, justifying uncovered code, generated C code coverage, type-size coveragereferences/code-coverage-guidance.md
Creating or modifying filter rule XML for coverage justificationsreferences/justification-filter-rules.md
Extracting data type and size coverage results from hidden propertyreferences/datatype-size-extraction.md

Conventions

  • Always: run tests via the run_matlab_test_file MCP tool when possible

  • Always: report pass/fail summary before taking further action

  • Always: check for Failed/Incomplete tests before analyzing coverage

  • Prefer: forFolder with IncludingSubfolders=true as the default source specification for coverage

  • Prefer: MC/DC as the default metric level because each level includes all lower levels

  • Never: generate coverage reports (HTML, Cobertura) unless the user explicitly requests them

  • Never: use CoverageResult as the only format — always use it as the primary programmatic format


Copyright 2026 The MathWorks, Inc.


Files

references/code-coverage-guidance.md

Code Coverage Collection and Analysis

Collect coverage metrics for MATLAB source code and generated C code, identify gaps, justify unreachable code, and write targeted tests to close gaps.

Workflow

1. Check environment

Before collecting coverage, verify what's available:

hasMLTest = ~isempty(ver('matlabtest'));
v = version('-release');
fprintf('MATLAB %s | MATLAB Test installed: %d\n', v, hasMLTest);

Advanced metrics (decision, condition, MC/DC, justifications) require MATLAB Test. Default to MetricLevel="mcdc" when MATLAB Test is installed; fall back to "statement" otherwise.

2. Collect coverage

Coverage collection requires programmatic TestRunner setup with plugins. Run all coverage code via mcp__matlab__evaluate_matlab_code — the mcp__matlab__run_matlab_test_file tool does not support coverage plugins.

Use CodeCoveragePlugin.forFolder with CoverageResult as the single output format:

import matlab.unittest.TestRunner
import matlab.unittest.TestSuite
import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult

suite = TestSuite.fromFolder("tests");
runner = TestRunner.withTextOutput;

format = CoverageResult;
runner.addPlugin(CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, Producing=format, MetricLevel="mcdc"));

results = runner.run(suite);
coverageResult = format.Result;  % Object array — one element per source file

Verify results has no Failed or Incomplete outcomes before analyzing coverage — failed tests produce incomplete coverage data. If tests fail, fix them first and re-run.

3. Report summary

coverageResult is an object array with one element per instrumented source file. coverageSummary accepts the full array and returns an Nx2 matrix (one row per file: [achieved, total]).

Report all metric levels up to the collected level — call coverageSummary once per level. Present the results in a single combined table with one row per file and one column per metric level.

Collected MetricLevelReport these levels
"statement"statement
"decision"statement, decision
"condition"statement, decision, condition
"mcdc"statement, decision, condition, mcdc
% 1-output form: Nx2 matrix [achieved, total] per file
summary = coverageSummary(coverageResult, "mcdc");
fprintf('Coverage: %d/%d (%.1f%%)\n', sum(summary(:,1)), sum(summary(:,2)), ...
    100*sum(summary(:,1))/sum(summary(:,2)));

After reporting the summary, tell the user the coverage percentage and ask what they'd like to do next: analyze gaps, generate a report, or write tests.

4. Act on results (user-directed)

Based on user request:

  • Generate interactive report — generateHTMLReport(coverageResult, "coverage-report") (folder-based, browseable per-file view)

  • Generate portable report — generateStandaloneReport(coverageResult, "report.html") (single file, shareable)

  • Generate CI artifact — generateCoberturaReport(coverageResult, "coverage.xml")

  • Analyze gaps — see Gap Analysis pattern below

  • Justify unreachable code — see Justification pattern (only when user asks)

  • Write tests for gaps — see Test Generation pattern

Alternatively, produce the interactive report during collection using CoverageReport:

import matlab.unittest.plugins.codecoverage.CoverageReport

runner.addPlugin(CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, ...
    Producing=[format, CoverageReport("coverage-report")], ...
    MetricLevel="mcdc"));

Key Functions

FunctionPurposeRequires
CodeCoveragePlugin.forFolderInstrument source folder for coverage(core)
CodeCoveragePlugin.forFileInstrument specific files(core)
CodeCoveragePlugin.forNamespaceInstrument source code in a specified namespace(core)
coverageSummary(result, level)Get coverage counts + gap details(core)
generateHTMLReportInteractive HTML report (folder-based, browseable)(core)
generateStandaloneReportSingle-file HTML report (portable, shareable)(core)
generateCoberturaReportCobertura XML for CI(core)
CodeCoveragePlugin(..., Filter=file)Apply filter rules during collectionMATLAB Test
result.applyFilter(file)Apply filter rules to existing resultsMATLAB Test
result.resetFilterRemove applied filters, return unfiltered resultsMATLAB Test
GeneratedCodeCoveragePluginCoverage for generated C codeMATLAB Test + MATLAB Coder

Patterns

Basic Coverage Collection

Always use forFolder with IncludingSubfolders=true for projects with nested source directories. Use forFile only when measuring specific individual files. Use forNamespace when source code is organized in namespaces (+packageName folders).

% CORRECT: covers all .m files in src/ and its subfolders
plugin = CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, Producing=format, MetricLevel="mcdc");

% For namespace-organized code: covers all files in +myPackage and inner namespaces
plugin = CodeCoveragePlugin.forNamespace("myPackage", ...
    IncludingInnerNamespaces=true, Producing=format, MetricLevel="mcdc");

% WRONG: verbose manual enumeration — don't do this
% srcFiles = dir(fullfile("src", '**', '*.m'));
% plugin = CodeCoveragePlugin.forFile(filePaths, ...);

MetricLevel values: "statement", "decision", "condition", "mcdc". Default to "mcdc" when MATLAB Test is installed, "statement" otherwise.

Each level includes all levels below it — "mcdc" gives you statement + decision + condition + MC/DC.

Release compatibility: On R2026b+, MetricLevel is deprecated in CodeCoveragePlugin — use Metrics instead. Metrics also supports "type-size" (see Data Type and Size Coverage below). When unsure of the release, use MetricLevel — it works on all supported versions.

% R2026b+ only: Metrics replaces MetricLevel for CodeCoveragePlugin
plugin = CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, Producing=format, Metrics="mcdc");
Generated C Code Coverage

For generated code from MATLAB Coder, use GeneratedCodeCoveragePlugin (not CodeCoveragePlugin). Always pass Producing=format to get programmatic access to results — without it, coverage is collected but not accessible.

import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageResult

suite = TestSuite.fromClass(?myCoderEquivalenceTest);
runner = TestRunner.withTextOutput;

format = CoverageResult;
runner.addPlugin(GeneratedCodeCoveragePlugin( ...
    MetricLevel="condition", Producing=format));

results = runner.run(suite);
coverageResult = format.Result;  % Object array, same as CodeCoveragePlugin

PreserveInFolder is only required for Cobertura XML output. When producing CoberturaFormat, the test must use PreserveInFolder in its build() call so build artifacts (C source files) remain on disk for report generation:

import matlab.unittest.plugins.codecoverage.CoberturaFormat

% Only needed when producing Cobertura XML:
runner.addPlugin(GeneratedCodeCoveragePlugin( ...
    MetricLevel="condition", Producing=CoberturaFormat("coverage.xml")));

% The test class must call:
%   testCase.build(fcn, Inputs=args, Configuration=cfg, PreserveInFolder=folder)
Data Type and Size Coverage

Tracks which (type, size) combinations of input arguments are exercised. Uses the Metrics parameter with "type-size" as an additional metric alongside a structural metric.

Available from R2026b. Check availability before using:

plugin = CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, Producing=format, ...
    Metrics=["statement", "type-size"]);

Note: "type-size" cannot be used alone — it must be combined with a structural metric ("statement", "decision", "condition", or "mcdc").

For extracting and interpreting the results, see datatype-size-extraction.md — consult when the user asks about type-size coverage data or input diversity.

Gap Analysis

Use the 2-output form of coverageSummary to get detailed gap information:

[summary, description] = coverageSummary(coverageResult, "mcdc");

% description.mcdc contains per-decision detail:
mcdcDetails = description.mcdc;
for i = 1:numel(mcdcDetails)
    conditions = mcdcDetails(i).Condition;
    for j = 1:numel(conditions)
        if ~conditions(j).Achieved
            fprintf('Decision %d, Cond %d: "%s" — needs: %s\n', ...
                i, j, conditions(j).Text, conditions(j).FalseResult);
        end
    end
end

For decision-level gaps:

[summary, description] = coverageSummary(coverageResult, "decision");
decisions = description.decision;
for i = 1:numel(decisions)
    for j = 1:numel(decisions(i).Outcome)
        if decisions(i).Outcome(j).ExecutionCount == 0
            fprintf('Decision %d: %s outcome uncovered\n', ...
                i, decisions(i).Outcome(j).Text);
        end
    end
end
printCoverageGaps

Prints all uncovered items across all metric levels (statement, function, decision, condition, MC/DC) present in the coverage result.

Script: ../scripts/printCoverageGaps.m

InputcovResults — matlab.coverage.Result array from CoverageResult format
OutputPrints to stdout; no return value
printCoverageGaps(covResults);
Programmatic Justification

Justify structurally unreachable code when the user requests it. Justifications can only be applied up to decision level — condition and MC/DC outcomes cannot be justified. This is a human-in-the-loop workflow: highlight the gaps, then justify when the user confirms.

The approach: create a filter rule XML file that can be reused across runs. This is preferable to constructing justification objects programmatically because the XML file is version-controllable, shareable, and can be supplied at collection time or applied after.

For the full XML schema, attribute reference, and usage patterns, see justification-filter-rules.md — consult whenever applying justifications.

Quick example:

% 1. Create the filter rule XML file
filterXml = fullfile(projectRoot, "resources", "CodeCoverageFilter.xml");
xmlContent = [ ...
    '<?xml version="1.0" encoding="utf-8"?>', newline, ...
    '<filter>', newline, ...
    '    <rule type="DECISION" fileName="shortest_path.m" ', ...
    'functionName="shortest_path" ', ...
    'expr="if ~isNodeValid(endIdx)" index="3" ', ...
    'mode="JUSTIFIED" ', ...
    'rationale="Structurally unreachable: early return prevents reaching this branch"/>', newline, ...
    '</filter>'];
writelines(string(xmlContent), filterXml);

% 2a. Supply at collection time (justifications applied during run)
plugin = CodeCoveragePlugin.forFolder("src", ...
    IncludingSubfolders=true, Producing=format, ...
    MetricLevel="decision", Filter=filterXml);

% 2b. Or apply after collection
filteredResult = coverageResult.applyFilter(filterXml);
generateStandaloneReport(filteredResult, "report.html");
Coverage-Driven Test Generation

When the user wants to close coverage gaps, follow this pattern:

  1. Collect coverage and extract gaps using coverageSummary 2-output form

  2. For each uncovered condition/decision, analyze the source code to determine what input would trigger it

  3. Write targeted test methods that exercise the specific uncovered path

  4. Re-run coverage to verify improvement

Do not use internal matlabtest.internal.testcreation.* APIs to generate tests (they produce empty stubs only). Write test methods directly instead.

% After identifying: condition "windowSize < 1" is uncovered (false branch)
% Write a test that makes windowSize < 1 to trigger that branch:

function testWindowSizeLessThanOne(testCase)
    data = randn(100, 1);
    testCase.verifyError(@() movingAverageFilter(data, 0), ...
        'movingAverageFilter:invalidWindow');
end

Common Mistakes

MistakeWhy It's WrongCorrect Approach
dir+arrayfun+forFile for multi-folderVerbose, fragile with package dirsforFolder(path, IncludingSubfolders=true)
CodeCoveragePlugin for generated C codeOnly instruments MATLAB codematlabtest.coder.plugins.GeneratedCodeCoveragePlugin
Missing PreserveInFolder when using CoberturaFormat with generated codeBuild artifacts cleaned up, empty Cobertura reportAdd PreserveInFolder=folder to testCase.build() (only needed for Cobertura XML)
Using Metrics= without checking releaseMetrics only works on R2026b+; errors on older releasesUse MetricLevel by default (works on all versions); use Metrics only when you confirm R2026b+
Metrics="type-size" aloneMust combine with a structural metricMetrics=["statement","type-size"]
Treating coverageSummary 2nd output as numeric2-output form returns [summary, description] — description is a struct, not a matrixUse 1-output summary = coverageSummary(result, level) for totals; 2-output only for gap analysis
Using coverageResult.FilesProperty does not existUse coverageResult(i).Filename in a loop, or [coverageResult.Filename] to get all paths
Generating multiple report formats unrequestedWastes tokens and confuses userUse CoverageResult only; generate reports when asked
Using matlabtest.coverage.Justification constructorError-prone 8-arg API, hard to reuseWrite a filter rule XML file and pass via Filter= or applyFilter
Lowercase type in filter rule XMLtype attribute requires uppercaseUse type="DECISION", type="MCDC_OUTCOME"

Conventions

  • Always: use CoverageResult as the primary output — produce reports only when the user asks

  • Always: report coverage numbers to the user before taking further action

  • Always: when MATLAB Test is not installed, fall back to statement-level coverage only

  • Prefer: forFolder with IncludingSubfolders=true as the default source specification

  • Prefer: advanced metrics (decision, MC/DC) — MATLAB Test provides capabilities beyond what most coverage tools offer

  • Prefer: MC/DC as the default metric level because each level includes all lower levels

  • Never: justify coverage proactively — wait for user to request justification

  • Never: use internal matlabtest.internal.* APIs


Copyright 2026 The MathWorks, Inc.


Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
MathWorks
Publisher
MathWorks (Official)
Source
matlab/matlab-agentic-toolkit · skills-catalog/matlab-software-development/matlab-run-tests · commit 2a7f215
Licence
BSD 2-Clause License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE.md)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by MathWorks.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download matlab-run-tests.zip