Medtech Model Evidence Export

Medtech Model Evidence Export to MLflow

Purpose

Mirror an existing medical-inference result or evidence pack into MLflow after the run and emit the export_result JSON contract. Keep the original evidence pack as the source of truth. Training skills should add MLflow inside their training loops instead.

Instructions

1. Run scripts/export_evidence_pack.py in the default dry-run mode.

2. Inspect params, metrics, artifact_plan, and mlflow.note.content.

3. Choose --mode local or --mode databricks only after checking the target.

4. Keep --artifact-policy metadata unless the target is approved for images.

5. For preview or all in a live mode, also pass --confirm-medical-artifact-upload.

6. Keep --source-ref, --note, config filenames, and artifact filenames free of patient or secret identifiers; always review the dry-run output first.

Hosts with a script helper can use run_script("scripts/export_evidence_pack.py", args=["PACK_OR_RESULT", "--mode", "dry-run"]).

Available Scripts

Prerequisites

• Python 3.10+.

• mlflow>=2.10,<4 for local or databricks mode.

• numpy>=1.24,<3 and nibabel>=4,<6 for NIfTI quality metrics and previews.

• MLFLOW_TRACKING_URI may select a caller-managed tracking server.

• Databricks mode uses the caller's DATABRICKS_HOST, DATABRICKS_TOKEN, or configured Databricks profile. The declared network endpoint is https://<caller-provided-mlflow-or-databricks-workspace>; Docker and GPU are not required.

• Local mode may write the MLflow store under <current-working-directory>/mlruns.

Examples

Preview the export without contacting MLflow:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/inference_pack --mode dry-run --artifact-policy metadata

Export a direct NV-Generate result with reproducibility metadata:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/nv-generate/result.json \
  --mode local \
  --experiment-name medical-ai-inference \
  --config configs/chest_lung_tumor.json \
  --seed 0 \
  --source-ref git:61c4ec709b84cad468852243c48e250bec732074

Log downsampled slice previews, but not raw NIfTI files:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/nv-generate/result.json \
  --mode databricks \
  --experiment-name /Shared/medical-ai-inference \
  --artifact-policy preview \
  --confirm-medical-artifact-upload

--artifact-policy all additionally uploads discovered or explicitly supplied NIfTI images and masks, subject to --max-artifact-mb. Use --image and --mask when paths are not present in the result JSON.

The exporter logs:

• scalar run and quality metrics, including sampled HU mean/std/min/max for CT (generic intensity statistics otherwise), a documented intensity-SNR heuristic, mask foreground percentage, and mapped tumor volume percentage when a tumor label mapping is available;

• generation parameters, model/checkpoint identity, RNG seed, and recipe hash;

• source config digest or --source-ref, plus a prompt digest when present;

• mlflow.note.content with a short human-readable run summary;

• a sanitized metadata bundle by default, optional PNG slice previews, and raw image/mask artifacts only under the explicit all policy.

Limitations

• This is post-hoc inference export, not live training-curve tracking.

• Global intensity SNR and downsampled volume statistics are engineering checks, not image-quality or clinical-performance claims.

• Preview and raw artifacts may contain sensitive medical information. The caller must approve the destination and data policy before upload.

• The exporter does not evaluate model quality, register models, or alter the source evidence pack.

Troubleshooting