AutoGineer
Generated banner for Huggingface Trackio
Generated logo for Huggingface Trackio

Huggingface Trackio

Hugging FaceOfficialPublished by Hugging Face

  • Skill
  • Tool integration
  • Data and AI
  • Apache-2.0
  • Medium risk
Download ZIP

From the author

Track and visualize ML training experiments with Trackio. Use when logging metrics during training (Python API), firing alerts for training diagnostics, or retrieving/analyzing logged metrics (CLI). Supports real-time dashboard visualization, alerts with webhooks, HF Space syncing, and JSON output for automation.
— the description by Hugging Face

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Trackio provides a Python API and CLI for tracking machine learning experiments. It supports local metric storage, real-time dashboard visualization, and synchronization with Hugging Face Spaces. Designed with a focus on autonomous workflows, it allows developers and LLM agents to log metrics, fire diagnostic alerts, and retrieve training data programmatically via JSON output.

What it does

  • Logs training metrics and hyperparameters using a Python API.
  • Provides real-time experiment visualization via local or remote Hugging Face Space dashboards.
  • Enables diagnostic alerting with severity levels (INFO, WARN, ERROR) and webhook support.
  • Allows programmatic retrieval of metrics and alerts via CLI with JSON output.
  • Supports automatic integration with TRL trainers.
  • Syncs local experiment data to persistent remote storage.

When to use it

  • When you need to track and visualize metrics during ML model training.
  • When building autonomous agents that need to monitor training health and iterate on hyperparameters.
  • When you require persistent dashboards for cloud-based training runs.
  • When you need to integrate training diagnostics with external notification systems like Slack or Discord.

Advantages

  • Lightweight and local-first design.
  • API-compatible with existing experiment tracking tools.
  • Enables autonomous experiment iteration through structured alerts.
  • Supports both local and cloud-based monitoring workflows.
  • Provides structured JSON output for easy integration with automation scripts.

Usage examples

“How do I log training metrics with Trackio?”
Initialize the run with trackio.init(), use trackio.log() to record metrics during your loop, and call trackio.finish() to save the data.
“How can an agent monitor for training failures?”
The agent can use trackio.alert() in the training script to flag issues, then poll for these alerts using the CLI command 'trackio list alerts --json'.
“How do I sync my local training metrics to a Hugging Face Space?”
Pass the space_id parameter to trackio.init() during initialization, or use trackio.sync() to upload an existing local project.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/huggingface/huggingface-trackio/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip huggingface-trackio.zip -d .agents/skills

Documentation by Hugging Face

Trackio - Experiment Tracking for ML Training

Trackio is an experiment tracking library for logging and visualizing ML training metrics. It syncs to Hugging Face Spaces for real-time monitoring dashboards.

Three Interfaces

TaskInterfaceReference
Logging metrics during trainingPython APIreferences/logging_metrics.md
Firing alerts for training diagnosticsPython APIreferences/alerts.md
Retrieving metrics & alerts after/during trainingCLIreferences/retrieving_metrics.md

When to Use Each

Python API → Logging

Use import trackio in your training scripts to log metrics:

  • Initialize tracking with trackio.init()

  • Log metrics with trackio.log() or use TRL's report_to="trackio"

  • Finalize with trackio.finish()

Key concept: For remote/cloud training, pass space_id — metrics sync to a Space dashboard so they persist after the instance terminates. Auto-created Spaces are public by default — pass private=True if the metrics should not be public.

→ See references/logging_metrics.md for setup, TRL integration, and configuration options.

Python API → Alerts

Insert trackio.alert() calls in training code to flag important events — like inserting print statements for debugging, but structured and queryable:

  • trackio.alert(title="...", level=trackio.AlertLevel.WARN) — fire an alert

  • Three severity levels: INFO, WARN, ERROR

  • Alerts are printed to terminal, stored in the database, shown in the dashboard, and optionally sent to webhooks (Slack/Discord)

Key concept for LLM agents: Alerts are the primary mechanism for autonomous experiment iteration. An agent should insert alerts into training code for diagnostic conditions (loss spikes, NaN gradients, low accuracy, training stalls). Since alerts are printed to the terminal, an agent that is watching the training script's output will see them automatically. For background or detached runs, the agent can poll via CLI instead.

→ See references/alerts.md for the full alerts API, webhook setup, and autonomous agent workflows.

CLI → Retrieving

Use the trackio command to query logged metrics and alerts:

  • trackio list projects/runs/metrics — discover what's available

  • trackio get project/run/metric — retrieve summaries and values

  • trackio list alerts --project <name> --json — retrieve alerts

  • trackio show — launch the dashboard

  • trackio sync — sync to HF Space

Key concept: Add --json for programmatic output suitable for automation and LLM agents.

→ See references/retrieving_metrics.md for all commands, workflows, and JSON output formats.

Minimal Logging Setup

import trackio

# Spaces are PUBLIC by default (good for shareable dashboards);
# pass private=True if the metrics should not be public
trackio.init(project="my-project", space_id="username/trackio", private=True)
trackio.log({"loss": 0.1, "accuracy": 0.9})
trackio.log({"loss": 0.09, "accuracy": 0.91})
trackio.finish()
Minimal Retrieval
trackio list projects --json
trackio get metric --project my-project --run my-run --metric loss --json

Autonomous ML Experiment Workflow

When running experiments autonomously as an LLM agent, the recommended workflow is:

  1. Set up training with alerts — insert trackio.alert() calls for diagnostic conditions

  2. Launch training — run the script in the background

  3. Poll for alerts — use trackio list alerts --project <name> --json --since <timestamp> to check for new alerts

  4. Read metrics — use trackio get metric ... to inspect specific values

  5. Iterate — based on alerts and metrics, stop the run, adjust hyperparameters, and launch a new run

import trackio

trackio.init(project="my-project", config={"lr": 1e-4})

for step in range(num_steps):
    loss = train_step()
    trackio.log({"loss": loss, "step": step})

    if step > 100 and loss > 5.0:
        trackio.alert(
            title="Loss divergence",
            text=f"Loss {loss:.4f} still high after {step} steps",
            level=trackio.AlertLevel.ERROR,
        )
    if step > 0 and abs(loss) < 1e-8:
        trackio.alert(
            title="Vanishing loss",
            text="Loss near zero — possible gradient collapse",
            level=trackio.AlertLevel.WARN,
        )

trackio.finish()

Then poll from a separate terminal/process:

trackio list alerts --project my-project --json --since "2025-01-01T00:00:00"

Files

references/logging_metrics.md

Logging Metrics with Trackio

Trackio is a lightweight, free experiment tracking library from Hugging Face. It provides a wandb-compatible API for logging metrics with local-first design.

Installation

pip install trackio
# or
uv pip install trackio

Core API

Basic Usage
import trackio

# Initialize a run
trackio.init(
    project="my-project",
    config={"learning_rate": 0.001, "epochs": 10}
)

# Log metrics during training
for epoch in range(10):
    loss = train_epoch()
    trackio.log({"loss": loss, "epoch": epoch})

# Finalize the run
trackio.finish()
Key Functions
FunctionPurpose
trackio.init(...)Start a new tracking run
trackio.log(dict)Log metrics (called repeatedly during training)
trackio.finish()Finalize run and ensure all metrics are saved
trackio.show()Launch the local dashboard
trackio.sync(...)Sync local project to HF Space

trackio.init() Parameters

trackio.init(
    project="my-project",           # Project name (groups runs together)
    name="run-name",                # Optional: name for this specific run
    config={...},                   # Hyperparameters and config to log
    space_id="username/trackio",    # Optional: sync to HF Space for remote dashboard
    private=True,                   # Optional: make an auto-created Space private.
                                    #   Default: PUBLIC (unless your org defaults to private)
    bucket_id="username/my-bucket", # Optional: pin the HF Bucket used for metric storage.
                                    #   Default: auto-derived from space_id
    group="experiment-group",       # Optional: group related runs
)

Local vs Remote Dashboard

Local (Default)

By default, trackio stores metrics in a local SQLite database and runs the dashboard locally:

trackio.init(project="my-project")
# ... training ...
trackio.finish()

# Launch local dashboard
trackio.show()

Or from terminal:

trackio show --project my-project
Remote (HF Space)

Pass space_id to sync metrics to a Hugging Face Space for persistent, shareable dashboards:

trackio.init(
    project="my-project",
    space_id="username/trackio",  # Auto-creates Space if it doesn't exist
    private=True,                 # Spaces are PUBLIC by default; omit for a shareable dashboard
)

⚠️ For remote training (cloud GPUs, HF Jobs, etc.): Always use space_id since local storage is lost when the instance terminates. If the metrics should not be public, also pass private=True — an auto-created Space is public by default (unless your org's default is private); the flag is ignored if the Space already exists.

Sync Local to Remote

Sync existing local projects to a Space:

trackio.sync(project="my-project", space_id="username/my-experiments")

wandb Compatibility

Trackio is API-compatible with wandb. Drop-in replacement:

import trackio as wandb

wandb.init(project="my-project")
wandb.log({"loss": 0.5})
wandb.finish()

TRL Integration

When using TRL trainers, set report_to="trackio" for automatic metric logging:

from trl import SFTConfig, SFTTrainer
import trackio

trackio.init(
    project="sft-training",
    space_id="username/trackio",
    private=True,  # Spaces are public by default; omit for a shareable dashboard
    config={"model": "Qwen/Qwen2.5-0.5B", "dataset": "trl-lib/Capybara"}
)

config = SFTConfig(
    output_dir="./output",
    report_to="trackio",  # Automatic metric logging
    # ... other config
)

trainer = SFTTrainer(model=model, args=config, ...)
trainer.train()
trackio.finish()

What Gets Logged

With TRL/Transformers integration, trackio automatically captures:

  • Training loss

  • Learning rate

  • Eval metrics

  • Training throughput

For manual logging, log any numeric metrics:

trackio.log({
    "train_loss": 0.5,
    "train_accuracy": 0.85,
    "val_loss": 0.4,
    "val_accuracy": 0.88,
    "epoch": 1
})

Grouping Runs

Use group to organize related experiments in the dashboard sidebar:

# Group by experiment type
trackio.init(project="my-project", name="baseline-v1", group="baseline")
trackio.init(project="my-project", name="augmented-v1", group="augmented")

# Group by hyperparameter
trackio.init(project="hyperparam-sweep", name="lr-0.001", group="lr_0.001")
trackio.init(project="hyperparam-sweep", name="lr-0.01", group="lr_0.01")

Configuration Best Practices

Keep config minimal — only log what's useful for comparing runs:

trackio.init(
    project="qwen-sft-capybara",
    name="baseline-lr2e5",
    config={
        "model": "Qwen/Qwen2.5-0.5B",
        "dataset": "trl-lib/Capybara",
        "learning_rate": 2e-5,
        "num_epochs": 3,
        "batch_size": 8,
    }
)

Embedding Dashboards

Embed Space dashboards in websites with query parameters:

<iframe 
  src="https://username-trackio.hf.space/?project=my-project&metrics=train_loss,val_loss&sidebar=hidden" 
  style="width:1600px; height:500px; border:0;">
</iframe>

Query parameters:

  • project: Filter to specific project

  • metrics: Comma-separated metric names to show

  • sidebar: hidden or collapsed

  • smoothing: 0-20 (smoothing slider value)

  • xmin, xmax: X-axis limits

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

  • Possible data-exfiltration endpoints
  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Hugging Face
Publisher
Hugging Face (Official)
Source
huggingface/skills · skills/huggingface-trackio · commit 25910b3
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Hugging Face.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download huggingface-trackio.zip