Train Sentence Transformers

Hugging FaceOfficialPublished by Hugging Face

Skill
Workflow
Data and AI
Apache-2.0
Medium risk
Has scripts

From the author

Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.

— the description by Hugging Face

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a standardized approach to training various embedding and ranking models using the sentence-transformers framework. It acts as a routing system that directs users to specific production templates and technical references based on the chosen model architecture. The workflow covers the entire lifecycle from model selection and data preparation to loss function configuration, evaluation, and publishing to the Hugging Face Hub.

What it does

Routes users to appropriate training templates for bi-encoders, cross-encoders, sparse encoders, and multi-vector encoders.
Provides technical references for loss functions, evaluators, and model architectures.
Offers guidance on hard-negative mining and dataset formatting for training.
Includes instructions for hardware-specific configurations and Hugging Face Hub integration.
Defines mandatory script requirements such as smoke testing, logging, and verdict reporting.

When to use it

When you need to train or fine-tune a sentence-transformers model for retrieval, similarity, or ranking tasks.
When you are unsure which model architecture (e.g., bi-encoder vs. cross-encoder) is best suited for your specific use case.
When you require a structured, repeatable process for training and evaluating embedding models.
When you need to integrate advanced training techniques like distillation, LoRA, or Matryoshka embeddings.

Advantages

Enforces consistent training practices through standardized production templates.
Reduces common errors by providing specific references for loss-to-data-shape mapping.
Facilitates systematic experimentation with built-in logging and verdict reporting.
Supports a wide range of model types including modern late-interaction and sparse architectures.

Usage examples

“I need to train a bi-encoder for semantic search on my internal documentation.”: The agent will identify the [SentenceTransformer] type, provide the relevant loss and evaluator references, and guide you to the bi-encoder production template.
“How do I implement a two-stage retrieval system using a reranker?”: The agent will direct you to the [CrossEncoder] documentation and template, explaining how to use it to score pairs for two-stage retrieval.
“I want to train a SPLADE model for sparse retrieval.”: The agent will provide the [SparseEncoder] references, including the required SpladeLoss wrapper and specific evaluators for sparse retrieval.
“How can I improve my retrieval model using hard negatives?”: The agent will guide you to the hard-negative mining script and explain how to generate n-tuple datasets for training.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/huggingface/train-sentence-transformers/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip train-sentence-transformers.zip -d .agents/skills

Documentation by Hugging Face

Train a sentence-transformers Model

This SKILL.md is a router, not a manual. It tells you which references and example scripts to load for your task. The actual content (recommended losses, evaluators, training-script structure, model selection, training-arg knobs, troubleshooting) lives in references/ and scripts/.

Do not synthesize a training script from this file alone. Open the per-type production template (scripts/train_<type>_example.py) and copy it as your starting point. The templates contain load-bearing scaffolding (autocast helper, model-card class, logger silencing list, force=True, seed, TF32, version-compatible imports, named-evaluator metric handling) that prior agent runs have repeatedly missed when rolling their own from a synthesized snippet.

1. Identify the model type

Tag	Class	What it does	When to pick
[SentenceTransformer]	`SentenceTransformer` (bi-encoder)	Maps each input to a fixed-dim dense vector	Retrieval, similarity, clustering, classification, paraphrase mining, dedup
[CrossEncoder]	`CrossEncoder` (reranker)	Scores `(query, passage)` pairs jointly	Two-stage retrieval (rerank top-100 from bi-encoder), pair classification
[SparseEncoder]	`SparseEncoder` (SPLADE)	Sparse vectors over the vocabulary	Learned-sparse retrieval, inverted-index backends (Elasticsearch / OpenSearch / Lucene)
[MultiVectorEncoder]	`MultiVectorEncoder` (ColBERT)	One embedding per token, scored with MaxSim	Late-interaction retrieval, recall gains over bi-encoders at higher storage cost, multimodal (ColPali / ColQwen2)

Tiebreakers when the request is ambiguous: "embedding model" / "vector search" / "similarity" → [SentenceTransformer]. "rerank" / "ranker" / "two-stage" → [CrossEncoder]. "SPLADE" / "sparse" / "inverted index" → [SparseEncoder]. "ColBERT" / "late interaction" / "multi-vector" / "MaxSim" / "ColPali" / "ColQwen" → [MultiVectorEncoder]. If still unclear, ask.

2. Required reading

Read these in full before writing any code. Do not triage by perceived relevance.

Per-type: always required

[SentenceTransformer]

references/losses_sentence_transformer.md: loss-to-data-shape mapping, BatchSamplers.NO_DUPLICATES requirement for MNRL-family, Cached* ↔ gradient_checkpointing incompatibility.
references/evaluators_sentence_transformer.md: evaluator-to-task mapping, metric_for_best_model key construction (named vs unnamed), per-evaluator primary_metric values.
references/model_architectures.md: encoder vs decoder vs static vs Router pipelines, pooling rules (mean / cls / lasttoken), auto-mean-pooling behavior for fresh-start MLM bases.
scripts/train_sentence_transformer_example.py: production template. Copy this as your starting point.

[CrossEncoder]

references/losses_cross_encoder.md: pointwise / pairwise / listwise / distillation, pos_weight derivation, activation_fn=Identity() mandatory for non-BCE losses (silent eval-rank collapse otherwise).
references/evaluators_cross_encoder.md: CrossEncoderRerankingEvaluator recipe, named-evaluator key format eval_{name}_{primary_metric}.
scripts/train_cross_encoder_example.py: production template. Copy this as your starting point.

[SparseEncoder]

references/losses_sparse_encoder.md: SpladeLoss wrapper requirement, FLOPS regularizer weights, smoke-test active-dim ramp behavior.
references/evaluators_sparse_encoder.md: SparseNanoBEIREvaluator (English-only) and the in-domain alternative, eval_{name}_{primary_metric} key format.
scripts/train_sparse_encoder_example.py: production template. Copy this as your starting point.

[MultiVectorEncoder]

references/losses_multi_vector_encoder.md: MaxSim scoring, scale choice per scoring mode (scale=1.0 for MaxSim, roughly the average query length for MeanMaxSim), MNRL / CachedMNRL / MarginMSE / DistillKLDiv, XTR-vs-ColBERT scoring, CachedMNRL ↔ gradient_checkpointing incompatibility.
references/evaluators_multi_vector_encoder.md: MultiVectorNanoBEIREvaluator (English-only) and the in-domain alternative, eval_NanoBEIR_mean_maxsim_ndcg@10 key format, distillation-eval spearman variant.
scripts/train_multi_vector_encoder_example.py: production template. Copy this as your starting point.

Cross-cutting: always required (regardless of task)

references/training_args.md: TrainingArguments knobs, precision rules (load fp32 + autocast bf16/fp16, never torch_dtype=bfloat16), warmup_steps (float) vs deprecated warmup_ratio, save_steps must be a multiple of eval_steps for load_best_model_at_end, schedulers, HPO, tracker, resume, hub-push variants.
references/dataset_formats.md: column-matching rules (label name auto-detection, column-order-not-name), reshaping recipes, hard-negative mining options.
references/base_model_selection.md: discovery commands, per-type model namespaces, ModernBERT-family max_seq_length=8192 trap, datasets >= 4 script-loader rejection, non-English starting-point shortcuts.
references/troubleshooting.md: symptom-indexed failure recipes. Skim the section headings on every run, even a healthy one. The "Metrics don't improve" and "Hub push fails" entries cover bugs that bite frequently and are cheaper to recognize before they fire than to debug after.

Cross-cutting: load when applicable

references/hardware_guide.md: VRAM sizing, multi-GPU, FSDP / DeepSpeed, HF Jobs flavors. Required for >24GB models, multi-GPU, or HF Jobs runs.
references/hf_jobs_execution.md: required when running on HF Jobs.
references/prompts_and_instructions.md: required when using prompt-tuned bases (E5, BGE, GTE, Qwen3-Embedding, Instructor, Nomic, etc.) or adding query: / passage: style prefixes.

Variant scripts (open when the task matches)

[SentenceTransformer] scripts/train_sentence_transformer_<matryoshka|multi_dataset|with_lora|distillation|make_multilingual|static_embedding>_example.py.
[CrossEncoder] scripts/train_cross_encoder_<distillation|listwise>_example.py.
[SparseEncoder] scripts/train_sparse_encoder_distillation_example.py.
Hard-negative mining CLI: scripts/mine_hard_negatives.py.

3. Defaults

Override only if the user specifies otherwise:

Local execution. Pitch HF Jobs only if local hardware can't fit the job.
Single run. After it completes, propose experimentation if the user would benefit (weak/marginal verdict, "see how high you can push it" framing, etc.). Iteration rules in references/training_args.md (Experimentation section).
Public Hub push at end-of-run, wrapped in try-except. On HF Jobs (ephemeral env) ALSO enable in-trainer push (push_to_hub=True + hub_strategy="every_save"). Details in references/hf_jobs_execution.md.

4. Constraints the produced script must satisfy

These are non-negotiable contracts. Implementation lives in the production templates and references. Do not reinvent.

Capture the pre-training evaluator score as baseline_eval before trainer.train().
Emit a single end-of-run line: VERDICT: WIN|MARGINAL|REGRESSION | score=... | baseline=... | delta=.... A monitor scrapes for this.
Silence httpx, httpcore, huggingface_hub, urllib3, filelock, fsspec to WARNING (otherwise HF download URLs flood the agent's context).
Tee logs to logs/{RUN_NAME}.log.
End with model.push_to_hub(...) wrapped in try/except.
Smoke-test before any long run (max_steps=1 + tiny dataset slice). The production templates show one common pattern (SMOKE_TEST env var).
[CrossEncoder] Include EarlyStoppingCallback(patience>=3). CE rerankers often peak mid-training and regress.
[SparseEncoder] Log query_active_dims / corpus_active_dims on the verdict line. High nDCG with collapsed sparsity is not a win. The keys come back name-prefixed (e.g. ..._query_active_dims). Use suffix matching to pluck them. See the SPARSE production template for the exact pattern.
[MultiVectorEncoder] Match scale to the scoring mode on any MNRL-family loss: near 1.0 for unnormalized MaxSim (do not copy scale=20.0 from bi-encoder MNRL), roughly the average query length with length-normalized MeanMaxSim, since each score is divided by its query's token count. XTRScores is a train-only similarity_fct: the evaluators reject it, so evaluation always scores with MaxSim, including for XTR-trained models.

5. Workflow

Identify the model type (§1). Ask if ambiguous.
Load the §2 required-reading files for that type.
Open scripts/train_<type>_example.py and copy it as your starting point.
Replace MODEL_NAME, DATASET_NAME, RUN_NAME, the loss, and the evaluator with the user's task. Cross-check loss/data-shape match against references/losses_<type>.md. Cross-check the metric_for_best_model key against references/evaluators_<type>.md (named evaluators format the key as eval_{name}_{primary_metric}).
Smoke-test (max_steps=1).
Run.
After the run, append to logs/experiments.md and propose iteration if the verdict is weak/marginal.

Prerequisites

pip install "sentence-transformers[train]>=5.0"        # add [train,image] / [audio] / [video] for [SentenceTransformer] multimodal
                                                       # [MultiVectorEncoder] requires >=6.0
pip install trackio                                    # optional tracker (or wandb / tensorboard / mlflow)
hf auth login                                          # or set HF_TOKEN with write scope (for Hub push)

GPU strongly recommended. CPU works only for demos and [SentenceTransformer] StaticEmbedding.

Files

ATTRIBUTION.mdAdded by AutoGineer898 B
LICENSELicence file added by AutoGineer11.1 KiB
SKILL.md10.3 KiB
references/base_model_selection.md7.5 KiB
references/dataset_formats.md6.3 KiB
references/evaluators_cross_encoder.md5.5 KiB
references/evaluators_multi_vector_encoder.md6.6 KiB
references/evaluators_sentence_transformer.md5.8 KiB
references/evaluators_sparse_encoder.md4.9 KiB
references/hardware_guide.md5.9 KiB
references/hf_jobs_execution.md7.1 KiB
references/losses_cross_encoder.md9.5 KiB
references/losses_multi_vector_encoder.md10.5 KiB
references/losses_sentence_transformer.md10.7 KiB
references/losses_sparse_encoder.md5.1 KiB
references/model_architectures.md9.0 KiB
references/prompts_and_instructions.md6.4 KiB
references/training_args.md15.0 KiB
references/troubleshooting.md12.9 KiB
scripts/mine_hard_negatives.pyScript8.3 KiB
scripts/train_cross_encoder_distillation_example.pyScript9.2 KiB
scripts/train_cross_encoder_example.pyScript9.0 KiB
scripts/train_cross_encoder_listwise_example.pyScript10.2 KiB
scripts/train_multi_vector_encoder_example.pyScript8.8 KiB
scripts/train_sentence_transformer_distillation_example.pyScript11.9 KiB
scripts/train_sentence_transformer_example.pyScript7.5 KiB
scripts/train_sentence_transformer_make_multilingual_example.pyScript11.5 KiB
scripts/train_sentence_transformer_matryoshka_example.pyScript7.1 KiB
scripts/train_sentence_transformer_multi_dataset_example.pyScript9.9 KiB
scripts/train_sentence_transformer_static_embedding_example.pyScript10.3 KiB
scripts/train_sentence_transformer_with_lora_example.pyScript10.3 KiB
scripts/train_sparse_encoder_distillation_example.pyScript9.4 KiB
scripts/train_sparse_encoder_example.pyScript8.1 KiB

references/hf_jobs_execution.md

Hugging Face Jobs Execution

Run training on Hugging Face's managed GPUs without provisioning any local infrastructure. The same training script runs locally and on Jobs. This reference covers only the Jobs-specific concerns.

Prerequisites

Hugging Face account with a Pro, Team, or Enterprise plan. Jobs are paid.
HF_TOKEN with write permission. Log in once locally with hf auth login (the modern command from the hf CLI, replacing the deprecated huggingface-cli login).
Access to the hf_jobs() MCP tool, or the hf CLI (curl -LsSf https://hf.co/cli/install.sh | bash -s).

The three submission paths

1. Inline script via MCP (recommended in Claude Code)

Pass the full training script as script. Dependencies come from the PEP 723 header.

hf_jobs("uv", {
    "script": """
# /// script
# requires-python = ">=3.10"
# dependencies = ["sentence-transformers[train]>=5.0", "trackio"]
# ///

# <full training script content>
""",
    "flavor": "a10g-large",
    "timeout": "3h",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"},
})

2. Script-from-URL via MCP

Upload the script to the Hub (as a model or dataset repo file) or a Gist, then reference by URL:

hf_jobs("uv", {
    "script": "https://huggingface.co/USERNAME/scripts/resolve/main/train_bi_encoder.py",
    "flavor": "a10g-large",
    "timeout": "3h",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"},
})

Local file paths (./train.py, /path/to/train.py) do not work. Jobs run in isolated containers without access to your filesystem.

3. CLI

hf jobs uv run \
    --flavor a10g-large \
    --timeout 3h \
    --secrets HF_TOKEN \
    "https://huggingface.co/USERNAME/scripts/resolve/main/train.py"

Syntax gotchas:

Command order is hf jobs uv run, not hf jobs run uv.
Flags (--flavor, --timeout, --secrets) go before the script URL.
--secrets (plural), not --secret.

Required script modifications for Jobs

Add these to your TrainingArguments:

args = SentenceTransformerTrainingArguments(
    ...,
    push_to_hub=True,
    hub_model_id="your-username/my-model",
    hub_strategy="every_save",        # push each checkpoint; timeout-safe
    save_strategy="steps",
    save_steps=0.1,                   # 10 saves/pushes per epoch; scales with dataset size
)

Why each matters:

Argument	Why
`push_to_hub=True`	The Jobs container is destroyed after the job finishes. Without Hub push, all weights are lost.
`hub_model_id`	Required to identify the destination repo.
`hub_strategy="every_save"`	Default, but worth being deliberate about on Jobs: each checkpoint is pushed as it's written, so a timeout leaves all completed checkpoints on the Hub. `"end"` only pushes once `trainer.train()` returns, so a timeout loses everything.
`save_strategy="steps"` + `save_steps=0.1`	Checkpoints must actually be saved for `hub_strategy="every_save"` to push them. Fractional `0.1` = save every 10% of training, auto-scales with dataset size.

Secrets

Secrets are environment variables injected into the Jobs container. They never appear in logs and are not part of the script.

Secret	Required when
`HF_TOKEN`	Always, for Hub push. Also covers Trackio auth.
`WANDB_API_KEY`	Using `report_to="wandb"`.
`MLFLOW_TRACKING_URI`, `MLFLOW_TRACKING_TOKEN`	Using MLflow with a remote server.

The $HF_TOKEN syntax in the job config references the value from your local environment at submission time. The literal string $HF_TOKEN is replaced with your token's value. Never hardcode tokens in the script itself.

Trackio (the default tracker in this skill) uses HF_TOKEN for auth, so no extra secrets are needed. Only switch to the W&B / MLflow rows above if you're using those trackers.

Timeout

Default is 30 minutes, which is too short for almost any real training. Set explicitly:

"timeout": "2h"       # 2 hours
"timeout": "90m"      # 90 minutes
"timeout": "1.5h"     # 90 minutes
"timeout": 7200       # seconds, as integer

Rule: estimated training time × 1.3. The extra buffer covers model loading, dataset caching, checkpoint saving, and Hub push.

On timeout, the container is killed immediately. Only data on the Hub (hub_strategy="every_save" saves you here) or in persistent volumes survives.

Dataset caching

Hugging Face datasets are cached at ~/.cache/huggingface/datasets by default. That's inside the container, which is destroyed after the job. Each Jobs run re-downloads the dataset.

For large datasets (>5 GB), this matters. Options:

Persistent /data volume (Jobs feature, check current documentation): set HF_DATASETS_CACHE=/data/datasets so caches persist across jobs.
Pre-cache locally, push to Hub: if the dataset is on Hub already, nothing to do. If it's local-only, dataset.push_to_hub(...) once so subsequent jobs load from Hub.

Monitoring a running job

hf jobs ps [--all]                        # running (or all) jobs
hf jobs inspect <job-id>                  # full config + status
hf jobs logs <job-id> [--follow|--tail N] # tail or stream
hf jobs cancel <job-id>
hf jobs hardware                          # list flavors + hourly rates

hf jobs logs <id> --follow under Bash run_in_background pairs nicely with a Monitor watching for the VERDICT: line emitted by your training script's verdict block.

MCP equivalents (signatures may vary by server version, so check the actual tool listing): hf_jobs("ps"), hf_jobs("logs", {"job_id": ...}), hf_jobs("cancel", {"job_id": ...}).

For recurring runs, hf jobs scheduled uv run "<cron>" <script> ... schedules. hf jobs scheduled ps/suspend/delete manages.

Common failures

"Model not found on Hub" after a successful-looking run

The run succeeded but push_to_hub was not enabled. The container is gone. The weights are gone.

Fix: always set push_to_hub=True + hub_model_id=... + secrets={"HF_TOKEN": "$HF_TOKEN"}.

Tracker not connecting

Trackio: HF_TOKEN missing or lacks write permission. Add "secrets": {"HF_TOKEN": "$HF_TOKEN"} and make sure the token has write access.
W&B: WANDB_API_KEY missing. Add "secrets": {"HF_TOKEN": "$HF_TOKEN", "WANDB_API_KEY": "$WANDB_API_KEY"}.

OOM on first step

Flavor too small. Move up one tier (see hardware_guide.md).

Training starts but eval hangs forever

eval_strategy="steps" with no eval_dataset. Always provide an eval dataset, or set eval_strategy="no".

Dataset download times out

Large dataset or slow cold-cache. Increase timeout or pre-cache to a persistent volume.

`CachedMultipleNegativesRankingLoss` + `gradient_checkpointing=True` crash

The cached losses are incompatible with gradient checkpointing. Disable gradient_checkpointing.

After submission, the MCP returns a job ID. Monitor with hf_jobs("logs", {"job_id": ...}) when you want an update. Don't poll in a tight loop. End-to-end submission templates live in scripts/train_sentence_transformer_example.py / scripts/train_cross_encoder_example.py / scripts/train_sparse_encoder_example.py. Wrap the script contents in the inline pattern from §1 above.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

Instructions that use the network
Downloads piped into a shell
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

Hugging Face

Publisher

Hugging Face (Official)

Source

huggingface/skills · skills/train-sentence-transformers · commit 25910b3

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by Hugging Face.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download train-sentence-transformers.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by Hugging Face

Train a sentence-transformers Model

1. Identify the model type

2. Required reading

Per-type: always required

Cross-cutting: always required (regardless of task)

Cross-cutting: load when applicable

Variant scripts (open when the task matches)

3. Defaults

4. Constraints the produced script must satisfy

5. Workflow

Prerequisites

Files

references/hf_jobs_execution.md

Hugging Face Jobs Execution

Prerequisites

The three submission paths

1. Inline script via MCP (recommended in Claude Code)

2. Script-from-URL via MCP

3. CLI

Required script modifications for Jobs

Secrets

Timeout

Dataset caching

Monitoring a running job

Common failures

"Model not found on Hub" after a successful-looking run

Tracker not connecting

OOM on first step

Training starts but eval hangs forever

Dataset download times out

CachedMultipleNegativesRankingLoss + gradient_checkpointing=True crash

Requirements and compatibility

Security

Attribution and licence

Related skills

Pytorch Lightning

Aeon

Umap Learn

Scikit Learn

Shap

Pufferlib

Download ZIP

`CachedMultipleNegativesRankingLoss` + `gradient_checkpointing=True` crash