AutoGineer
Generated banner for Finetuning Technique
Generated logo for Finetuning Technique

Finetuning Technique

AWS LabsOfficialPublished by AWS Labs

  • Skill
  • Workflow
  • Data and AI
  • Apache-2.0
  • Low risk
  • Has scripts
Download ZIP

From the author

Selects a fine-tuning technique (SFT, DPO, RLVR, or RLAIF) for the user's use case and validates it against the selected model's available recipes. Use when the user has decided to finetune and needs to choose a technique, or when the technique needs to be validated against a model. Requires a base model to already be selected (via model-selection skill).
— the description by AWS Labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill assists users in choosing the most appropriate fine-tuning technique—SFT, DPO, RLVR, or RLAIF—based on their specific use case requirements. It ensures technical feasibility by validating the selected technique against the capabilities of a previously chosen base model, preventing incompatible configurations.

What it does

  • Recommends fine-tuning techniques based on task requirements and data characteristics.
  • Validates chosen techniques against the specific recipes supported by the selected base model.
  • Provides clear distinctions between SFT, DPO, RLVR, and RLAIF methodologies.
  • Offers guidance on when to pivot to a different base model if the desired technique is unsupported.
  • Summarizes the final configuration of the base model and chosen fine-tuning approach.

When to use it

  • When you have decided to fine-tune a model but are unsure which technique best fits your task.
  • When you need to verify if a specific fine-tuning method is compatible with your chosen base model.
  • When you need to switch your fine-tuning approach and require validation for the new method.

Advantages

  • Ensures alignment between the chosen fine-tuning method and the model's architectural capabilities.
  • Reduces trial-and-error by providing a structured selection guide for different AI training paradigms.
  • Prevents deployment failures by checking recipe availability before starting the training process.

Usage examples

“I have a base model selected and need to choose a fine-tuning technique for a code generation task.”
The agent will recommend RLVR due to the programmatic verifiability of code and then validate if the selected model supports RLVR recipes.
“Can I use DPO for my summarization task with this model?”
The agent will check the model's available recipes to confirm if DPO is supported and provide guidance on whether DPO is the right fit compared to RLAIF.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/awslabs/finetuning-technique/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip finetuning-technique.zip -d .agents/skills

Documentation by AWS Labs

Finetuning Technique

Guides the user through selecting a fine-tuning technique based on their use case and validates compatibility with the selected model.

When to Use

  • User has decided to finetune and needs to choose a technique

  • User wants to change their finetuning technique

  • Technique needs to be validated against a selected model

Prerequisites

  • A base model has been selected (via model-selection skill). The model name and hub must be known.

  • A use_case_spec.md file exists. If not, activate the use-case-specification skill to generate it first.

Workflow

Step 1: Determine Finetuning Technique

Consult references/finetune_technique_selection_guide.md to recommend the best-fit technique based on the use case and the user's needs (SFT, DPO, RLVR, RLAIF).

Present the recommendation and reasoning to the user. Ask if they'd like to go with the recommendation or prefer a different technique.

Step 2: Validate Technique Availability

  1. Once the user confirms a technique, retrieve the finetuning techniques available for the selected model by running: python finetuning-technique/scripts/get_recipes.py <model-name> <hub-name>

    • This returns only the techniques the model actually supports, filtered to SFT, DPO, RLVR, and RLAIF. Only these four techniques are supported — ignore any other techniques even if the model's recipes include them.

  2. If the chosen technique is available for the model, proceed to Step 3.

  3. If the chosen technique is not available for the model, explain that the selected model does not support it on SageMaker and offer to go back to model-selection to pick a different model that supports the chosen technique.

Step 3: Confirm Selections

Present a summary to the user:

Here's what we've selected:
- Base model: [model name]
- Fine-tuning technique: [SFT/DPO/RLVR/RLAIF]

References

  • references/finetune_technique_selection_guide.md — Technique guidance (SFT/DPO/RLVR/RLAIF)

Files

references/finetune_technique_selection_guide.md

Finetuning Technique Selection Guide

Not all models support all techniques. Always validate technique availability against the selected model's recipes before recommending. Only SFT, DPO, RLVR, and RLAIF are supported.

Technique Overview

SFT (Supervised Fine-Tuning)

Use when:

  • Task has clear right/wrong answers

  • Single optimal output per input

  • Output represents exemplary responses

  • Classification, extraction, structured generation

DPO (Direct Preference Optimization)

Use when:

  • Multiple valid outputs, some better than others

  • Subjective quality (tone, style, helpfulness)

  • Creative tasks with preference judgments

RLVR (Reinforcement Learning from Verifiable Rewards)

Use when:

  • Outputs can be verified programmatically

  • Want to reward similarity to gold responses

  • Code generation (passes tests = reward)

  • Math problems (correct answer = reward)

  • Constraint satisfaction (meets criteria = reward)

Key difference from SFT:

  • SFT: Model learns to imitate gold responses directly

  • RLVR: Model learns to maximize rewards (can be gold similarity or verification-based)

RLAIF (Reinforcement Learning from AI Feedback)

Use when:

  • Quality is subjective and hard to define with rules (tone, helpfulness, brand voice, safety)

  • No human preference data is available and collecting it is too expensive or slow

  • You want RLHF-level alignment without human annotators

  • Task involves summarization, dialogue, or open-ended generation where "better" is a judgment call

  • You need scalable preference signals that can be regenerated as the model improves

Key difference from DPO:

  • DPO: Requires a static dataset of preference pairs (chosen/rejected) upfront

  • RLAIF: Uses an AI judge model to generate preference signals or reward scores dynamically, enabling iterative improvement

Key difference from RLVR:

  • RLVR: Reward is rule-based and programmatic (correct/incorrect, passes tests)

  • RLAIF: Reward comes from an AI model evaluating subjective quality (helpfulness, coherence, safety)

When NOT to use RLAIF:

  • Task has objectively verifiable answers → use RLVR instead

  • You already have high-quality human preference data → use DPO instead

  • You have clear gold-standard outputs → use SFT instead

  • The AI judge model is weaker than the model being trained (judge quality bounds training quality)

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Low risk

  • Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
AWS Labs
Publisher
AWS Labs (Official)
Source
awslabs/agent-plugins · plugins/sagemaker-ai/skills/finetuning-technique · commit fbefb37
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • NOTICE (added by AutoGineer from NOTICE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by AWS Labs.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download finetuning-technique.zip