Slo Implementation

From the author

Define and implement Service Level Indicators (SLIs) and Service Level Objectives (SLOs) with error budgets and alerting. Use when establishing reliability targets, implementing SRE practices, or measuring service performance.

— the description by Seth Hobson

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to implementing Site Reliability Engineering (SRE) practices by defining Service Level Indicators (SLIs) and Service Level Objectives (SLOs). It covers the hierarchy of reliability targets, error budget calculations, and policy enforcement. The guide includes practical examples for Prometheus recording rules, alerting configurations for burn rates, and guidance on establishing a review process to balance system reliability with development velocity.

What it does

Defines the hierarchy between SLAs, SLOs, and SLIs.
Provides formulas and PromQL examples for measuring availability, latency, and durability.
Outlines error budget calculation and policy-based development workflows.
Supplies configuration templates for Prometheus recording and alerting rules.
Suggests structural layouts for SLO dashboards in Grafana.
Establishes a cadence for weekly, monthly, and quarterly reliability reviews.

When to use it

When establishing reliability targets for production services.
When implementing SRE practices to measure user-perceived reliability.
When creating automated alerts based on error budget consumption.
When needing to balance feature development speed with system stability.
When performing regular reviews of service performance and SLO relevance.

Advantages

Reduces alert noise through multi-window burn rate detection.
Provides a clear, data-driven basis for prioritizing reliability work.
Aligns technical performance metrics with business expectations.
Offers ready-to-use templates for common monitoring stacks.
Encourages a consistent, repeatable process for reliability management.

Usage examples

“How do I calculate the error budget for a 99.9% availability SLO?”: The skill provides the formula (1 - SLO Target) and explains that for 99.9%, the budget is 0.1%, which equates to 43.2 minutes of downtime per month.
“What is the recommended way to alert on SLO violations?”: The skill suggests using multi-window burn rate alerts to detect fast and slow burns, providing specific PromQL examples for Prometheus alert rules.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/wshobson/slo-implementation/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip slo-implementation.zip -d .agents/skills

Documentation by Seth Hobson

Framework for defining and implementing Service Level Indicators (SLIs), Service Level Objectives (SLOs), and error budgets.

Purpose

Implement measurable reliability targets using SLIs, SLOs, and error budgets to balance reliability with innovation velocity.

When to Use

Define service reliability targets
Measure user-perceived reliability
Implement error budgets
Create SLO-based alerts
Track reliability goals

SLI/SLO/SLA Hierarchy

SLA (Service Level Agreement)
  ↓ Contract with customers
SLO (Service Level Objective)
  ↓ Internal reliability target
SLI (Service Level Indicator)
  ↓ Actual measurement

Defining SLIs

Common SLI Types

1. Availability SLI

# Successful requests / Total requests
sum(rate(http_requests_total{status!~"5.."}[28d]))
/
sum(rate(http_requests_total[28d]))

2. Latency SLI

# Requests below latency threshold / Total requests
sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d]))
/
sum(rate(http_request_duration_seconds_count[28d]))

3. Durability SLI

# Successful writes / Total writes
sum(storage_writes_successful_total)
/
sum(storage_writes_total)

Setting SLO Targets

Availability SLO Examples

SLO %	Downtime/Month	Downtime/Year
99%	7.2 hours	3.65 days
99.9%	43.2 minutes	8.76 hours
99.95%	21.6 minutes	4.38 hours
99.99%	4.32 minutes	52.56 minutes

Choose Appropriate SLOs

Consider:

User expectations
Business requirements
Current performance
Cost of reliability
Competitor benchmarks

Example SLOs:

slos:
  - name: api_availability
    target: 99.9
    window: 28d
    sli: |
      sum(rate(http_requests_total{status!~"5.."}[28d]))
      /
      sum(rate(http_requests_total[28d]))

  - name: api_latency_p95
    target: 99
    window: 28d
    sli: |
      sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d]))
      /
      sum(rate(http_request_duration_seconds_count[28d]))

Error Budget Calculation

Error Budget Formula

Error Budget = 1 - SLO Target

Example:

SLO: 99.9% availability
Error Budget: 0.1% = 43.2 minutes/month
Current Error: 0.05% = 21.6 minutes/month
Remaining Budget: 50%

Error Budget Policy

error_budget_policy:
  - remaining_budget: 100%
    action: Normal development velocity
  - remaining_budget: 50%
    action: Consider postponing risky changes
  - remaining_budget: 10%
    action: Freeze non-critical changes
  - remaining_budget: 0%
    action: Feature freeze, focus on reliability

SLO Implementation

Prometheus Recording Rules

# SLI Recording Rules
groups:
  - name: sli_rules
    interval: 30s
    rules:
      # Availability SLI
      - record: sli:http_availability:ratio
        expr: |
          sum(rate(http_requests_total{status!~"5.."}[28d]))
          /
          sum(rate(http_requests_total[28d]))

      # Latency SLI (requests < 500ms)
      - record: sli:http_latency:ratio
        expr: |
          sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d]))
          /
          sum(rate(http_request_duration_seconds_count[28d]))

  - name: slo_rules
    interval: 5m
    rules:
      # SLO compliance (1 = meeting SLO, 0 = violating)
      - record: slo:http_availability:compliance
        expr: sli:http_availability:ratio >= bool 0.999

      - record: slo:http_latency:compliance
        expr: sli:http_latency:ratio >= bool 0.99

      # Error budget remaining (percentage)
      - record: slo:http_availability:error_budget_remaining
        expr: |
          (sli:http_availability:ratio - 0.999) / (1 - 0.999) * 100

      # Error budget burn rate
      - record: slo:http_availability:burn_rate_5m
        expr: |
          (1 - (
            sum(rate(http_requests_total{status!~"5.."}[5m]))
            /
            sum(rate(http_requests_total[5m]))
          )) / (1 - 0.999)

SLO Alerting Rules

groups:
  - name: slo_alerts
    interval: 1m
    rules:
      # Fast burn: 14.4x rate, 1 hour window
      # Consumes 2% error budget in 1 hour
      - alert: SLOErrorBudgetBurnFast
        expr: |
          slo:http_availability:burn_rate_1h > 14.4
          and
          slo:http_availability:burn_rate_5m > 14.4
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Fast error budget burn detected"
          description: "Error budget burning at {{ $value }}x rate"

      # Slow burn: 6x rate, 6 hour window
      # Consumes 5% error budget in 6 hours
      - alert: SLOErrorBudgetBurnSlow
        expr: |
          slo:http_availability:burn_rate_6h > 6
          and
          slo:http_availability:burn_rate_30m > 6
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "Slow error budget burn detected"
          description: "Error budget burning at {{ $value }}x rate"

      # Error budget exhausted
      - alert: SLOErrorBudgetExhausted
        expr: slo:http_availability:error_budget_remaining < 0
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "SLO error budget exhausted"
          description: "Error budget remaining: {{ $value }}%"

SLO Dashboard

Grafana Dashboard Structure:

┌────────────────────────────────────┐
│ SLO Compliance (Current)           │
│ ✓ 99.95% (Target: 99.9%)          │
├────────────────────────────────────┤
│ Error Budget Remaining: 65%        │
│ ████████░░ 65%                     │
├────────────────────────────────────┤
│ SLI Trend (28 days)                │
│ [Time series graph]                │
├────────────────────────────────────┤
│ Burn Rate Analysis                 │
│ [Burn rate by time window]         │
└────────────────────────────────────┘

Example Queries:

# Current SLO compliance
sli:http_availability:ratio * 100

# Error budget remaining
slo:http_availability:error_budget_remaining

# Days until error budget exhausted (at current burn rate)
(slo:http_availability:error_budget_remaining / 100)
*
28
/
(1 - sli:http_availability:ratio) * (1 - 0.999)

Additional patterns and templates

More detailed templates and worked examples live in references/details.md. Read that file for the full pattern library.

Files

ATTRIBUTION.mdAdded by AutoGineer1002 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md7.0 KiB
references/details.md1.3 KiB

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Spark Memory Thermal Ops
Seth Hobson· Community
Manage unified memory and thermal constraints for long-running machine learning jobs on NVIDIA DGX Spark.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskHas scriptsGitHub stars of the source repository42.3k
Observability And Instrumentation
Addy Osmani· Community
A systematic approach to instrumenting production code with structured logs, metrics, and traces for effective system observability.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository113.9k
Gke Alert Configuration
GoogleOfficial
Configure GKE alerting policies in Terraform using PromQL and Google Cloud Managed Service for Prometheus.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
Apache-2.0Low riskHas scriptsGitHub stars of the source repository23.2k
Application Performance Observability Engineer
Seth Hobson· Community
Expert observability engineer for designing production-grade monitoring, logging, tracing, and reliability systems.
Cloud and DevOpsClaude Code
MITGitHub stars of the source repository42.3k
Distributed Debugging Devops Troubleshooter
Seth Hobson· Community
An expert DevOps troubleshooter for incident response, system reliability, and complex distributed system debugging.
Cloud and DevOpsClaude Code
MITGitHub stars of the source repository42.3k
Distributed Tracing
Seth Hobson· Community
Implement distributed tracing using Jaeger and Tempo to monitor request flows and performance across microservices.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository42.3k

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download slo-implementation.zip