Mimir — Agent Skill by Grafana Labs

From the author

Stand up Grafana Mimir for horizontally scalable, multi-tenant, long-term Prometheus + OTLP metrics storage. Covers monolithic / read-write / microservices deployment, S3 / GCS / Azure / filesystem block storage, Prometheus `remote_write` and OTLP ingestion, multi-tenancy with `X-Scope-OrgID`, ingester replication factor, compactor retention, and per-tenant limits. Use when running Mimir locally or on Kubernetes (Helm `mimir-distributed`), scaling Prometheus past a single node, picking ingest / query / backend split, configuring tenants and ingestion rate, debugging `/ready` 503s or `429 Too Many Requests`, or pointing Grafana at a Mimir datasource — even when the user says "I need long-term Prometheus storage", "scale Prometheus", "multi-tenant metrics backend", "Cortex replacement", "remote_write target", or "store 10M active series" without naming Mimir.

— the description by Grafana Labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Grafana Mimir provides a horizontally scalable, multi-tenant architecture for long-term storage of Prometheus and OpenTelemetry metrics. This skill covers the deployment of Mimir in various modes—from monolithic setups for local development to distributed microservices on Kubernetes. It details configuration for object storage backends (S3, GCS, Azure), ingestion via remote_write, and management of multi-tenancy using X-Scope-OrgID headers. The documentation also includes architectural references for components like distributors, ingesters, and compactors, alongside guidance on troubleshooting common issues like 503 readiness errors and 429 rate-limiting.

What it does

Provides configuration templates for monolithic and microservices Mimir deployments.
Explains how to integrate Prometheus and Grafana Alloy via remote_write.
Details setup for object storage backends including S3, GCS, and Azure.
Outlines multi-tenancy configuration and per-tenant limit management.
Lists key architectural components and their roles in the read/write path.
Offers troubleshooting steps for common ingestion and readiness issues.

When to use it

When you need to scale Prometheus metrics storage beyond a single node.
When implementing a multi-tenant metrics backend for multiple teams or clients.
When setting up long-term storage for Prometheus or OTLP metrics.
When deploying Mimir on Kubernetes using the official Helm charts.
When debugging ingestion rate limits or readiness probe failures in a Mimir cluster.

Advantages

Enables horizontal scalability for massive metrics workloads.
Supports multi-tenancy with isolated data access per tenant.
Integrates natively with existing Prometheus and OTLP ecosystems.
Provides flexible deployment options from local dev to production microservices.

Usage examples

“How do I configure Prometheus to send metrics to Mimir?”: You must add a remote_write section to your Prometheus configuration pointing to the Mimir push endpoint, including the X-Scope-OrgID header if multi-tenancy is enabled.
“Why am I getting 429 Too Many Requests errors when pushing metrics?”: This indicates you have hit your configured ingestion limits; you should increase the limits.ingestion_rate and ingestion_burst_size in your Mimir configuration.
“How do I deploy Mimir on Kubernetes?”: Use the grafana/mimir-distributed Helm chart, providing a custom values.yaml file to configure your specific storage backend and component scaling requirements.
“What is the difference between the distributor and the ingester?”: The distributor validates and routes incoming samples via a hash ring, while the ingester handles in-memory storage, maintains the write-ahead log, and flushes blocks to object storage.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/grafana/mimir/download
Unzip it into the cross-client skills folder of your project:
```
mkdir -p .agents/skills && unzip mimir.zip -d .agents/skills
```

Documentation by Grafana Labs

Grafana Mimir

Docs: https://grafana.com/docs/mimir/latest/

Horizontally scalable, multi-tenant, long-term storage for Prometheus + OpenTelemetry metrics.

Prerequisites

Docker (for quick start) or a Kubernetes cluster (for Helm)
An object-storage bucket for production (S3/GCS/Azure) — filesystem only for dev
Prometheus or Alloy able to remote_write to the Mimir push endpoint

Common Workflows

1. Stand up monolithic Mimir locally

# demo.yaml
target: all
multitenancy_enabled: false

blocks_storage:
  backend: filesystem
  bucket_store:
    sync_dir: /tmp/mimir/tsdb-sync
  filesystem:
    dir: /tmp/mimir/data/tsdb
  tsdb:
    dir: /tmp/mimir/tsdb

compactor:
  data_dir: /tmp/mimir/compactor
  sharding_ring:
    kvstore: { store: memberlist }

distributor:
  ring:
    instance_addr: 127.0.0.1
    kvstore: { store: memberlist }

ingester:
  ring:
    instance_addr: 127.0.0.1
    kvstore: { store: memberlist }
    replication_factor: 1

server:
  http_listen_port: 9009
  grpc_listen_port: 9095
  log_level: error

# 1. Run it
docker run --rm -p 9009:9009 -v $(pwd)/demo.yaml:/etc/mimir/demo.yaml \
  grafana/mimir:latest --config.file=/etc/mimir/demo.yaml

# 2. Verify readiness (expect HTTP 200, body: "ready")
curl -sf http://localhost:9009/ready

# 3. Verify it's serving the API
curl -s http://localhost:9009/api/v1/labels | jq '.status'   # → "success"

# 4. Verify self-metrics scrape
curl -s http://localhost:9009/metrics | grep -E '^mimir_(distributor|ingester)_' | head

2. Send metrics — Prometheus remote_write

remote_write:
  - url: http://localhost:9009/api/v1/push
    headers:
      X-Scope-OrgID: tenant1   # required when multitenancy_enabled: true

# Verify samples are landing
curl -s -H 'X-Scope-OrgID: tenant1' \
  'http://localhost:9009/api/v1/query?query=up' | jq '.data.result | length'
# Expect > 0 within ~30s of Prometheus scraping

3. Send metrics — Grafana Alloy

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
    headers = { "X-Scope-OrgID" = "tenant1" }
  }
}

4. Deploy on Kubernetes (Helm, microservices)

# 1. Install
helm repo add grafana https://grafana.github.io/helm-charts
helm install mimir grafana/mimir-distributed --version 6.0.6 -f values.yaml

# 2. Verify every pod is Running / Ready
kubectl get pods -n mimir
#   distributor, ingester, querier, query-frontend, store-gateway, compactor, ruler

# 3. Verify the gateway is ready
kubectl port-forward -n mimir svc/mimir-nginx 9009:80 &
curl -sf http://localhost:9009/ready

Multi-tenancy

multitenancy_enabled: true
# Every request must include header:  X-Scope-OrgID: <tenant-id>

For storage backends (S3 / GCS / Azure / filesystem) see references/storage.md. For component roles, ring options, limits, and API endpoint dumps see references/architecture.md.

Troubleshooting

/ready returns 503 → ingester still joining ring; check mimir_ring_members and ingester logs
429 Too Many Requests on push → bump limits.ingestion_rate / ingestion_burst_size
Samples written but query returns empty → confirm X-Scope-OrgID matches between write and read
Query for old data returns nothing → check compactor logs and that store-gateway has synced blocks

Resources

Files

ATTRIBUTION.mdAdded by AutoGineer830 B
LICENSELicence file added by AutoGineer11.1 KiB
SKILL.md4.5 KiB
references/architecture.md2.0 KiB
references/storage.md665 B

references/architecture.md

Mimir architecture & API

Write path

Client → Distributor → Ingester (WAL) → Object Storage (blocks)

Read path

Client → Query-frontend (cache/sharding) → Query-scheduler → Querier ← Store-gateway

Components

Component	Role
Distributor	Validates & routes incoming samples via hash ring
Ingester	In-memory storage, WAL, flushes blocks to object store
Compactor	Merges blocks, enforces retention
Querier	Executes PromQL across ingesters + object store
Query-frontend	Request splitting, caching, parallelization
Store-gateway	Provides access to blocks in object storage
Ruler	Evaluates recording and alerting rules
Alertmanager	Routes and deduplicates alerts

HTTP API

Method	Endpoint	Description
POST	/api/v1/push	Prometheus remote write
POST	/otlp/v1/metrics	OTLP ingestion
GET	/api/v1/query	Instant PromQL query
GET	/api/v1/query_range	Range PromQL query
GET	/api/v1/labels	List labels
GET	/api/v1/series	List series
GET	/api/v1/rules	List rules
GET	/ready	Readiness probe
GET	/metrics	Self metrics

Deployment modes

# Monolithic — single binary, all targets
mimir -config.file=mimir.yaml -target=all

# Read-Write split
mimir -target=write    # distributor + ingester
mimir -target=read     # query-frontend + querier
mimir -target=backend  # store-gateway + compactor + ruler

# Microservices — each target as a separate deployment

Common limits

limits:
  ingestion_rate: 10000
  ingestion_burst_size: 200000
  max_global_series_per_user: 1500000
  max_label_names_per_series: 30
  compactor_blocks_retention_period: 30d
  ruler_max_rules_per_rule_group: 20

ingester:
  ring:
    replication_factor: 3

querier:
  max_concurrent: 20

frontend:
  max_outstanding_per_tenant: 100
  compress_responses: true

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs network access, Needs a command-line tool

Security

What our scanner noticed· Low risk

Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Spark Memory Thermal Ops
Seth Hobson· Community
Manage unified memory and thermal constraints for long-running machine learning jobs on NVIDIA DGX Spark.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskHas scriptsGitHub stars of the source repository42.3k
Observability And Instrumentation
Addy Osmani· Community
A systematic approach to instrumenting production code with structured logs, metrics, and traces for effective system observability.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository113.9k
Gke Alert Configuration
GoogleOfficial
Configure GKE alerting policies in Terraform using PromQL and Google Cloud Managed Service for Prometheus.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
Apache-2.0Low riskHas scriptsGitHub stars of the source repository23.2k
Application Performance Observability Engineer
Seth Hobson· Community
Expert observability engineer for designing production-grade monitoring, logging, tracing, and reliability systems.
Cloud and DevOpsClaude Code
MITGitHub stars of the source repository42.3k
Distributed Debugging Devops Troubleshooter
Seth Hobson· Community
An expert DevOps troubleshooter for incident response, system reliability, and complex distributed system debugging.
Cloud and DevOpsClaude Code
MITGitHub stars of the source repository42.3k
Distributed Tracing
Seth Hobson· Community
Implement distributed tracing using Jaeger and Tempo to monitor request flows and performance across microservices.
Cloud and DevOpsClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository42.3k

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download mimir.zip