AutoGineer
Generated banner for Prometheus
Generated logo for Prometheus

Prometheus

Grafana LabsOfficialPublished by Grafana Labs

  • Skill
  • Tool integration
  • Cloud and DevOps
  • Apache-2.0
  • Low risk
Download ZIP

From the author

Prometheus and Grafana Cloud Metrics overview including PromQL query language, Metrics Drilldown, alerting, recording rules, and integration patterns. Use when working with Prometheus, writing PromQL queries, configuring alerting, or discussing metrics architecture and best practices.
— the description by Grafana Labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured reference for working with Prometheus and Grafana Cloud Metrics. It covers the essential components of the Prometheus ecosystem, including PromQL syntax for data manipulation, the configuration of alerting rules and routing via Alertmanager, and the implementation of recording rules to optimize dashboard performance. Additionally, it introduces the Metrics Drilldown feature for queryless exploration of telemetry data.

What it does

  • Provides syntax examples for PromQL instant and range vector selectors
  • Demonstrates common aggregation patterns for rates, histograms, and saturation
  • Outlines the structure for Prometheus alerting rules and Alertmanager routing
  • Explains how to create and deploy recording rules to pre-compute expensive queries
  • Details validation procedures for alerting and recording configurations
  • Introduces queryless metrics exploration using the Grafana Metrics Drilldown app

When to use it

  • When writing or debugging complex PromQL queries for dashboards
  • When setting up alerting rules to monitor system health and error rates
  • When configuring Alertmanager to route notifications to different channels
  • When optimizing dashboard performance using recording rules
  • When exploring metrics without needing to write manual queries

Advantages

  • Centralizes essential PromQL patterns and configuration templates
  • Includes practical examples for both alerting and recording rules
  • Provides clear validation steps to ensure configuration correctness
  • Covers modern Grafana integration features for improved observability

Usage examples

“How do I calculate the error rate percentage for my API?”
The skill provides a PromQL formula that divides the sum of 5xx status rates by the total request rate, multiplied by 100.
“How can I validate my Alertmanager configuration?”
The skill lists specific commands using amtool to check configuration syntax and test routing rules.
“How do I improve the performance of a slow dashboard query?”
The skill explains how to define a recording rule to pre-compute the expensive PromQL expression and provides steps to deploy it.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/grafana/prometheus/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip prometheus.zip -d .agents/skills

Documentation by Grafana Labs

Metrics with Prometheus and Grafana

Docs: https://prometheus.io/docs/ | Grafana Cloud Metrics: https://grafana.com/docs/grafana-cloud/send-data/metrics/

PromQL Quick Reference

Instant Vector Selectors
# By metric name
http_requests_total

# Label filter
http_requests_total{job="api-server"}

# Multiple labels (AND)
http_requests_total{job="api-server", method="GET"}

# Regex
http_requests_total{job=~"api.*", status=~"5.."}

# Negative
http_requests_total{status!="200"}
Range Vectors & Rates
# Per-second rate over 5 minutes
rate(http_requests_total[5m])

# Increase over interval
increase(http_requests_total[1h])

# Instant rate (last two samples)
irate(http_requests_total[5m])

# Offset (5 minutes ago)
rate(http_requests_total[5m] offset 5m)
Aggregations
# Sum by label
sum by (job) (rate(http_requests_total[5m]))

# Average
avg by (instance) (node_cpu_seconds_total)

# Top-K
topk(5, rate(http_requests_total[5m]))

# Histogram quantiles
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))

# Count distinct
count(up{job="api"})
Common Patterns
# Error rate percentage
sum(rate(http_requests_total{status=~"5.."}[5m]))
  / sum(rate(http_requests_total[5m])) * 100

# Saturation (CPU usage %)
100 - (avg by(instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# Memory usage
node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes

# Predict disk full (linear extrapolation)
predict_linear(node_filesystem_free_bytes[6h], 24*3600) < 0

Alerting Rules

Prometheus Alerting Rule
groups:
  - name: api_alerts
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            / sum(rate(http_requests_total[5m])) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High 5xx error rate ({{ $value | humanizePercentage }})"
Alertmanager Routing
# alertmanager.yml
route:
  receiver: default
  group_by: [alertname, job]
  group_wait: 30s
  group_interval: 5m
  routes:
    - match:
        severity: critical
      receiver: pagerduty
    - match:
        severity: warning
      receiver: slack

receivers:
  - name: pagerduty
    pagerduty_configs:
      - service_key: "<key>"
  - name: slack
    slack_configs:
      - channel: "#alerts"
        api_url: "<webhook_url>"
  - name: default
    email_configs:
      - to: "[email protected]"
Validate Alerting Configuration
promtool check rules rules.yml
amtool check-config alertmanager.yml
amtool config routes test --config.file=alertmanager.yml severity=critical

Recording Rules

Pre-compute expensive PromQL for dashboard performance:

groups:
  - name: api_rules
    interval: 1m
    rules:
      - record: job:http_requests:rate5m
        expr: sum by (job) (rate(http_requests_total[5m]))
      - record: job:http_request_duration_seconds:p99
        expr: histogram_quantile(0.99, sum by (job, le) (rate(http_request_duration_seconds_bucket[5m])))
Deploy and Verify Recording Rules
# 1. Validate rule syntax
promtool check rules rules/recording.yml

# 2. Reload Prometheus (after adding to rule_files in prometheus.yml)
curl -X POST http://localhost:9090/-/reload

# 3. Verify rules are active
curl -s http://localhost:9090/api/v1/rules | jq '.data.groups[].rules[] | {name, health}'

Metrics Drilldown (Grafana 12+)

Queryless Prometheus exploration — browse metrics without writing PromQL. Navigate to Explore > Metrics Drilldown or use <grafana-url>/a/grafana-metricsdrilldown-app. Provides metric search with label breakdown, smart segmentation for anomaly detection, auto-visualization, and telemetry pivoting from metrics to related logs and traces.

Resources

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs network access

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Grafana Labs
Publisher
Grafana Labs (Official)
Source
grafana/skills · skills/grafana-lgtm/prometheus · commit 8fb72a5
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Grafana Labs.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download prometheus.zip