AutoGineer
Generated banner for Python Observability
Generated logo for Python Observability

Python Observability

Seth HobsonCommunityCreated by Seth Hobson

  • Skill
  • Guidelines
  • Software development
  • MIT
  • No issues found
Download ZIP

From the author

Python observability patterns including structured logging, metrics, and distributed tracing. Use when adding logging, implementing metrics collection, setting up tracing, or debugging production systems.
— the description by Seth Hobson

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a comprehensive framework for instrumenting Python applications to ensure production observability. It covers the implementation of structured JSON logging, the tracking of the Four Golden Signals (latency, traffic, errors, and saturation) using Prometheus, and the propagation of correlation IDs to maintain request context across distributed services. The guidelines emphasize best practices such as maintaining bounded cardinality in metrics and using appropriate log levels to ensure that production systems remain debuggable and performant.

What it does

  • Configures structured JSON logging for machine-readable production logs.
  • Implements the Four Golden Signals for monitoring service health.
  • Provides patterns for propagating correlation IDs across request chains.
  • Defines semantic log levels to distinguish between diagnostic and critical events.
  • Offers reusable context managers for timing and logging operations.
  • Establishes best practices for metric label management to prevent cardinality explosion.

When to use it

  • When you need to improve the debuggability of production Python services.
  • When implementing or standardizing logging across a microservices architecture.
  • When setting up Prometheus metrics for service monitoring.
  • When you need to trace requests across multiple service boundaries.
  • When building observability dashboards for application performance.

Advantages

  • Enables efficient querying and alerting through structured, machine-readable logs.
  • Ensures consistent observability patterns across different application components.
  • Helps prevent common pitfalls like metric cardinality explosion.
  • Facilitates faster incident response by providing clear request context.
  • Promotes clean code by separating observability logic from business logic.

Usage examples

“How do I configure structured JSON logging in my Python application?”
The skill provides a configuration pattern using structlog that formats logs as JSON with timestamps and consistent fields, suitable for production ingestion.
“How can I track the latency of my API endpoints using Prometheus?”
The skill demonstrates how to use a Histogram metric and a decorator to automatically track request duration, status codes, and methods.
“How do I ensure a request ID is passed to downstream services?”
The skill provides a middleware pattern using contextvars to generate or extract a correlation ID and inject it into outbound HTTP headers.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/wshobson/python-observability/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip python-observability.zip -d .agents/skills

Documentation by Seth Hobson

Python Observability

Instrument Python applications with structured logs, metrics, and traces. When something breaks in production, you need to answer "what, where, and why" without deploying new code.

When to Use This Skill

  • Adding structured logging to applications

  • Implementing metrics collection with Prometheus

  • Setting up distributed tracing across services

  • Propagating correlation IDs through request chains

  • Debugging production issues

  • Building observability dashboards

Core Concepts

1. Structured Logging

Emit logs as JSON with consistent fields for production environments. Machine-readable logs enable powerful queries and alerts. For local development, consider human-readable formats.

2. The Four Golden Signals

Track latency, traffic, errors, and saturation for every service boundary.

3. Correlation IDs

Thread a unique ID through all logs and spans for a single request, enabling end-to-end tracing.

4. Bounded Cardinality

Keep metric label values bounded. Unbounded labels (like user IDs) explode storage costs.

Quick Start

import structlog

structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
)

logger = structlog.get_logger()
logger.info("Request processed", user_id="123", duration_ms=45)

Fundamental Patterns

Pattern 1: Structured Logging with Structlog

Configure structlog for JSON output with consistent fields.

import logging
import structlog

def configure_logging(log_level: str = "INFO") -> None:
    """Configure structured logging for the application."""
    structlog.configure(
        processors=[
            structlog.contextvars.merge_contextvars,
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(fmt="iso"),
            structlog.processors.StackInfoRenderer(),
            structlog.processors.format_exc_info,
            structlog.processors.JSONRenderer(),
        ],
        wrapper_class=structlog.make_filtering_bound_logger(
            getattr(logging, log_level.upper())
        ),
        context_class=dict,
        logger_factory=structlog.PrintLoggerFactory(),
        cache_logger_on_first_use=True,
    )

# Initialize at application startup
configure_logging("INFO")
logger = structlog.get_logger()
Pattern 2: Consistent Log Fields

Every log entry should include standard fields for filtering and correlation.

import structlog
from contextvars import ContextVar

# Store correlation ID in context
correlation_id: ContextVar[str] = ContextVar("correlation_id", default="")

logger = structlog.get_logger()

def process_request(request: Request) -> Response:
    """Process request with structured logging."""
    logger.info(
        "Request received",
        correlation_id=correlation_id.get(),
        method=request.method,
        path=request.path,
        user_id=request.user_id,
    )

    try:
        result = handle_request(request)
        logger.info(
            "Request completed",
            correlation_id=correlation_id.get(),
            status_code=200,
            duration_ms=elapsed,
        )
        return result
    except Exception as e:
        logger.error(
            "Request failed",
            correlation_id=correlation_id.get(),
            error_type=type(e).__name__,
            error_message=str(e),
        )
        raise
Pattern 3: Semantic Log Levels

Use log levels consistently across the application.

LevelPurposeExamples
DEBUGDevelopment diagnosticsVariable values, internal state
INFORequest lifecycle, operationsRequest start/end, job completion
WARNINGRecoverable anomaliesRetry attempts, fallback used
ERRORFailures needing attentionExceptions, service unavailable
# DEBUG: Detailed internal information
logger.debug("Cache lookup", key=cache_key, hit=cache_hit)

# INFO: Normal operational events
logger.info("Order created", order_id=order.id, total=order.total)

# WARNING: Abnormal but handled situations
logger.warning(
    "Rate limit approaching",
    current_rate=950,
    limit=1000,
    reset_seconds=30,
)

# ERROR: Failures requiring investigation
logger.error(
    "Payment processing failed",
    order_id=order.id,
    error=str(e),
    payment_provider="stripe",
)

Never log expected behavior at ERROR. A user entering a wrong password is INFO, not ERROR.

Pattern 4: Correlation ID Propagation

Generate a unique ID at ingress and thread it through all operations.

from contextvars import ContextVar
import uuid
import structlog

correlation_id: ContextVar[str] = ContextVar("correlation_id", default="")

def set_correlation_id(cid: str | None = None) -> str:
    """Set correlation ID for current context."""
    cid = cid or str(uuid.uuid4())
    correlation_id.set(cid)
    structlog.contextvars.bind_contextvars(correlation_id=cid)
    return cid

# FastAPI middleware example
from fastapi import Request

async def correlation_middleware(request: Request, call_next):
    """Middleware to set and propagate correlation ID."""
    # Use incoming header or generate new
    cid = request.headers.get("X-Correlation-ID") or str(uuid.uuid4())
    set_correlation_id(cid)

    response = await call_next(request)
    response.headers["X-Correlation-ID"] = cid
    return response

Propagate to outbound requests:

import httpx

async def call_downstream_service(endpoint: str, data: dict) -> dict:
    """Call downstream service with correlation ID."""
    async with httpx.AsyncClient() as client:
        response = await client.post(
            endpoint,
            json=data,
            headers={"X-Correlation-ID": correlation_id.get()},
        )
        return response.json()

Detailed worked examples and patterns

Detailed sections (starting with ## Advanced Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices Summary

  1. Use structured logging - JSON logs with consistent fields

  2. Propagate correlation IDs - Thread through all requests and logs

  3. Track the four golden signals - Latency, traffic, errors, saturation

  4. Bound label cardinality - Never use unbounded values as metric labels

  5. Log at appropriate levels - Don't cry wolf with ERROR

  6. Include context - User ID, request ID, operation name in logs

  7. Use context managers - Consistent timing and error handling

  8. Separate concerns - Observability code shouldn't pollute business logic

  9. Test your observability - Verify logs and metrics in integration tests

  10. Set up alerts - Metrics are useless without alerting

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Seth Hobson
Publisher
Seth Hobson (Community)
Source
wshobson/agents · plugins/python-development/skills/python-observability · commit 4bde316
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Seth Hobson.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download python-observability.zip