AutoGineer
Generated banner for Cloud Run Alert Configuration
Generated logo for Cloud Run Alert Configuration

Cloud Run Alert Configuration

GoogleOfficialPublished by Google

  • Skill
  • Workflow
  • Cloud and DevOps
  • Apache-2.0
  • No issues found
Download ZIP

From the author

Configures best-practice, high-signal alerting policies for Cloud Run resources on Google Cloud (services, jobs, and worker pools) based on seasoned SRE practices. Use when analyzing, recommending, writing, or deploying Terraform PromQL alerting policies to monitor Cloud Run error rates (4xx/5xx), request latency, container instance saturation (warning/critical), container CPU/memory utilization and allocation, billable instance time, job execution status, and worker pool queue backlog. Don't use for GKE workloads (use gke-alert-configuration) or Compute Engine VMs.
— the description by Google

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a standardized approach to configuring high-signal alerting policies for Google Cloud Run resources. By leveraging Terraform and PromQL, it enables developers and SREs to implement observability based on seasoned SRE practices. It covers HTTP services, batch jobs, and worker pools, ensuring that critical metrics like error rates, latency, and saturation are monitored effectively. The skill emphasizes parameterization, mandatory labeling for governance, and adherence to specific lookback and duration rules for Cloud Monitoring.

What it does

  • Generates Terraform HCL code for Google Cloud Monitoring alert policies.
  • Applies SRE-based best practices for monitoring Cloud Run services, jobs, and worker pools.
  • Configures PromQL queries for error rates, latency, saturation, and execution status.
  • Enforces mandatory user labels for alert policy governance.
  • Provides a fast-path workflow for users who specify resource names in their prompt.
  • Includes logic for handling PromQL duration rules based on lookback windows.

When to use it

  • When you need to set up production-grade alerting for Cloud Run resources.
  • When you want to define monitoring policies as code using Terraform.
  • When you need to implement specific SRE alerting patterns for HTTP services or batch jobs.
  • When you require consistent alerting configurations across multiple Cloud Run environments.

Advantages

  • Reduces configuration errors by using standardized, SRE-vetted templates.
  • Ensures consistent observability across different Cloud Run workload types.
  • Promotes infrastructure-as-code practices for monitoring policies.
  • Simplifies the creation of complex PromQL queries for Cloud Monitoring.
  • Supports centralized metrics scoping via parameterized project variables.

Usage examples

“Generate a Terraform alert policy for my Cloud Run service named 'api-gateway' to monitor 5xx error rates.”
The agent generates a complete Terraform resource block for the 'api-gateway' service, including the PromQL query for 5xx errors, mandatory labels, and alert strategy settings.
“Create an alert policy for a Cloud Run job called 'nightly-sync' that pages immediately on failure.”
The agent provides a Terraform configuration using a 0s duration PromQL query to trigger an immediate alert upon any failed execution of the 'nightly-sync' job.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/google/cloud-run-alert-configuration/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip cloud-run-alert-configuration.zip -d .agents/skills

Documentation by Google

Cloud Run Alert Configuration

Production-grade observability for Cloud Run using Terraform and PromQL (Cloud Monitoring). Grounded in SRE practices, this skill focuses strictly on actionable user impact and scaling bounds.

Attribution

MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:cloud-run-alert-configuration)" and append agent-skills/1.0 (skill:cloud-run-alert-configuration) to User-Agent headers.

CRITICAL RULES

  • Prompt-First Fast Path (Skip Discovery When Named):

    • If the user prompt explicitly specifies the target Cloud Run service, job, or worker pool name (e.g., 'video-encoder', 'nightly-reconciliation', 'web-frontend', 'catalog-service', 'api-gateway', 'order-processor'), SKIP all workspace .tf file scanning (find_by_name, code_search, list_dir) and gcloud CLI discovery commands entirely.

    • Do NOT run gcloud, terraform, or file search tools when the target name is already provided in the prompt. Instead, parameterize the project ID (variable "scoping_project_id" { default = "my-gcp-project" }) and target resource name in Terraform variables and proceed immediately to Step 2 (Configure Alerts).

  • Autonomous Discovery (Only When Target Name is Omitted):

    • Never Scan Root Monorepo or Unbounded Directories: Never run find_by_name or ls across root workspace directories.

    • Config First: Only if the prompt omits the resource name, check .tf files in the immediate working directory for google_cloud_run_v2_service, google_cloud_run_service, or google_cloud_run_v2_job.

    • CLI Second (Graceful Fallback): Only if unconfigured in prompt or local .tf files, attempt gcloud config get-value project and gcloud run services list. If any gcloud command fails (e.g., auth or metadata errors) or terraform is missing, immediately stop running CLI commands and output parameterized HCL using explicit variable defaults.

  • Workload Routing: Always classify the workload target and follow its specific reference guide:

  • Explicit Defaults & User Overrides:

    • Always use explicit defaults for all constants specified in the target workload's reference file (SLO targets, latency thresholds, SLAs, saturation ceilings, rate guards).

    • State the defaults being applied in the final summary output and clearly notify the user that any default constant can be customized or overridden via Terraform variables or prompt input.

  • Metric Scope Centralization: Parameterize project = var.scoping_project_id in all Terraform google_monitoring_alert_policy resources so the policy can target either a single project or a centralized Cloud Monitoring Metrics Scope.

  • PromQL duration (Retest Window) Rules:

    • Lookbacks $\le$ 25h: Set duration = "300s" (5m buffer) to absorb transient blips and scale-up lag (except immediate job failure alerts which use duration = "0s").

    • Lookbacks $> 25$h (e.g. 3d/7d Slow Burn): Omit duration entirely (or set to 0s). Cloud Monitoring rejects PromQL queries with duration set on lookbacks >25h (INVALID_ARGUMENT).

  • Terraform Standards & Mandatory Labels:

    • Output clean, complete .tf configurations using google_monitoring_alert_policy and condition_prometheus_query_language directly in your response.

    • Mandatory User Labels: Every google_monitoring_alert_policy resource MUST include a user_labels block containing:

      user_labels = {
  created-with-google-skill = "cloud-run-alert-configuration"
}

    • Include alert_strategy { auto_close = "604800s" } and parameterize notification_channels = var.notification_channels.

WORKFLOW STEPS

1. Discovery & Target Identification

  • Fast Path (Target Named in Prompt): If the user prompt names the target Cloud Run service, job, or worker pool, skip all discovery commands and file searches and proceed directly to Step 2.

  • Discovery Fallback (Target Unnamed): Only if no resource name is provided in the prompt, check local .tf files or run gcloud to identify the target workload type and name. If gcloud auth fails, fall back immediately to default Terraform variables (var.scoping_project_id).

2. Configure Alerts

  • Route to the corresponding guide to generate the alert policies:

    • HTTP Services: Open services.md. Apply the requested alerting policy or standard suite covering availability SLOs (5xx), request latency (P95/P99), client errors (4xx), container instance saturation, container CPU/memory utilization, traffic anomalies (drop/surge), and billable instance time.

    • Batch Jobs: Open jobs.md. Apply immediate job execution failure alerts (duration = "0s").

    • Worker Pools: Open worker_pools.md. Apply the 4-policy standard suite (Task Success SLO Fast/Slow Burn, Backlog ETD, Message Age SLA).

3. Terraform Generation & Review

  • Provide the complete HCL configuration in your response with explicitly parameterized defaults and the mandatory user_labels block (created-with-google-skill = "cloud-run-alert-configuration").

  • State the applied defaults and remind the user of their ability to override any constant.

  • Provide a clear plain-English breakdown of the PromQL logic and triggering thresholds.

Additional Resources

Files

references/worker_pools.md

Cloud Run Worker Pools Alert Configuration

Best-practice observability for Cloud Run Worker Pools.

Table of Contents

  • Fast-Path Defaults

  • Default Constants & Override Protocol

  • SLO Delegation & Fallback Rule

  • Workload Target Discovery

    • Autonomous Parameter Discovery

  • PromQL & Terraform Query Templates

    • 1. Task Success Rate SLO Burn Rate Queries

    • 2. Queue Backlog Drain Time (ETD)

    • 3. Oldest Unacked Message Age

    • 4. Health Probe Success Rate SLO Burn Rate Queries

  • Telemetry Data Sources Reference

Fast-Path Defaults

When standard alerting is requested or thresholds are unspecified, suggest the following 4-policy standard suite:

  1. Task Success Rate SLO (Fast Burn): 99.9% target, 1h & 5m windows, $14.4\times$ burn rate, duration = "300s" (Critical/Paging).

  2. Task Success Rate SLO (Slow Burn): 99.9% target, 3d & 6h windows, $1.0\times$ burn rate, omit duration (Warning/Ticket).

  3. Queue Backlog Drain Time (ETD): Estimated time to drain queue exceeds processing SLA threshold (e.g., >30m) with duration = "300s" retest buffer (Warning/Paging).

  4. Oldest Unacked Message Age: Oldest unacknowledged message age exceeds SLA threshold (Critical/Paging).

[!NOTE] Unlike HTTP Services (where container saturation directly causes HTTP 504 errors), Worker Pools operating at max_instances are operating at maximum efficiency. As long as Queue Drain Time (ETD) and Message Age remain within SLA, instance saturation is expected and does not trigger alerts.

Default Constants & Override Protocol

All generated Terraform alert policies use explicit default constants. Users can override any parameter in HCL variables or via prompt instruction:

Constant / ParameterExplicit DefaultHCL Variable NameDescription & Override Option
task_success_slo_target0.999 (99.9%)var.task_success_slo_targetTask processing success rate target. Override to 0.99 or 0.9999.
health_probe_slo_target0.999 (99.9%)var.health_probe_slo_targetProcess health probe attempt success target.
max_drain_time_second1800 (30m)var.max_drain_time_secondMaximum allowed Queue Estimated Time to Drain (ETD) SLA.
oldest_message_age_second1800 (30m)var.oldest_message_age_secondMaximum allowed unacknowledged message age SLA.
retest_duration_buffer"300s" (5m)var.retest_duration_bufferRetest window for lookbacks $\le 25$h. Omitted for lookbacks $>25$h.

SLO Delegation & Fallback Rule

  • If google-cloud-slo-alert-configuration skill is available: use it to inform SLO creation.

  • If google-cloud-slo-alert-configuration skill is NOT available: Use the standard PromQL burn-rate templates provided directly in this reference guide as a complete, standalone solution.

Workload Target Discovery

Autonomous Parameter Discovery

Before proposing alerts, ensure the Google Cloud SDK is installed and configured properly. Then discover worker pool telemetry and queue bindings:

  • Queue Integration (Direct API & Config Scan):

    1. Worker Pool API: Run gcloud run worker-pools describe [NAME] --format="json" to check container environment variables (spec.template.spec.containers[].env) for PUBSUB_SUBSCRIPTION, SUBSCRIPTION_ID, or TOPIC_ID.

    2. Service Account IAM Binding: Extract serviceAccountName from the worker pool description and run gcloud pubsub subscriptions list --format="json" to check which subscriptions grant roles/pubsub.subscriber to that service account.

    3. Terraform Scan: Search .tf files for google_pubsub_subscription resources associated with the worker pool module.

    • Outcome: If a Pub/Sub subscription is identified, apply Task Success SLOs, Backlog Drain Time (ETD), and Message Age alerts.

  • Non-Pub/Sub & Alternative Consumer Workloads: Worker pools can also operate as pull-based consumers for alternative queue systems (e.g., Apache Kafka, RabbitMQ) or run custom background event loops without Pub/Sub bindings. When not bound to Pub/Sub, the primary alerting signals to configure are container CPU/Memory utilization and process health probes (run.googleapis.com/container/completed_probe_attempt_count).

PromQL & Terraform Query Templates

1. Task Success Rate SLO Burn Rate Queries
Fast Burn (1h & 5m Windows)
(
  (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[5m])) or vector(0))
  /
  (
    (sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="[SUBSCRIPTION_ID]"}[5m])) or vector(0))
    +
    (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[5m])) or vector(0))
    + 0.001
  )
  > (1 - [SLO_TARGET]) * 14.4
)
and
(
  (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[1h])) or vector(0))
  /
  (
    (sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="[SUBSCRIPTION_ID]"}[1h])) or vector(0))
    +
    (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[1h])) or vector(0))
    + 0.001
  )
  > (1 - [SLO_TARGET]) * 14.4
)
Slow Burn (3d & 6h Windows)

[!WARNING] Lookback windows >25h (like [3d]) require omitting duration in HCL.

(
  (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[6h])) or vector(0))
  /
  (
    (sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="[SUBSCRIPTION_ID]"}[6h])) or vector(0))
    +
    (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[6h])) or vector(0))
    + 0.001
  )
  > (1 - [SLO_TARGET]) * 1.0
)
and
(
  (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[3d])) or vector(0))
  /
  (
    (sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="[SUBSCRIPTION_ID]"}[3d])) or vector(0))
    +
    (sum(rate(pubsub_googleapis_com:subscription_nack_requests{subscription_id="[SUBSCRIPTION_ID]"}[3d])) or vector(0))
    + 0.001
  )
  > (1 - [SLO_TARGET]) * 1.0
)
2. Queue Backlog Drain Time (ETD)

Monitors estimated queue drain processing time in seconds.

[!IMPORTANT] PromQL conditions for ETD MUST specify duration = "300s" (5-minute retest buffer) in HCL. This buffer absorbs temporary ETD spikes caused by autoscaler scale-up lag during sudden bursty traffic.

PromQL Query
(
  (sum(pubsub_googleapis_com:subscription_num_undelivered_messages{subscription_id="[SUBSCRIPTION_ID]"}) or vector(0))
  /
  clamp_min((sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="[SUBSCRIPTION_ID]"}[5m])) or vector(0)), 0.05)
) > [MAX_DRAIN_TIME_SECONDS]
Terraform Snippet (ETD Alert Policy)
variable "worker_pools_etd" {
  description = "Map of worker pools to subscription IDs and maximum drain time SLAs (in seconds)"
  type = map(object({
    subscription_id       = string
    max_drain_time_second = number
  }))
  default = {
    "order-processor" = {
      subscription_id       = "order-topic-sub"
      max_drain_time_second = 1800 # 30 minutes SLA
    }
  }
}

variable "retest_duration_buffer" {
  description = "Retest duration buffer for lookbacks <= 25h"
  type        = string
  default     = "300s"
}

resource "google_monitoring_alert_policy" "queue_backlog_etd" {
  for_each     = var.worker_pools_etd
  project      = var.scoping_project_id
  display_name = "[Worker Pool] Queue Backlog Estimated Time to Drain (ETD) Exceeded - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Estimated drain time on ${each.value.subscription_id} exceeds ${each.value.max_drain_time_second}s"
    condition_prometheus_query_language {
      query = <<-EOT
        (
          (sum(pubsub_googleapis_com:subscription_num_undelivered_messages{subscription_id="${each.value.subscription_id}"}) or vector(0))
          /
          clamp_min((sum(rate(pubsub_googleapis_com:subscription_ack_message_count{subscription_id="${each.value.subscription_id}"}[5m])) or vector(0)), 0.05)
        ) > ${each.value.max_drain_time_second}
      EOT
      duration = var.retest_duration_buffer # 5m retest buffer to absorb autoscaling scale-up lag
    }
  }

  user_labels = {
    severity                    = "warning"
    worker                      = each.key
    tier                        = "saturation"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
3. Oldest Unacked Message Age
(
  (pubsub_googleapis_com:subscription_oldest_unacked_message_age{subscription_id="[SUBSCRIPTION_ID]"} or vector(0))
) > [AGE_THRESHOLD_SECONDS]
4. Health Probe Success Rate SLO Burn Rate Queries
Fast Burn (1h & 5m Windows)
(
  (sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]", is_healthy="false"}[5m])) or vector(0))
  /
  sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]"}[5m]))
  > (1 - [SLO_TARGET]) * 14.4
)
and
(
  (sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]", is_healthy="false"}[1h])) or vector(0))
  /
  sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]"}[1h]))
  > (1 - [SLO_TARGET]) * 14.4
)
Slow Burn (3d & 6h Windows)

[!WARNING] Lookback windows >25h (like [3d]) require omitting duration in HCL.

(
  (sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]", is_healthy="false"}[6h])) or vector(0))
  /
  sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]"}[6h]))
  > (1 - [SLO_TARGET]) * 1.0
)
and
(
  (sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]", is_healthy="false"}[3d])) or vector(0))
  /
  sum(rate(run_googleapis_com:container_completed_probe_attempt_count{monitored_resource="cloud_run_worker_pool", worker_pool_name="[WORKER_POOL_NAME]"}[3d]))
  > (1 - [SLO_TARGET]) * 1.0
)

Telemetry Data Sources Reference

<!-- mdformat off -->

Signal IndicatorNative MetricSRE Strategy
Task Availabilitypubsub.googleapis.com/subscription/nack_requestsSLO Skill (google-cloud-slo-alert-configuration): Fast Burn (1h/5m) & Slow Burn (3d/6h)
Process Health SLOrun.googleapis.com/container/completed_probe_attempt_countSLO Skill (google-cloud-slo-alert-configuration): Probe Failure Ratio (is_healthy="false") Fast & Slow Burn
Queue Backlog ETDpubsub.googleapis.com/subscription/num_undelivered_messages & pubsub.googleapis.com/subscription/ack_message_countEstimated Time to Drain (ETD): Drain time > SLA threshold with duration = "300s" retest buffer
Message Agepubsub.googleapis.com/subscription/oldest_unacked_message_ageThreshold Alert: Message age > SLA limit (e.g., 30m)

<!-- mdformat on -->

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Declares allowed tools

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Google
Publisher
Google (Official)
Source
google/skills · skills/cloud/cloud-run-alert-configuration · commit 2194d3b
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Google.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download cloud-run-alert-configuration.zip