AutoGineer
Generated banner for Cloud Run Alert Configuration
Generated logo for Cloud Run Alert Configuration

Cloud Run Alert Configuration

GoogleOfficialPublished by Google

  • Skill
  • Workflow
  • Cloud and DevOps
  • Apache-2.0
  • No issues found
Download ZIP

From the author

Configures best-practice, high-signal alerting policies for Cloud Run resources on Google Cloud (services, jobs, and worker pools) based on seasoned SRE practices. Use when analyzing, recommending, writing, or deploying Terraform PromQL alerting policies to monitor Cloud Run error rates (4xx/5xx), request latency, container instance saturation (warning/critical), container CPU/memory utilization and allocation, billable instance time, job execution status, and worker pool queue backlog. Don't use for GKE workloads (use gke-alert-configuration) or Compute Engine VMs.
— the description by Google

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a standardized approach to configuring high-signal alerting policies for Google Cloud Run resources. By leveraging Terraform and PromQL, it enables developers and SREs to implement observability based on seasoned SRE practices. It covers HTTP services, batch jobs, and worker pools, ensuring that critical metrics like error rates, latency, and saturation are monitored effectively. The skill emphasizes parameterization, mandatory labeling for governance, and adherence to specific lookback and duration rules for Cloud Monitoring.

What it does

  • Generates Terraform HCL code for Google Cloud Monitoring alert policies.
  • Applies SRE-based best practices for monitoring Cloud Run services, jobs, and worker pools.
  • Configures PromQL queries for error rates, latency, saturation, and execution status.
  • Enforces mandatory user labels for alert policy governance.
  • Provides a fast-path workflow for users who specify resource names in their prompt.
  • Includes logic for handling PromQL duration rules based on lookback windows.

When to use it

  • When you need to set up production-grade alerting for Cloud Run resources.
  • When you want to define monitoring policies as code using Terraform.
  • When you need to implement specific SRE alerting patterns for HTTP services or batch jobs.
  • When you require consistent alerting configurations across multiple Cloud Run environments.

Advantages

  • Reduces configuration errors by using standardized, SRE-vetted templates.
  • Ensures consistent observability across different Cloud Run workload types.
  • Promotes infrastructure-as-code practices for monitoring policies.
  • Simplifies the creation of complex PromQL queries for Cloud Monitoring.
  • Supports centralized metrics scoping via parameterized project variables.

Usage examples

“Generate a Terraform alert policy for my Cloud Run service named 'api-gateway' to monitor 5xx error rates.”
The agent generates a complete Terraform resource block for the 'api-gateway' service, including the PromQL query for 5xx errors, mandatory labels, and alert strategy settings.
“Create an alert policy for a Cloud Run job called 'nightly-sync' that pages immediately on failure.”
The agent provides a Terraform configuration using a 0s duration PromQL query to trigger an immediate alert upon any failed execution of the 'nightly-sync' job.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/google/cloud-run-alert-configuration/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip cloud-run-alert-configuration.zip -d .agents/skills

Documentation by Google

Cloud Run Alert Configuration

Production-grade observability for Cloud Run using Terraform and PromQL (Cloud Monitoring). Grounded in SRE practices, this skill focuses strictly on actionable user impact and scaling bounds.

Attribution

MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:cloud-run-alert-configuration)" and append agent-skills/1.0 (skill:cloud-run-alert-configuration) to User-Agent headers.

CRITICAL RULES

  • Prompt-First Fast Path (Skip Discovery When Named):

    • If the user prompt explicitly specifies the target Cloud Run service, job, or worker pool name (e.g., 'video-encoder', 'nightly-reconciliation', 'web-frontend', 'catalog-service', 'api-gateway', 'order-processor'), SKIP all workspace .tf file scanning (find_by_name, code_search, list_dir) and gcloud CLI discovery commands entirely.

    • Do NOT run gcloud, terraform, or file search tools when the target name is already provided in the prompt. Instead, parameterize the project ID (variable "scoping_project_id" { default = "my-gcp-project" }) and target resource name in Terraform variables and proceed immediately to Step 2 (Configure Alerts).

  • Autonomous Discovery (Only When Target Name is Omitted):

    • Never Scan Root Monorepo or Unbounded Directories: Never run find_by_name or ls across root workspace directories.

    • Config First: Only if the prompt omits the resource name, check .tf files in the immediate working directory for google_cloud_run_v2_service, google_cloud_run_service, or google_cloud_run_v2_job.

    • CLI Second (Graceful Fallback): Only if unconfigured in prompt or local .tf files, attempt gcloud config get-value project and gcloud run services list. If any gcloud command fails (e.g., auth or metadata errors) or terraform is missing, immediately stop running CLI commands and output parameterized HCL using explicit variable defaults.

  • Workload Routing: Always classify the workload target and follow its specific reference guide:

  • Explicit Defaults & User Overrides:

    • Always use explicit defaults for all constants specified in the target workload's reference file (SLO targets, latency thresholds, SLAs, saturation ceilings, rate guards).

    • State the defaults being applied in the final summary output and clearly notify the user that any default constant can be customized or overridden via Terraform variables or prompt input.

  • Metric Scope Centralization: Parameterize project = var.scoping_project_id in all Terraform google_monitoring_alert_policy resources so the policy can target either a single project or a centralized Cloud Monitoring Metrics Scope.

  • PromQL duration (Retest Window) Rules:

    • Lookbacks $\le$ 25h: Set duration = "300s" (5m buffer) to absorb transient blips and scale-up lag (except immediate job failure alerts which use duration = "0s").

    • Lookbacks $> 25$h (e.g. 3d/7d Slow Burn): Omit duration entirely (or set to 0s). Cloud Monitoring rejects PromQL queries with duration set on lookbacks >25h (INVALID_ARGUMENT).

  • Terraform Standards & Mandatory Labels:

    • Output clean, complete .tf configurations using google_monitoring_alert_policy and condition_prometheus_query_language directly in your response.

    • Mandatory User Labels: Every google_monitoring_alert_policy resource MUST include a user_labels block containing:

      user_labels = {
  created-with-google-skill = "cloud-run-alert-configuration"
}

    • Include alert_strategy { auto_close = "604800s" } and parameterize notification_channels = var.notification_channels.

WORKFLOW STEPS

1. Discovery & Target Identification

  • Fast Path (Target Named in Prompt): If the user prompt names the target Cloud Run service, job, or worker pool, skip all discovery commands and file searches and proceed directly to Step 2.

  • Discovery Fallback (Target Unnamed): Only if no resource name is provided in the prompt, check local .tf files or run gcloud to identify the target workload type and name. If gcloud auth fails, fall back immediately to default Terraform variables (var.scoping_project_id).

2. Configure Alerts

  • Route to the corresponding guide to generate the alert policies:

    • HTTP Services: Open services.md. Apply the requested alerting policy or standard suite covering availability SLOs (5xx), request latency (P95/P99), client errors (4xx), container instance saturation, container CPU/memory utilization, traffic anomalies (drop/surge), and billable instance time.

    • Batch Jobs: Open jobs.md. Apply immediate job execution failure alerts (duration = "0s").

    • Worker Pools: Open worker_pools.md. Apply the 4-policy standard suite (Task Success SLO Fast/Slow Burn, Backlog ETD, Message Age SLA).

3. Terraform Generation & Review

  • Provide the complete HCL configuration in your response with explicitly parameterized defaults and the mandatory user_labels block (created-with-google-skill = "cloud-run-alert-configuration").

  • State the applied defaults and remind the user of their ability to override any constant.

  • Provide a clear plain-English breakdown of the PromQL logic and triggering thresholds.

Additional Resources

Files

references/services.md

Cloud Run Services Alert Configuration

Best-practice observability for HTTP-triggered Cloud Run Services.

Table of Contents

  • Fast-Path Defaults (#fast-path-defaults)

    • Core SLO & Instance Saturation Suite

    • Extended Dashboard & Resource Protection Suite

  • Default Constants & Override Protocol

  • SLO Delegation & Fallback Rule (~line 83)

  • PromQL & Terraform Query Templates

    • 1. Container Instance Saturation

    • 2. Container Memory & CPU Utilization (OOM Prevention & Throttling)

    • 3. Availability SLO Burn Rate Queries

    • 4. Request Latency SLO Queries (P95 & P99)

    • 5. Client Error Surge (4xx Response Code Storm)

    • 6. Traffic & FinOps Anomalies

    • 7. Conditional Diagnostic: Cold Start Latency

  • Telemetry Data Sources Reference

Fast-Path Defaults

When standard alerting is requested or thresholds are unspecified, apply the standard suite tailored to user requirements:

Core SLO & Instance Saturation Suite

  1. Availability SLO (Fast Burn): 99.9% target, 1h & 5m windows, $14.4\times$ burn rate, duration = "300s" (Critical/Paging).

  2. Availability SLO (Slow Burn): 99.9% target, 3d & 6h windows, $1.0\times$ burn rate, omit duration (Warning/Ticket).

  3. Request Latency SLO (P95): Window-based 1h lookback, P95 $\le 500\text{ms}$ in 99% of 5m windows (Warning).

  4. Request Latency SLO (P99): Window-based 1h lookback, P99 $\le 1000\text{ms}$ in 99% of 5m windows (Warning).

  5. Instance Saturation (Warning): >80% of service max_instances ceiling (Warning).

  6. Instance Saturation (Critical): $\ge 95%$ of service max_instances ceiling (Critical).

  7. Traffic Drop Anomaly: 5m throughput drop vs. 1h moving average baseline with min-volume guard (rate(...[1h]) > 1.0).

Extended Dashboard & Resource Protection Suite

  1. Container Memory Saturation & OOM Risk: P95 container memory utilization $\ge 90%$ (Critical - Exit Code 137 prevention).

  2. Container CPU Saturation & Throttling: P95 container CPU utilization $> 85%$ (Warning).

  3. Client Error Surge (4xx): 4xx response code ratio $> 10%$ with 5 req/s min-volume guard (Warning).

  4. Billable Instance Time Surge: 5m billable time rate $> 3\times$ 1h moving average baseline (FinOps Warning).

  5. Traffic Surge Anomaly: 5m request rate $> 3\times$ 1h moving average baseline with min-volume guard (Warning).

Default Constants & Override Protocol

All generated Terraform alert policies use explicit default constants. Users can override any parameter in HCL variables or via prompt instruction:

Constant / ParameterExplicit DefaultHCL Variable NameDescription & Override Option
availability_slo_target0.999 (99.9%)var.availability_slo_targetAvailability target ratio. Override to 0.99 or 0.9999.
latency_p95_threshold_ms500 (500ms)var.latency_p95_threshold_msP95 latency limit in milliseconds.
latency_p99_threshold_ms1000 (1000ms)var.latency_p99_threshold_msP99 latency limit in milliseconds.
saturation_warning_ratio0.80 (80%)var.saturation_warning_ratioInstance saturation warning threshold relative to max_instances.
saturation_critical_ratio0.95 (95%)var.saturation_critical_ratioInstance saturation critical threshold relative to max_instances.
memory_saturation_ratio0.90 (90%)var.memory_saturation_ratioP95 container memory utilization critical limit (OOM prevention).
cpu_saturation_ratio0.85 (85%)var.cpu_saturation_ratioP95 container CPU utilization warning limit.
client_error_4xx_ratio0.10 (10%)var.client_error_4xx_ratioMaximum allowable ratio of 4xx client errors.
traffic_drop_ratio0.20 (80% drop)var.traffic_drop_ratioThroughput drop threshold vs 1h moving average.
traffic_min_hourly_rate1.0 (1 req/s)var.traffic_min_hourly_rateMin hourly rate guard to prevent scale-to-zero false alarms.
traffic_surge_multiplier3.0 (3x surge)var.traffic_surge_multiplierThroughput spike multiplier vs 1h moving average.
traffic_surge_min_rate15.0 (15 req/s)var.traffic_surge_min_rateMinimum 5m request rate guard (req/s) to suppress traffic surge false alarms on low volume.
billable_time_surge_multiplier3.0 (3x surge)var.billable_time_surge_multiplierBillable compute time multiplier vs 1h moving average.
billable_time_min_rate5.0 (5 s/s)var.billable_time_min_rateMinimum 5m billable instance-seconds rate guard to prevent FinOps false alarms on low baseline usage.
retest_duration_buffer"300s" (5m)var.retest_duration_bufferRetest window for lookbacks $\le 25$h. Omitted for lookbacks $>25$h.

SLO Delegation & Fallback Rule

  • If google-cloud-slo-alert-configuration skill is available: use it to inform SLO creation.

  • If google-cloud-slo-alert-configuration skill is NOT available: Use the standard PromQL burn-rate templates provided directly in this reference guide as a complete, standalone solution.

PromQL & Terraform Query Templates

1. Container Instance Saturation

Monitors when active container instances approach configured scaling ceilings (max_instances).

[!NOTE] PromQL queries for container_instance_count MUST include the label matcher monitored_resource="cloud_run_revision" to prevent API validation errors resulting from metric ambiguity with cloud_run_worker_pool.

Warning Query (>80% Capacity)
sum(
  max_over_time(run_googleapis_com:container_instance_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", state="active"}[5m])
) by (service_name, location) / [SERVICE_MAX_INSTANCES] > 0.80
Critical Query (>=95% Capacity)
sum(
  max_over_time(run_googleapis_com:container_instance_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", state="active"}[5m])
) by (service_name, location) / [SERVICE_MAX_INSTANCES] >= 0.95
Terraform Snippet (Instance Saturation Policies)
variable "cloud_run_services" {
  description = "Map of Cloud Run services to monitor and their max_instances ceilings"
  type = map(object({
    max_instances = number
  }))
  default = {
    "api-gateway" = { max_instances = 1000 }
  }
}

variable "saturation_warning_ratio" {
  description = "Ratio threshold for container instance saturation warning"
  type        = number
  default     = 0.80
}

variable "saturation_critical_ratio" {
  description = "Ratio threshold for container instance saturation critical"
  type        = number
  default     = 0.95
}

resource "google_monitoring_alert_policy" "instance_saturation_warning" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Container Instance Saturation Warning (>80% Limit) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Active Instances Exceed 80% of ${each.key} Ceiling (${each.value.max_instances})"
    condition_prometheus_query_language {
      query = <<-EOT
        sum(
          max_over_time(run_googleapis_com:container_instance_count{monitored_resource="cloud_run_revision", service_name="${each.key}", state="active"}[5m])
        ) by (service_name, location) / ${each.value.max_instances} > ${var.saturation_warning_ratio}
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "saturation"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}

resource "google_monitoring_alert_policy" "instance_saturation_critical" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Container Instance Saturation Critical (>=95% Limit) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Active Instances Exceed 95% of ${each.key} Ceiling (${each.value.max_instances})"
    condition_prometheus_query_language {
      query = <<-EOT
        sum(
          max_over_time(run_googleapis_com:container_instance_count{monitored_resource="cloud_run_revision", service_name="${each.key}", state="active"}[5m])
        ) by (service_name, location) / ${each.value.max_instances} >= ${var.saturation_critical_ratio}
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "critical"
    service                     = each.key
    tier                        = "saturation"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
2. Container Memory & CPU Utilization (OOM Prevention & Throttling)

Monitors actual CPU and Memory resource consumption inside container instances.

2.1 Container Memory Saturation (OOM Risk)

Triggers when the 95th percentile memory utilization reaches critical levels, preventing container abrupt termination via Linux kernel OOMKilled (exit code 137).

histogram_quantile(0.95,
  sum(
    rate(run_googleapis_com:container_memory_utilizations_bucket{
      monitored_resource="cloud_run_revision",
      service_name="[SERVICE_NAME]"
    }[5m])
  ) by (le, service_name, location)
) >= 0.90
2.2 Container CPU Saturation & Throttling

Triggers when the 95th percentile container CPU utilization exceeds 85%, indicating compute starvation or inadequate vCPU sizing.

histogram_quantile(0.95,
  sum(
    rate(run_googleapis_com:container_cpu_utilizations_bucket{
      monitored_resource="cloud_run_revision",
      service_name="[SERVICE_NAME]"
    }[5m])
  ) by (le, service_name, location)
) > 0.85
Terraform Snippet (Resource Saturation Policies)
variable "memory_saturation_ratio" {
  description = "P95 container memory utilization threshold (OOM danger)"
  type        = number
  default     = 0.90
}

variable "cpu_saturation_ratio" {
  description = "P95 container CPU utilization threshold (throttling danger)"
  type        = number
  default     = 0.85
}

resource "google_monitoring_alert_policy" "container_memory_saturation_critical" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Container Memory Saturation Critical (P95 >= 90% OOM Risk) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "P95 Memory Utilization >= 90% on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        histogram_quantile(0.95,
          sum(
            rate(run_googleapis_com:container_memory_utilizations_bucket{
              monitored_resource = "cloud_run_revision",
              service_name       = "${each.key}"
            }[5m])
          ) by (le, service_name, location)
        ) >= ${var.memory_saturation_ratio}
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "critical"
    service                     = each.key
    tier                        = "saturation"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}

resource "google_monitoring_alert_policy" "container_cpu_saturation_warning" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Container CPU Saturation Warning (P95 > 85%) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "P95 CPU Utilization > 85% on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        histogram_quantile(0.95,
          sum(
            rate(run_googleapis_com:container_cpu_utilizations_bucket{
              monitored_resource = "cloud_run_revision",
              service_name       = "${each.key}"
            }[5m])
          ) by (le, service_name, location)
        ) > ${var.cpu_saturation_ratio}
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "saturation"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
3. Availability SLO Burn Rate Queries
Availability Fast Burn (1h & 5m Windows)
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", response_code_class="5xx"}[5m])) by (service_name, location)
  /
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location)
  > (1 - [SLO_TARGET]) * 14.4
)
and
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", response_code_class="5xx"}[1h])) by (service_name, location)
  /
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location)
  > (1 - [SLO_TARGET]) * 14.4
)
and
(
  # Min-volume guard: ensure at least 1 req/s to prevent false alerts on scale-to-zero or low-traffic services
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location) > 1.0
)
Availability Slow Burn (3d & 6h Windows)

[!WARNING] Lookback windows >25h (like [3d]) require omitting duration in HCL.

(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", response_code_class="5xx"}[6h])) by (service_name, location)
  /
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[6h])) by (service_name, location)
  > (1 - [SLO_TARGET]) * 1.0
)
and
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", response_code_class="5xx"}[3d])) by (service_name, location)
  /
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[3d])) by (service_name, location)
  > (1 - [SLO_TARGET]) * 1.0
)
and
(
  # Min-volume guard: ensure at least 1 req/s to prevent false alerts on scale-to-zero or low-traffic services
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[6h])) by (service_name, location) > 1.0
)
4. Request Latency SLO Queries (P95 & P99)
P95 Request Latency SLO
avg_over_time(
  (
    histogram_quantile(0.95, sum(rate(run_googleapis_com:request_latencies_bucket{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (le, service_name, location)) > [LATENCY_THRESHOLD_MS]
  )[1h:5m]
) > (1 - [WINDOW_SLO_TARGET])
P99 Request Latency SLO
avg_over_time(
  (
    histogram_quantile(0.99, sum(rate(run_googleapis_com:request_latencies_bucket{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (le, service_name, location)) > [LATENCY_THRESHOLD_MS]
  )[1h:5m]
) > (1 - [WINDOW_SLO_TARGET])
5. Client Error Surge (4xx Response Code Storm)

Detects client authorization failures (401/403) or concurrency rate-limiting (429) across inbound requests.

(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]", response_code_class="4xx"}[5m])) by (service_name, location)
  /
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location)
  > [CLIENT_ERROR_RATIO]
)
and
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location) > [MIN_REQ_RATE]
)
Terraform Snippet (4xx Client Error Policy)
variable "client_error_4xx_ratio" {
  description = "Ratio threshold for 4xx client errors"
  type        = number
  default     = 0.10
}

variable "client_error_min_rate" {
  description = "Minimum request rate (req/s) to guard against low-volume noise"
  type        = number
  default     = 5.0
}

resource "google_monitoring_alert_policy" "client_error_4xx_surge" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Client Error Surge (4xx > 10%) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "4xx Error Ratio Exceeds 10% on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}", response_code_class="4xx"}[5m])) by (service_name, location)
          /
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location)
          > ${var.client_error_4xx_ratio}
        )
        and
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location) > ${var.client_error_min_rate}
        )
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "client_error"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
6. Traffic & FinOps Anomalies
6.1 Traffic Drop Anomaly

Detects sudden traffic drops compared to 1h moving average, guarded against scale-to-zero.

(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location)
  /
  (sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location) + 0.001)
) < [DROP_THRESHOLD_RATIO]
and
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location) > [MIN_HOURLY_RATE]
)
Terraform Snippet (Traffic Drop Policy)
variable "traffic_drop_ratio" {
  description = "Ratio threshold vs 1h average for traffic drop anomaly"
  type        = number
  default     = 0.20 # 80% drop
}

variable "traffic_min_hourly_rate" {
  description = "Minimum hourly request rate (req/s) guard to prevent scale-to-zero false alarms"
  type        = number
  default     = 1.0 # 1 req/s
}

resource "google_monitoring_alert_policy" "traffic_drop_anomaly" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Traffic Drop Anomaly (>80% Drop) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Traffic Dropped >80% vs 1h Average on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location)
          /
          (sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[1h])) by (service_name, location) + 0.001)
        ) < ${var.traffic_drop_ratio}
        and
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[1h])) by (service_name, location) > ${var.traffic_min_hourly_rate}
        )
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "traffic"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
6.2 Traffic Surge Anomaly

Detects sudden volumetric traffic spikes (e.g. DDoS or unexpected viral load) compared to the 1h moving average baseline (> 3x baseline with a minimum rate guard of > 15 req/s).

PromQL Query
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location)
  /
  (sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location) + 0.1)
  > 3.0
)
and
(
  sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location) > 15.0
)
Terraform Snippet
variable "traffic_surge_multiplier" {
  description = "Throughput spike multiplier vs 1h moving average baseline"
  type        = number
  default     = 3.0 # 3x surge
}

variable "traffic_surge_min_rate" {
  description = "Minimum 5m request rate guard (req/s) to suppress false alarms on low volume"
  type        = number
  default     = 15.0 # 15 req/s
}

resource "google_monitoring_alert_policy" "traffic_surge_anomaly" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] Traffic Surge Anomaly (>3x Baseline) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Traffic Surged >3x vs 1h Average on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location)
          /
          (sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[1h])) by (service_name, location) + 0.1)
          > ${var.traffic_surge_multiplier}
        )
        and
        (
          sum(rate(run_googleapis_com:request_count{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location) > ${var.traffic_surge_min_rate}
        )
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "traffic"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
6.3 Billable Instance Time Surge (FinOps Cost Control)

Monitors runaway autoscaling or unoptimized compute consumption by tracking surges in billable container instance time (> 3x 1-hour moving average baseline with a minimum billable instance-seconds guard > 5.0).

PromQL Query
(
  sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location)
  /
  (sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[1h])) by (service_name, location) + 0.1)
  > 3.0
)
and
(
  sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[5m])) by (service_name, location) > 5.0
)
Terraform Snippet
variable "billable_time_surge_multiplier" {
  description = "Billable compute time surge multiplier vs 1h moving average baseline"
  type        = number
  default     = 3.0 # 3x surge
}

variable "billable_time_min_rate" {
  description = "Minimum 5m billable instance-seconds rate guard to suppress alerts on low baseline usage"
  type        = number
  default     = 5.0 # 5 billable instance-seconds/s
}

resource "google_monitoring_alert_policy" "billable_instance_time_surge" {
  for_each     = var.cloud_run_services
  project      = var.scoping_project_id
  display_name = "[Cloud Run] FinOps Billable Instance Time Surge (>3x Baseline) - ${each.key}"
  combiner     = "OR"

  conditions {
    display_name = "Billable Instance Time >3x vs 1h Average on ${each.key}"
    condition_prometheus_query_language {
      query = <<-EOT
        (
          sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location)
          /
          (sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="${each.key}"}[1h])) by (service_name, location) + 0.1)
          > ${var.billable_time_surge_multiplier}
        )
        and
        (
          sum(rate(run_googleapis_com:container_billable_instance_time{monitored_resource="cloud_run_revision", service_name="${each.key}"}[5m])) by (service_name, location) > ${var.billable_time_min_rate}
        )
      EOT
      duration = "300s"
    }
  }

  user_labels = {
    severity                    = "warning"
    service                     = each.key
    tier                        = "finops"
    "created-with-google-skill" = "cloud-run-alert-configuration"
  }

  alert_strategy { auto_close = "604800s" }
  notification_channels = var.notification_channels
}
7. Conditional Diagnostic: Cold Start Latency
histogram_quantile(0.95,
  sum(
    rate(run_googleapis_com:container_startup_latencies_bucket{monitored_resource="cloud_run_revision", service_name="[SERVICE_NAME]"}[30m])
  ) by (le, service_name, location)
) > [STARTUP_LATENCY_SECONDS]

Telemetry Data Sources Reference

Signal IndicatorNative MetricSRE Observability Strategy
Availability (5xx)run.googleapis.com/request_countSLO Fast Burn (1h/5m) & Slow Burn (3d/6h) (response_code_class="5xx")
Request Latency (P50/P95/P99)run.googleapis.com/request_latenciesSLO P95 & P99 Window Evaluation (request_latencies_bucket)
Performance / Client Errors (4xx)run.googleapis.com/request_countClient Error Spike Alert: 4xx ratio $> 10%$ with 5 req/s guard
Container Instance Saturationrun.googleapis.com/container/instance_countScaling Ceiling Warning (0.80) & Critical (0.95) vs max_instances
Container Memory Utilizationrun.googleapis.com/container/memory/utilizationsOOM Prevention Alert: P95 memory utilization $\ge 90%$ (Exit 137 guard)
Container CPU Utilizationrun.googleapis.com/container/cpu/utilizationsCompute Throttling Alert: P95 CPU utilization $> 85%$
Billable Instance Timerun.googleapis.com/container/billable_instance_timeFinOps Cost Spike Alert: 5m rate $> 3\times$ 1h moving average baseline
Container CPU Allocationrun.googleapis.com/container/cpu/allocation_timeRegional vCPU Quota Headroom Monitoring
Container Memory Allocationrun.googleapis.com/container/memory/allocation_timeRegional RAM Quota Headroom Monitoring
Traffic Drop Anomalyrun.googleapis.com/request_countGuarded Anomaly Alert: Drop vs. 1h baseline (rate(...[1h]) > 1.0)
Cold Start Latencyrun.googleapis.com/container/startup_latenciesConditional Diagnostic: P95 startup latency over 30m window

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Declares allowed tools

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Google
Publisher
Google (Official)
Source
google/skills · skills/cloud/cloud-run-alert-configuration · commit 2194d3b
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Google.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download cloud-run-alert-configuration.zip