Azure Aigateway

MicrosoftOfficialCreated by Microsoft

Skill
Tool integration
Cloud and DevOps
MIT
Low risk

From the author

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway.

— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides the configuration patterns and policy references needed to transform Azure API Management (APIM) into a robust AI Gateway. It enables centralized governance for AI models, including Azure OpenAI and AI Foundry, by implementing essential features like semantic caching, token usage tracking, load balancing, and content safety filtering. The skill covers the full lifecycle from backend setup and managed identity authentication to advanced policy application and troubleshooting.

What it does

Connects AI models as backends within Azure API Management.
Implements cost control through token limits and semantic caching.
Enforces security policies like jailbreak detection and content safety.
Enables load balancing across multiple AI backend instances.
Provides observability by emitting token usage metrics.
Facilitates MCP tool protection and rate limiting.

When to use it

When you need to govern and secure access to AI models in production.
When you want to implement cost controls for LLM usage.
When you need to load balance requests across multiple AI regions.
When you need to protect AI agents with content filtering and jailbreak detection.
When you want to convert existing APIs into MCP-compatible tools.

Advantages

Centralized management of AI policies and security.
Significant cost savings via semantic caching.
Improved reliability through load balancing and retry logic.
Enhanced security using managed identities for backend authentication.
Detailed observability into token consumption and model performance.

Usage examples

“Add a new Azure OpenAI backend to my existing APIM instance.”: The agent provides the CLI commands to create the backend and the XML policy snippet to authenticate using managed identity.
“How do I implement semantic caching to save on AI costs?”: The agent explains how to apply the azure-openai-semantic-cache-lookup/store policies within the APIM inbound processing block.
“Set up load balancing for my AI models across two regions.”: The agent provides a policy pattern that uses a variable to rotate between backend URLs based on request hash.
“How can I protect my AI gateway from jailbreak attempts?”: The agent guides you to apply the llm-content-safety policy to filter harmful content and detect malicious inputs.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/microsoft/azure-aigateway/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip azure-aigateway.zip -d .agents/skills

Documentation by Microsoft

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

Category	Triggers
Model Governance	"semantic caching", "token limits", "load balance AI", "track token usage"
Tool Governance	"rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP"
Agent Governance	"content safety", "jailbreak detection", "filter harmful content"
Configuration	"add Azure OpenAI backend", "configure my model", "add AI Foundry model"
Testing	"test AI gateway", "call OpenAI through gateway"

Quick Reference

Policy	Purpose	Details
`azure-openai-token-limit`	Cost control	Model Policies
`azure-openai-semantic-cache-lookup/store`	60-80% cost savings	Model Policies
`azure-openai-emit-token-metric`	Observability	Model Policies
`llm-content-safety`	Safety & compliance	Agent Policies
`rate-limit-by-key`	MCP/tool protection	Tool Policies

Get Gateway Details

# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>

Test AI Endpoint

GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

Common Tasks

Add AI Backend

See references/patterns.md for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>

Apply AI Governance Policy

Recommended policy order in <inbound>:

Authentication - Managed identity to backend
Semantic Cache Lookup - Check cache before calling AI
Token Limits - Cost control
Content Safety - Filter harmful content
Backend Selection - Load balancing
Metrics - Token usage tracking

See references/policies.md for complete example.

Troubleshooting

Issue	Solution
Token limit 429	Increase `tokens-per-minute` or add load balancing
No cache hits	Lower `score-threshold` to 0.7
Content false positives	Increase category thresholds (5-6)
Backend auth 401	Grant APIM "Cognitive Services User" role

See references/troubleshooting.md for details.

References

Detailed Policies - Full policy examples
Configuration Patterns - Step-by-step patterns
Troubleshooting - Common issues
AI-Gateway Samples
GenAI Gateway Docs

SDK Quick References

Content Safety: Python | TypeScript
API Management: Python | .NET

Files

ATTRIBUTION.mdAdded by AutoGineer977 B
LICENSELicence file added by AutoGineer1.1 KiB
SKILL.md4.9 KiB
references/auth-best-practices.md6.1 KiB
references/patterns.md6.6 KiB
references/policies.md9.2 KiB
references/sdk/azure-ai-contentsafety-py.md1.3 KiB
references/sdk/azure-ai-contentsafety-ts.md1.4 KiB
references/sdk/azure-mgmt-apimanagement-dotnet.md1.3 KiB
references/sdk/azure-mgmt-apimanagement-py.md1.0 KiB
references/troubleshooting.md7.7 KiB

references/auth-best-practices.md

Azure Authentication Best Practices

Source: Microsoft — Passwordless connections for Azure services and Azure Identity client libraries.

Golden Rule

Use managed identities and Azure RBAC in production. Reserve DefaultAzureCredential for local development only.

Authentication by Environment

Environment	Recommended Credential	Why
Production (Azure-hosted)	`ManagedIdentityCredential` (system- or user-assigned)	No secrets to manage; auto-rotated by Azure
Production (on-premises)	`ClientCertificateCredential` or `WorkloadIdentityCredential`	Deterministic; no fallback chain overhead
CI/CD pipelines	`AzurePipelinesCredential` / `WorkloadIdentityCredential`	Scoped to pipeline identity
Local development	`DefaultAzureCredential`	Chains CLI, PowerShell, and VS Code credentials for convenience

Why Not `DefaultAzureCredential` in Production?

Unpredictable fallback chain — walks through multiple credential types, adding latency and making failures harder to diagnose.
Broad surface area — checks environment variables, CLI tokens, and other sources that should not exist in production.
Non-deterministic — which credential actually authenticates depends on the environment, making behavior inconsistent across deployments.
Performance — each failed credential attempt adds network round-trips before falling back to the next.

Production Patterns

.NET

using Azure.Identity;

var credential = Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT") == "Development"
    ? new DefaultAzureCredential()                          // local dev — uses CLI/VS credentials
    : new ManagedIdentityCredential();                      // production — deterministic, no fallback chain
// For user-assigned identity: new ManagedIdentityCredential("<client-id>")

TypeScript / JavaScript

import { DefaultAzureCredential, ManagedIdentityCredential } from "@azure/identity";

const credential = process.env.NODE_ENV === "development"
  ? new DefaultAzureCredential()                          // local dev — uses CLI/VS credentials
  : new ManagedIdentityCredential();                      // production — deterministic, no fallback chain
// For user-assigned identity: new ManagedIdentityCredential("<client-id>")

Python

import os
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential

credential = (
    DefaultAzureCredential()                              # local dev — uses CLI/VS credentials
    if os.getenv("AZURE_FUNCTIONS_ENVIRONMENT") == "Development"
    else ManagedIdentityCredential()                      # production — deterministic, no fallback chain
)
# For user-assigned identity: ManagedIdentityCredential(client_id="<client-id>")

Java

import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.identity.ManagedIdentityCredentialBuilder;

var credential = "Development".equals(System.getenv("AZURE_FUNCTIONS_ENVIRONMENT"))
    ? new DefaultAzureCredentialBuilder().build()          // local dev — uses CLI/VS credentials
    : new ManagedIdentityCredentialBuilder().build();      // production — deterministic, no fallback chain
// For user-assigned identity: new ManagedIdentityCredentialBuilder().clientId("<client-id>").build()

Local Development Setup

DefaultAzureCredential is ideal for local dev because it automatically picks up credentials from developer tools:

Azure CLI — az login
Azure Developer CLI — azd auth login
Azure PowerShell — Connect-AzAccount
Visual Studio / VS Code — sign in via Azure extension

import { DefaultAzureCredential } from "@azure/identity";

// Local development only — uses CLI/PowerShell/VS Code credentials
const credential = new DefaultAzureCredential();

Environment-Aware Pattern

Detect the runtime environment and select the appropriate credential. The key principle: use DefaultAzureCredential only when running locally, and a specific credential in production.

Tip: Azure Functions sets AZURE_FUNCTIONS_ENVIRONMENT to "Development" when running locally. For App Service or containers, use any environment variable you control (e.g. NODE_ENV, ASPNETCORE_ENVIRONMENT).

import { DefaultAzureCredential, ManagedIdentityCredential } from "@azure/identity";

function getCredential() {
  if (process.env.NODE_ENV === "development") {
    return new DefaultAzureCredential();          // picks up az login / VS Code creds
  }
  return process.env.AZURE_CLIENT_ID
    ? new ManagedIdentityCredential(process.env.AZURE_CLIENT_ID)  // user-assigned
    : new ManagedIdentityCredential();                            // system-assigned
}

Security Checklist

[ ] Use managed identity for all Azure-hosted apps
[ ] Never hardcode credentials, connection strings, or keys
[ ] Apply least-privilege RBAC roles at the narrowest scope
[ ] Use ManagedIdentityCredential (not DefaultAzureCredential) in production
[ ] Store any required secrets in Azure Key Vault
[ ] Rotate secrets and certificates on a schedule
[ ] Enable Microsoft Defender for Cloud on production resources

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs a command-line tool

The author says: Requires Azure CLI (az) for configuration and testing

Security

What our scanner noticed· Low risk

Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Microsoft

Publisher

Microsoft (Official)

Source

microsoft/skills · .github/plugins/azure-skills/skills/azure-aigateway · commit 0696600

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from .github/plugins/azure-skills/LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

Also distributed in

microsoft/azure-skills

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-aigateway.zip