AutoGineer
Generated banner for Azure Monitor Ingestion Py
Generated logo for Azure Monitor Ingestion Py

Azure Monitor Ingestion Py

MicrosoftOfficialCreated by Microsoft

  • Skill
  • Tool integration
  • Cloud and DevOps
  • MIT
  • Low risk
Download ZIP

From the author

Azure Monitor Ingestion SDK for Python. Use for sending custom logs to Log Analytics workspace via Logs Ingestion API. Triggers: "azure-monitor-ingestion", "LogsIngestionClient", "custom logs", "DCR", "data collection rule", "Log Analytics".
— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The azure-monitor-ingestion-py skill provides the Python client library for interacting with the Azure Logs Ingestion API. It enables developers to programmatically upload custom log data to an Azure Monitor Log Analytics workspace. The SDK supports both synchronous and asynchronous operations, handles automatic batching and compression of log data, and integrates with Azure Identity for authentication.

What it does

  • Uploads custom log data to Azure Monitor Log Analytics workspaces.
  • Supports both synchronous and asynchronous client implementations.
  • Provides automatic chunking, gzip compression, and parallel uploading of log data.
  • Offers error handling callbacks for managing partial upload failures.
  • Supports authentication via Azure Identity, including DefaultAzureCredential.
  • Enables configuration for sovereign cloud environments.

When to use it

  • When you need to send custom application or system logs to Azure Monitor.
  • When working with Data Collection Rules (DCR) and Data Collection Endpoints (DCE) in Azure.
  • When you require a programmatic way to ingest data into custom tables in Log Analytics.
  • When building Python applications that need to integrate with Azure observability pipelines.

Advantages

  • Simplifies ingestion by managing log batching and compression automatically.
  • Provides consistent authentication patterns through Azure Identity.
  • Includes built-in support for handling and retrying failed log uploads.
  • Offers both sync and async clients to fit different application architectures.

Usage examples

“How do I upload a list of logs to Azure Monitor using the Python SDK?”
The agent provides a code snippet using LogsIngestionClient, demonstrating how to initialize the client with credentials and call the upload method with a rule ID, stream name, and log data.
“Show me how to handle errors when uploading logs to Azure.”
The agent provides an example of using an on_error callback function to capture failed logs and implement custom retry logic.
“How can I use the async client for log ingestion?”
The agent provides an example using the azure.monitor.ingestion.aio module, showing the correct usage of async with context managers for both the client and credentials.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/microsoft/azure-monitor-ingestion-py/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip azure-monitor-ingestion-py.zip -d .agents/skills

Documentation by Microsoft

Azure Monitor Ingestion SDK for Python

Send custom logs to Azure Monitor Log Analytics workspace using the Logs Ingestion API.

Installation

pip install azure-monitor-ingestion
pip install azure-identity

Environment Variables

# Data Collection Endpoint (DCE)
AZURE_DCE_ENDPOINT=https://<dce-name>.<region>.ingest.monitor.azure.com  # Required for all auth methods

# Data Collection Rule (DCR) immutable ID
AZURE_DCR_RULE_ID=dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  # Required for all auth methods

# Stream name from DCR
AZURE_DCR_STREAM_NAME=Custom-MyTable_CL  # Required for all auth methods
AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production

Prerequisites

Before using this SDK, you need:

  1. Log Analytics Workspace — Target for your logs

  2. Data Collection Endpoint (DCE) — Ingestion endpoint

  3. Data Collection Rule (DCR) — Defines schema and destination

  4. Custom Table — In Log Analytics (created via DCR or manually)

Authentication & Lifecycle

🔑 Two rules apply to every code sample below:

  1. Prefer DefaultAzureCredential. It works locally (Azure CLI / VS Code / Developer CLI) and in Azure (managed identity, workload identity) with no code change. Avoid connection strings, account/API keys — they bypass Entra audit and rotation.

    • Local dev: DefaultAzureCredential works as-is.

    • Production: set AZURE_TOKEN_CREDENTIALS=prod (or AZURE_TOKEN_CREDENTIALS=<specific_credential>) to constrain the credential chain to production-safe credentials.

  2. Wrap every client in a context manager so HTTP transports, sockets, and token caches are released deterministically:

    • Sync: with <Client>(...) as client:

    • Async: async with <Client>(...) as client: and async with DefaultAzureCredential() as credential: (from azure.identity.aio)

Snippets may abbreviate this setup, but production code should always follow both rules.

from azure.monitor.ingestion import LogsIngestionClient
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
import os

# Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
credential = DefaultAzureCredential(require_envvar=True)
# Or use a specific credential directly in production:
# See https://learn.microsoft.com/python/api/overview/azure/identity-readme?view=azure-python#credential-classes
# credential = ManagedIdentityCredential()

with LogsIngestionClient(
    endpoint=os.environ["AZURE_DCE_ENDPOINT"],
    credential=credential
) as client:
    # Use `client.upload(...)` for all subsequent operations (see examples below)
    ...

Upload Custom Logs

from azure.monitor.ingestion import LogsIngestionClient
from azure.identity import DefaultAzureCredential
import os

rule_id = os.environ["AZURE_DCR_RULE_ID"]
stream_name = os.environ["AZURE_DCR_STREAM_NAME"]

logs = [
    {"TimeGenerated": "2024-01-15T10:00:00Z", "Computer": "server1", "Message": "Application started"},
    {"TimeGenerated": "2024-01-15T10:01:00Z", "Computer": "server1", "Message": "Processing request"},
    {"TimeGenerated": "2024-01-15T10:02:00Z", "Computer": "server2", "Message": "Connection established"}
]

with LogsIngestionClient(
    endpoint=os.environ["AZURE_DCE_ENDPOINT"],
    credential=DefaultAzureCredential()
) as client:
    client.upload(rule_id=rule_id, stream_name=stream_name, logs=logs)

Upload from JSON File

import json

with open("logs.json", "r") as f:
    logs = json.load(f)

client.upload(rule_id=rule_id, stream_name=stream_name, logs=logs)

Custom Error Handling

Handle partial failures with a callback:

failed_logs = []

def on_error(error):
    print(f"Upload failed: {error.error}")
    failed_logs.extend(error.failed_logs)

client.upload(
    rule_id=rule_id,
    stream_name=stream_name,
    logs=logs,
    on_error=on_error
)

# Retry failed logs
if failed_logs:
    print(f"Retrying {len(failed_logs)} failed logs...")
    client.upload(rule_id=rule_id, stream_name=stream_name, logs=failed_logs)

Ignore Errors

def ignore_errors(error):
    pass  # Silently ignore upload failures

client.upload(
    rule_id=rule_id,
    stream_name=stream_name,
    logs=logs,
    on_error=ignore_errors
)

Async Client

import asyncio
from azure.monitor.ingestion.aio import LogsIngestionClient
from azure.identity.aio import DefaultAzureCredential

async def upload_logs():
    async with LogsIngestionClient(
        endpoint=endpoint,
        credential=DefaultAzureCredential()
    ) as client:
        await client.upload(
            rule_id=rule_id,
            stream_name=stream_name,
            logs=logs
        )

asyncio.run(upload_logs())

Sovereign Clouds

from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
from azure.monitor.ingestion import LogsIngestionClient

# Azure Government
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
with LogsIngestionClient(
    endpoint="https://example.ingest.monitor.azure.us",
    credential=credential,
    credential_scopes=["https://monitor.azure.us/.default"]
) as client:
    # client.upload(...)
    ...

Batching Behavior

The SDK automatically:

  • Splits logs into chunks of 1MB or less

  • Compresses each chunk with gzip

  • Uploads chunks in parallel

No manual batching needed for large log sets.

Client Types

ClientPurpose
LogsIngestionClientSync client for uploading logs
LogsIngestionClient (aio)Async client for uploading logs

Key Concepts

ConceptDescription
DCEData Collection Endpoint — ingestion URL
DCRData Collection Rule — defines schema, transformations, destination
StreamNamed data flow within a DCR
Custom TableTarget table in Log Analytics (ends with _CL)

DCR Stream Name Format

Stream names follow patterns:

  • Custom-<TableName>_CL — For custom tables

  • Microsoft-<TableName> — For built-in tables

Best Practices

  1. Pick sync OR async and stay consistent. Do not mix azure.xxx sync clients with azure.xxx.aio async clients in the same call path. Choose one mode per module.

  2. Always use context managers for clients and async credentials. Wrap every client in with Client(...) as client: (sync) or async with Client(...) as client: (async) to ensure proper cleanup. For async DefaultAzureCredential from azure.identity.aio, also use async with credential: so tokens and transports are cleaned up.

  3. Use DefaultAzureCredential for code that runs locally. Use a specific token credential for code that runs in Azure.

  4. Handle errors gracefully — use on_error callback for partial failures

  5. Include TimeGenerated — Required field for all logs

  6. Match DCR schema — Log fields must match DCR column definitions

  7. Use async client for high-throughput scenarios

  8. Batch uploads — SDK handles batching, but send reasonable chunks

  9. Monitor ingestion — Check Log Analytics for ingestion status

Reference Files

FileContents
references/capabilities.mdAdditional non-hero capabilities, operation-group coverage, and production checklists.
references/non-hero-scenarios.mdDedicated non-hero examples for secondary/advanced scenarios.

Files

references/capabilities.md

azure-monitor-ingestion-py capability coverage

SDK/package: azure-monitor-ingestion

This index maps hero scenarios in SKILL.md and links non-hero scenarios documented in dedicated reference files.

Hero scenarios covered in SKILL.md

  • Upload Custom Logs

  • Upload from JSON File

  • Custom Error Handling

  • Ignore Errors

Non-hero scenarios

Related deep-dive references

API breadth checklist

  • Verify client/auth mode for the environment before coding.

  • Confirm operation-group/method names against current Microsoft Learn API reference.

  • For Python SDKs with both sync and async clients, document both forms without a blanket preference.

  • Include cleanup/delete paths for created resources in examples.

  • Prefer idempotent create/update operations where available.

  • Validate paging/LRO/error-handling patterns for production paths.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key, Needs Python

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Microsoft
Publisher
Microsoft (Official)
Source
microsoft/skills · .github/plugins/azure-sdk-python/skills/azure-monitor-ingestion-py · commit 0696600
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-monitor-ingestion-py.zip