Generated banner for Azure AI Vision Imageanalysis Py
Generated logo for Azure AI Vision Imageanalysis Py

Azure AI Vision Imageanalysis Py

MicrosoftOfficialCreated by Microsoft

  • Skill
  • Tool integration
  • Data and AI
  • MIT
  • Low risk

From the author

Azure AI Vision Image Analysis SDK for captions, tags, objects, OCR, people detection, and smart cropping. Use for computer vision and image understanding tasks. Triggers: "image analysis", "computer vision", "OCR", "object detection", "ImageAnalysisClient", "image caption".
— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The azure-ai-vision-imageanalysis-py skill provides a Python client library for interacting with Azure AI Vision 4.0. It enables developers to perform advanced image understanding tasks, including generating image captions, identifying tags, detecting objects and people, performing OCR, and calculating smart crop regions. The library supports both synchronous and asynchronous operations and emphasizes secure authentication using DefaultAzureCredential.

What it does

  • Generates descriptive captions for images and specific regions.
  • Extracts text from images using OCR capabilities.
  • Detects and localizes objects and people within images.
  • Provides smart cropping suggestions for various aspect ratios.
  • Identifies content tags for scenes and actions.
  • Supports both synchronous and asynchronous client implementations.

When to use it

  • When integrating computer vision features into Python applications.
  • When you need to extract text or identify objects from image files or URLs.
  • When building automated image processing pipelines requiring metadata generation.
  • When implementing thumbnail generation with smart cropping logic.

Advantages

  • Supports secure, modern authentication via DefaultAzureCredential.
  • Provides both sync and async client options for flexible integration.
  • Comprehensive feature set covering common computer vision tasks.
  • Includes robust error handling for API interactions.

Usage examples

“How do I generate a caption for an image using the Azure AI Vision SDK?”
The agent provides a code snippet using ImageAnalysisClient with the VisualFeatures.CAPTION flag to retrieve a descriptive text string for a provided image URL.
“Show me how to perform OCR on an image.”
The agent demonstrates using the VisualFeatures.READ flag to extract text blocks, lines, and individual words along with their bounding polygons.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/microsoft/azure-ai-vision-imageanalysis-py/download
  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip azure-ai-vision-imageanalysis-py.zip -d .agents/skills

Documentation by Microsoft

Azure AI Vision Image Analysis SDK for Python

Client library for Azure AI Vision 4.0 image analysis including captions, tags, objects, OCR, and more.

Installation

pip install azure-ai-vision-imageanalysis

Environment Variables

VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com  # Required for all auth methods
AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production
VISION_KEY=<your-api-key>  # Only required for the legacy API-key auth path below

Authentication & Lifecycle

🔑 Two rules apply to every code sample below:

  1. Prefer DefaultAzureCredential. It works locally (Azure CLI / VS Code / Developer CLI) and in Azure (managed identity, workload identity) with no code change. Avoid connection strings, account/API keys — they bypass Entra audit and rotation.

    • Local dev: DefaultAzureCredential works as-is.

    • Production: set AZURE_TOKEN_CREDENTIALS=prod (or AZURE_TOKEN_CREDENTIALS=<specific_credential>) to constrain the credential chain to production-safe credentials.

  2. Wrap every client in a context manager so HTTP transports, sockets, and token caches are released deterministically:

    • Sync: with <Client>(...) as client:

    • Async: async with <Client>(...) as client: and async with DefaultAzureCredential() as credential: (from azure.identity.aio)

Snippets may abbreviate this setup, but production code should always follow both rules.

import os
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures

# Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
credential = DefaultAzureCredential(require_envvar=True)
# Or use a specific credential directly in production:
# See https://learn.microsoft.com/python/api/overview/azure/identity-readme?view=azure-python#credential-classes
# credential = ManagedIdentityCredential()

with ImageAnalysisClient(
    endpoint=os.environ["VISION_ENDPOINT"],
    credential=credential,
) as client:
    result = client.analyze_from_url(
        image_url="https://aka.ms/azsdk/image-analysis/sample.jpg",
        visual_features=[VisualFeatures.CAPTION],
    )
Legacy: API Key (existing keyed deployments)

New code should use DefaultAzureCredential above. Use AzureKeyCredential only if you have an existing keyed deployment that hasn't been migrated to Entra ID yet — for example, regulated environments still completing their Entra rollout.

import os
from azure.core.credentials import AzureKeyCredential
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures

with ImageAnalysisClient(
    endpoint=os.environ["VISION_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["VISION_KEY"]),
) as client:
    result = client.analyze_from_url(
        image_url="https://aka.ms/azsdk/image-analysis/sample.jpg",
        visual_features=[VisualFeatures.CAPTION],
    )

Analyze Image from URL

from azure.ai.vision.imageanalysis.models import VisualFeatures

image_url = "https://example.com/image.jpg"

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[
        VisualFeatures.CAPTION,
        VisualFeatures.TAGS,
        VisualFeatures.OBJECTS,
        VisualFeatures.READ,
        VisualFeatures.PEOPLE,
        VisualFeatures.SMART_CROPS,
        VisualFeatures.DENSE_CAPTIONS
    ],
    gender_neutral_caption=True,
    language="en"
)

Analyze Image from File

with open("image.jpg", "rb") as f:
    image_data = f.read()

result = client.analyze(
    image_data=image_data,
    visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS]
)

Image Caption

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.CAPTION],
    gender_neutral_caption=True
)

if result.caption:
    print(f"Caption: {result.caption.text}")
    print(f"Confidence: {result.caption.confidence:.2f}")

Dense Captions (Multiple Regions)

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.DENSE_CAPTIONS]
)

if result.dense_captions:
    for caption in result.dense_captions.list:
        print(f"Caption: {caption.text}")
        print(f"  Confidence: {caption.confidence:.2f}")
        print(f"  Bounding box: {caption.bounding_box}")

Tags

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.TAGS]
)

if result.tags:
    for tag in result.tags.list:
        print(f"Tag: {tag.name} (confidence: {tag.confidence:.2f})")

Object Detection

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.OBJECTS]
)

if result.objects:
    for obj in result.objects.list:
        print(f"Object: {obj.tags[0].name}")
        print(f"  Confidence: {obj.tags[0].confidence:.2f}")
        box = obj.bounding_box
        print(f"  Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")

OCR (Text Extraction)

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.READ]
)

if result.read:
    for block in result.read.blocks:
        for line in block.lines:
            print(f"Line: {line.text}")
            print(f"  Bounding polygon: {line.bounding_polygon}")
            
            # Word-level details
            for word in line.words:
                print(f"  Word: {word.text} (confidence: {word.confidence:.2f})")

People Detection

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.PEOPLE]
)

if result.people:
    for person in result.people.list:
        print(f"Person detected:")
        print(f"  Confidence: {person.confidence:.2f}")
        box = person.bounding_box
        print(f"  Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")

Smart Cropping

result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.SMART_CROPS],
    smart_crops_aspect_ratios=[0.9, 1.33, 1.78]  # Portrait, 4:3, 16:9
)

if result.smart_crops:
    for crop in result.smart_crops.list:
        print(f"Aspect ratio: {crop.aspect_ratio}")
        box = crop.bounding_box
        print(f"  Crop region: x={box.x}, y={box.y}, w={box.width}, h={box.height}")

Async Client

from azure.ai.vision.imageanalysis.aio import ImageAnalysisClient
from azure.identity.aio import DefaultAzureCredential

async def analyze_image():
    async with DefaultAzureCredential() as credential:
        async with ImageAnalysisClient(
            endpoint=endpoint,
            credential=credential
        ) as client:
            result = await client.analyze_from_url(
                image_url=image_url,
                visual_features=[VisualFeatures.CAPTION]
            )
            print(result.caption.text)

Visual Features

FeatureDescription
CAPTIONSingle sentence describing the image
DENSE_CAPTIONSCaptions for multiple regions
TAGSContent tags (objects, scenes, actions)
OBJECTSObject detection with bounding boxes
READOCR text extraction
PEOPLEPeople detection with bounding boxes
SMART_CROPSSuggested crop regions for thumbnails

Error Handling

from azure.core.exceptions import HttpResponseError

try:
    result = client.analyze_from_url(
        image_url=image_url,
        visual_features=[VisualFeatures.CAPTION]
    )
except HttpResponseError as e:
    print(f"Status code: {e.status_code}")
    print(f"Reason: {e.reason}")
    print(f"Message: {e.error.message}")

Image Requirements

  • Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO

  • Max size: 20 MB

  • Dimensions: 50x50 to 16000x16000 pixels

Best Practices

  1. Pick sync OR async and stay consistent. Do not mix azure.ai.vision.imageanalysis sync clients with azure.ai.vision.imageanalysis.aio async clients in the same call path. Choose one mode per module.

  2. Always use context managers for clients and async credentials. Wrap every client in with ImageAnalysisClient(...) as client: (sync) or async with ImageAnalysisClient(...) as client: (async). For async DefaultAzureCredential from azure.identity.aio, also use async with credential: so tokens and transports are cleaned up.

  3. Select only needed features to optimize latency and cost

  4. Use async client for high-throughput scenarios

  5. Handle HttpResponseError for invalid images or auth issues

  6. Enable gender_neutral_caption for inclusive descriptions

  7. Specify language for localized captions

  8. Use smart_crops_aspect_ratios matching your thumbnail requirements

  9. Cache results when analyzing the same image multiple times

Reference Files

FileContents
references/capabilities.mdAdditional non-hero capabilities, operation-group coverage, and production checklists.
references/non-hero-scenarios.mdDedicated non-hero examples for secondary/advanced scenarios.

Files

references/capabilities.md

azure-ai-vision-imageanalysis-py capability coverage

SDK/package: azure-ai-vision-imageanalysis

This index maps hero scenarios in SKILL.md and links non-hero scenarios documented in dedicated reference files.

Hero scenarios covered in SKILL.md

  • Analyze Image from URL

  • Analyze Image from File

  • Image Caption

  • Dense Captions (Multiple Regions)

Non-hero scenarios

API breadth checklist

  • Verify client/auth mode for the environment before coding.

  • Confirm operation-group/method names against current Microsoft Learn API reference.

  • For Python SDKs with both sync and async clients, document both forms without a blanket preference.

  • Include cleanup/delete paths for created resources in examples.

  • Prefer idempotent create/update operations where available.

  • Validate paging/LRO/error-handling patterns for production paths.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key, Needs Python

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Microsoft
Publisher
Microsoft (Official)
Source
microsoft/skills · .github/plugins/azure-sdk-python/skills/azure-ai-vision-imageanalysis-py · commit 0696600
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-ai-vision-imageanalysis-py.zip