Azure AI Vision Imageanalysis Py
MicrosoftOfficialCreated by Microsoft
- Skill
- Tool integration
- Data and AI
- MIT
- Low risk
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
The azure-ai-vision-imageanalysis-py skill provides a Python client library for interacting with Azure AI Vision 4.0. It enables developers to perform advanced image understanding tasks, including generating image captions, identifying tags, detecting objects and people, performing OCR, and calculating smart crop regions. The library supports both synchronous and asynchronous operations and emphasizes secure authentication using DefaultAzureCredential.
What it does
- Generates descriptive captions for images and specific regions.
- Extracts text from images using OCR capabilities.
- Detects and localizes objects and people within images.
- Provides smart cropping suggestions for various aspect ratios.
- Identifies content tags for scenes and actions.
- Supports both synchronous and asynchronous client implementations.
When to use it
- When integrating computer vision features into Python applications.
- When you need to extract text or identify objects from image files or URLs.
- When building automated image processing pipelines requiring metadata generation.
- When implementing thumbnail generation with smart cropping logic.
Advantages
- Supports secure, modern authentication via DefaultAzureCredential.
- Provides both sync and async client options for flexible integration.
- Comprehensive feature set covering common computer vision tasks.
- Includes robust error handling for API interactions.
Usage examples
- “How do I generate a caption for an image using the Azure AI Vision SDK?”
- The agent provides a code snippet using ImageAnalysisClient with the VisualFeatures.CAPTION flag to retrieve a descriptive text string for a provided image URL.
- “Show me how to perform OCR on an image.”
- The agent demonstrates using the VisualFeatures.READ flag to extract text blocks, lines, and individual words along with their bounding polygons.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/microsoft/azure-ai-vision-imageanalysis-py/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip azure-ai-vision-imageanalysis-py.zip -d .agents/skills
Documentation by Microsoft
Azure AI Vision Image Analysis SDK for Python
Client library for Azure AI Vision 4.0 image analysis including captions, tags, objects, OCR, and more.
Installation
pip install azure-ai-vision-imageanalysisEnvironment Variables
VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com # Required for all auth methods
AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production
VISION_KEY=<your-api-key> # Only required for the legacy API-key auth path belowAuthentication & Lifecycle
🔑 Two rules apply to every code sample below:
Prefer
DefaultAzureCredential. It works locally (Azure CLI / VS Code / Developer CLI) and in Azure (managed identity, workload identity) with no code change. Avoid connection strings, account/API keys — they bypass Entra audit and rotation.
Local dev:
DefaultAzureCredentialworks as-is.Production: set
AZURE_TOKEN_CREDENTIALS=prod(orAZURE_TOKEN_CREDENTIALS=<specific_credential>) to constrain the credential chain to production-safe credentials.Wrap every client in a context manager so HTTP transports, sockets, and token caches are released deterministically:
Sync:
with <Client>(...) as client:Async:
async with <Client>(...) as client:andasync with DefaultAzureCredential() as credential:(fromazure.identity.aio)Snippets may abbreviate this setup, but production code should always follow both rules.
import os
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
# Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
credential = DefaultAzureCredential(require_envvar=True)
# Or use a specific credential directly in production:
# See https://learn.microsoft.com/python/api/overview/azure/identity-readme?view=azure-python#credential-classes
# credential = ManagedIdentityCredential()
with ImageAnalysisClient(
endpoint=os.environ["VISION_ENDPOINT"],
credential=credential,
) as client:
result = client.analyze_from_url(
image_url="https://aka.ms/azsdk/image-analysis/sample.jpg",
visual_features=[VisualFeatures.CAPTION],
)Legacy: API Key (existing keyed deployments)
New code should use DefaultAzureCredential above. Use AzureKeyCredential only if you have an existing keyed deployment that hasn't been migrated to Entra ID yet — for example, regulated environments still completing their Entra rollout.
import os
from azure.core.credentials import AzureKeyCredential
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
with ImageAnalysisClient(
endpoint=os.environ["VISION_ENDPOINT"],
credential=AzureKeyCredential(os.environ["VISION_KEY"]),
) as client:
result = client.analyze_from_url(
image_url="https://aka.ms/azsdk/image-analysis/sample.jpg",
visual_features=[VisualFeatures.CAPTION],
)Analyze Image from URL
from azure.ai.vision.imageanalysis.models import VisualFeatures
image_url = "https://example.com/image.jpg"
result = client.analyze_from_url(
image_url=image_url,
visual_features=[
VisualFeatures.CAPTION,
VisualFeatures.TAGS,
VisualFeatures.OBJECTS,
VisualFeatures.READ,
VisualFeatures.PEOPLE,
VisualFeatures.SMART_CROPS,
VisualFeatures.DENSE_CAPTIONS
],
gender_neutral_caption=True,
language="en"
)Analyze Image from File
with open("image.jpg", "rb") as f:
image_data = f.read()
result = client.analyze(
image_data=image_data,
visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS]
)Image Caption
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.CAPTION],
gender_neutral_caption=True
)
if result.caption:
print(f"Caption: {result.caption.text}")
print(f"Confidence: {result.caption.confidence:.2f}")Dense Captions (Multiple Regions)
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.DENSE_CAPTIONS]
)
if result.dense_captions:
for caption in result.dense_captions.list:
print(f"Caption: {caption.text}")
print(f" Confidence: {caption.confidence:.2f}")
print(f" Bounding box: {caption.bounding_box}")Tags
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.TAGS]
)
if result.tags:
for tag in result.tags.list:
print(f"Tag: {tag.name} (confidence: {tag.confidence:.2f})")Object Detection
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.OBJECTS]
)
if result.objects:
for obj in result.objects.list:
print(f"Object: {obj.tags[0].name}")
print(f" Confidence: {obj.tags[0].confidence:.2f}")
box = obj.bounding_box
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")OCR (Text Extraction)
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.READ]
)
if result.read:
for block in result.read.blocks:
for line in block.lines:
print(f"Line: {line.text}")
print(f" Bounding polygon: {line.bounding_polygon}")
# Word-level details
for word in line.words:
print(f" Word: {word.text} (confidence: {word.confidence:.2f})")People Detection
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.PEOPLE]
)
if result.people:
for person in result.people.list:
print(f"Person detected:")
print(f" Confidence: {person.confidence:.2f}")
box = person.bounding_box
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")Smart Cropping
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.SMART_CROPS],
smart_crops_aspect_ratios=[0.9, 1.33, 1.78] # Portrait, 4:3, 16:9
)
if result.smart_crops:
for crop in result.smart_crops.list:
print(f"Aspect ratio: {crop.aspect_ratio}")
box = crop.bounding_box
print(f" Crop region: x={box.x}, y={box.y}, w={box.width}, h={box.height}")Async Client
from azure.ai.vision.imageanalysis.aio import ImageAnalysisClient
from azure.identity.aio import DefaultAzureCredential
async def analyze_image():
async with DefaultAzureCredential() as credential:
async with ImageAnalysisClient(
endpoint=endpoint,
credential=credential
) as client:
result = await client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.CAPTION]
)
print(result.caption.text)Visual Features
| Feature | Description |
|---|---|
CAPTION | Single sentence describing the image |
DENSE_CAPTIONS | Captions for multiple regions |
TAGS | Content tags (objects, scenes, actions) |
OBJECTS | Object detection with bounding boxes |
READ | OCR text extraction |
PEOPLE | People detection with bounding boxes |
SMART_CROPS | Suggested crop regions for thumbnails |
Error Handling
from azure.core.exceptions import HttpResponseError
try:
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.CAPTION]
)
except HttpResponseError as e:
print(f"Status code: {e.status_code}")
print(f"Reason: {e.reason}")
print(f"Message: {e.error.message}")Image Requirements
Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO
Max size: 20 MB
Dimensions: 50x50 to 16000x16000 pixels
Best Practices
Pick sync OR async and stay consistent. Do not mix
azure.ai.vision.imageanalysissync clients withazure.ai.vision.imageanalysis.aioasync clients in the same call path. Choose one mode per module.Always use context managers for clients and async credentials. Wrap every client in
with ImageAnalysisClient(...) as client:(sync) orasync with ImageAnalysisClient(...) as client:(async). For asyncDefaultAzureCredentialfromazure.identity.aio, also useasync with credential:so tokens and transports are cleaned up.Select only needed features to optimize latency and cost
Use async client for high-throughput scenarios
Handle HttpResponseError for invalid images or auth issues
Enable gender_neutral_caption for inclusive descriptions
Specify language for localized captions
Use smart_crops_aspect_ratios matching your thumbnail requirements
Cache results when analyzing the same image multiple times
Reference Files
| File | Contents |
|---|---|
| references/capabilities.md | Additional non-hero capabilities, operation-group coverage, and production checklists. |
| references/non-hero-scenarios.md | Dedicated non-hero examples for secondary/advanced scenarios. |
Files
- ATTRIBUTION.mdAdded by AutoGineer1007 B
- LICENSELicence file added by AutoGineer1.1 KiB
- SKILL.md9.7 KiB
- references/capabilities.md2.3 KiB
- references/non-hero-scenarios.md3.7 KiB
references/non-hero-scenarios.md
azure-ai-vision-imageanalysis-py non-hero scenarios
These scenarios are intentionally separate from hero flows in SKILL.md.
They cover secondary/advanced patterns typically used after the primary end-to-end path is working.
Tags
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.TAGS]
)
if result.tags:
for tag in result.tags.list:
print(f"Tag: {tag.name} (confidence: {tag.confidence:.2f})")Object Detection
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.OBJECTS]
)
if result.objects:
for obj in result.objects.list:
print(f"Object: {obj.tags[0].name}")
print(f" Confidence: {obj.tags[0].confidence:.2f}")
box = obj.bounding_box
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")OCR (Text Extraction)
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.READ]
)
if result.read:
for block in result.read.blocks:
for line in block.lines:
print(f"Line: {line.text}")
print(f" Bounding polygon: {line.bounding_polygon}")
# Word-level details
for word in line.words:
print(f" Word: {word.text} (confidence: {word.confidence:.2f})")People Detection
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.PEOPLE]
)
if result.people:
for person in result.people.list:
print(f"Person detected:")
print(f" Confidence: {person.confidence:.2f}")
box = person.bounding_box
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")Smart Cropping
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.SMART_CROPS],
smart_crops_aspect_ratios=[0.9, 1.33, 1.78] # Portrait, 4:3, 16:9
)
if result.smart_crops:
for crop in result.smart_crops.list:
print(f"Aspect ratio: {crop.aspect_ratio}")
box = crop.bounding_box
print(f" Crop region: x={box.x}, y={box.y}, w={box.width}, h={box.height}")Async Client
from azure.ai.vision.imageanalysis.aio import ImageAnalysisClient
from azure.identity.aio import DefaultAzureCredential
async def analyze_image():
async with DefaultAzureCredential() as credential:
async with ImageAnalysisClient(
endpoint=endpoint,
credential=credential
) as client:
result = await client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.CAPTION]
)
print(result.caption.text)Visual Features
| Feature | Description |
|---|---|
CAPTION | Single sentence describing the image |
DENSE_CAPTIONS | Captions for multiple regions |
TAGS | Content tags (objects, scenes, actions) |
OBJECTS | Object detection with bounding boxes |
READ | OCR text extraction |
PEOPLE | People detection with bounding boxes |
SMART_CROPS | Suggested crop regions for thumbnails |
Error Handling
from azure.core.exceptions import HttpResponseError
try:
result = client.analyze_from_url(
image_url=image_url,
visual_features=[VisualFeatures.CAPTION]
)
except HttpResponseError as e:
print(f"Status code: {e.status_code}")
print(f"Reason: {e.reason}")
print(f"Message: {e.error.message}")Image Requirements
Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO
Max size: 20 MB
Dimensions: 50x50 to 16000x16000 pixels
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Needs an API key, Needs Python
Security
What our scanner noticed· Low risk
- Instructions that use the network
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- Microsoft
- Publisher
- Microsoft (Official)
- Source
- microsoft/skills · .github/plugins/azure-sdk-python/skills/azure-ai-vision-imageanalysis-py · commit 0696600
- Licence
- MIT License (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by Microsoft.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-ai-vision-imageanalysis-py.zip