Azure AI Vision Imageanalysis Java

MicrosoftOfficialCreated by Microsoft

Skill
Tool integration
Software development
MIT
No issues found

From the author

Build image analysis applications with Azure AI Vision SDK for Java. Use when implementing image captioning, OCR text extraction, object detection, tagging, or smart cropping.

— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a comprehensive guide for using the Azure AI Vision Image Analysis SDK for Java. It covers client initialization, authentication methods including API keys and DefaultAzureCredential, and detailed patterns for performing various computer vision tasks. Developers can use this SDK to extract insights from images, such as generating captions, performing OCR, detecting objects and people, and creating smart thumbnails.

What it does

Generates human-readable captions for images.
Extracts text from images using OCR capabilities.
Detects and locates objects and people within images.
Provides smart cropping regions for image thumbnails.
Supports both synchronous and asynchronous analysis patterns.
Allows for multiple visual features to be analyzed in a single request.

When to use it

When building Java applications that require automated image understanding.
When you need to extract text from documents or images (OCR).
When implementing object or person detection features.
When you need to generate descriptive metadata or tags for image libraries.
When optimizing image display using smart cropping.

Advantages

Native Java support for Azure AI Vision services.
Flexible authentication options including managed identity.
Supports both local file and URL-based image analysis.
Comprehensive feature set for diverse computer vision requirements.
Includes robust error handling patterns for production environments.

Usage examples

“How do I generate a caption for an image using the Azure AI Vision Java SDK?”: The agent provides code to initialize the ImageAnalysisClient and use the analyze method with the VisualFeatures.CAPTION parameter.
“Show me how to perform OCR on a local image file in Java.”: The agent demonstrates loading a file into BinaryData and calling the analyze method with the VisualFeatures.READ feature.
“How can I detect objects in an image and get their bounding boxes?”: The agent provides a code snippet using VisualFeatures.OBJECTS and iterating through the returned DetectedObject list to access bounding box coordinates.
“What is the recommended way to authenticate the Azure AI Vision client in production?”: The agent explains the use of DefaultAzureCredentialBuilder to securely manage credentials.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/microsoft/azure-ai-vision-imageanalysis-java/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip azure-ai-vision-imageanalysis-java.zip -d .agents/skills

Documentation by Microsoft

Azure AI Vision Image Analysis SDK for Java

Build image analysis applications using the Azure AI Vision Image Analysis SDK for Java.

Installation

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-vision-imageanalysis</artifactId>
    <version>1.1.0-beta.1</version>
</dependency>

Client Creation

With API Key

import com.azure.ai.vision.imageanalysis.ImageAnalysisClient;
import com.azure.ai.vision.imageanalysis.ImageAnalysisClientBuilder;
import com.azure.core.credential.KeyCredential;

String endpoint = System.getenv("VISION_ENDPOINT");
String key = System.getenv("VISION_KEY");

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new KeyCredential(key))
    .buildClient();

Async Client

import com.azure.ai.vision.imageanalysis.ImageAnalysisAsyncClient;

ImageAnalysisAsyncClient asyncClient = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new KeyCredential(key))
    .buildAsyncClient();

With DefaultAzureCredential

import com.azure.core.credential.TokenCredential;
import com.azure.identity.AzureIdentityEnvVars;
import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.identity.ManagedIdentityCredentialBuilder;

TokenCredential credential = new DefaultAzureCredentialBuilder()
    .requireEnvVars(AzureIdentityEnvVars.AZURE_TOKEN_CREDENTIALS)
    .build();
// Or use a specific credential directly in production:
// See https://learn.microsoft.com/java/api/overview/azure/identity-readme?view=azure-java-stable#credential-classes
// TokenCredential credential = new ManagedIdentityCredentialBuilder().build();

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(credential)
    .buildClient();

Visual Features

Feature	Description
`CAPTION`	Generate human-readable image description
`DENSE_CAPTIONS`	Captions for up to 10 regions
`READ`	OCR - Extract text from images
`TAGS`	Content tags for objects, scenes, actions
`OBJECTS`	Detect objects with bounding boxes
`SMART_CROPS`	Smart thumbnail regions
`PEOPLE`	Detect people with locations

Core Patterns

Generate Caption

import com.azure.ai.vision.imageanalysis.models.*;
import com.azure.core.util.BinaryData;
import java.io.File;
import java.util.Arrays;

// From file
BinaryData imageData = BinaryData.fromFile(new File("image.jpg").toPath());

ImageAnalysisResult result = client.analyze(
    imageData,
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.printf("Caption: \"%s\" (confidence: %.4f)%n",
    result.getCaption().getText(),
    result.getCaption().getConfidence());

Generate Caption from URL

ImageAnalysisResult result = client.analyzeFromUrl(
    "https://example.com/image.jpg",
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.printf("Caption: \"%s\"%n", result.getCaption().getText());

Extract Text (OCR)

ImageAnalysisResult result = client.analyze(
    BinaryData.fromFile(new File("document.jpg").toPath()),
    Arrays.asList(VisualFeatures.READ),
    null);

for (DetectedTextBlock block : result.getRead().getBlocks()) {
    for (DetectedTextLine line : block.getLines()) {
        System.out.printf("Line: '%s'%n", line.getText());
        System.out.printf("  Bounding polygon: %s%n", line.getBoundingPolygon());
        
        for (DetectedTextWord word : line.getWords()) {
            System.out.printf("  Word: '%s' (confidence: %.4f)%n",
                word.getText(),
                word.getConfidence());
        }
    }
}

Detect Objects

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.OBJECTS),
    null);

for (DetectedObject obj : result.getObjects()) {
    System.out.printf("Object: %s (confidence: %.4f)%n",
        obj.getTags().get(0).getName(),
        obj.getTags().get(0).getConfidence());
    
    ImageBoundingBox box = obj.getBoundingBox();
    System.out.printf("  Location: x=%d, y=%d, w=%d, h=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
}

Get Tags

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.TAGS),
    null);

for (DetectedTag tag : result.getTags()) {
    System.out.printf("Tag: %s (confidence: %.4f)%n",
        tag.getName(),
        tag.getConfidence());
}

Detect People

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.PEOPLE),
    null);

for (DetectedPerson person : result.getPeople()) {
    ImageBoundingBox box = person.getBoundingBox();
    System.out.printf("Person at x=%d, y=%d (confidence: %.4f)%n",
        box.getX(), box.getY(), person.getConfidence());
}

Smart Cropping

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.SMART_CROPS),
    new ImageAnalysisOptions().setSmartCropsAspectRatios(Arrays.asList(1.0, 1.5)));

for (CropRegion crop : result.getSmartCrops()) {
    System.out.printf("Crop region: aspect=%.2f, x=%d, y=%d, w=%d, h=%d%n",
        crop.getAspectRatio(),
        crop.getBoundingBox().getX(),
        crop.getBoundingBox().getY(),
        crop.getBoundingBox().getWidth(),
        crop.getBoundingBox().getHeight());
}

Dense Captions

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.DENSE_CAPTIONS),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

for (DenseCaption caption : result.getDenseCaptions()) {
    System.out.printf("Caption: \"%s\" (confidence: %.4f)%n",
        caption.getText(),
        caption.getConfidence());
    System.out.printf("  Region: x=%d, y=%d, w=%d, h=%d%n",
        caption.getBoundingBox().getX(),
        caption.getBoundingBox().getY(),
        caption.getBoundingBox().getWidth(),
        caption.getBoundingBox().getHeight());
}

Multiple Features

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(
        VisualFeatures.CAPTION,
        VisualFeatures.TAGS,
        VisualFeatures.OBJECTS,
        VisualFeatures.READ),
    new ImageAnalysisOptions()
        .setGenderNeutralCaption(true)
        .setLanguage("en"));

// Access all results
System.out.println("Caption: " + result.getCaption().getText());
System.out.println("Tags: " + result.getTags().size());
System.out.println("Objects: " + result.getObjects().size());
System.out.println("Text blocks: " + result.getRead().getBlocks().size());

Async Analysis

asyncClient.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.CAPTION),
    null)
    .subscribe(
        result -> System.out.println("Caption: " + result.getCaption().getText()),
        error -> System.err.println("Error: " + error.getMessage()),
        () -> System.out.println("Complete")
    );

Error Handling

import com.azure.core.exception.HttpResponseException;

try {
    client.analyzeFromUrl(imageUrl, Arrays.asList(VisualFeatures.CAPTION), null);
} catch (HttpResponseException e) {
    System.out.println("Status: " + e.getResponse().getStatusCode());
    System.out.println("Error: " + e.getMessage());
}

Environment Variables

VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com/ # Required for all auth methods
VISION_KEY=<your-api-key> # Only required for AzureKeyCredential auth
AZURE_TOKEN_CREDENTIALS=prod  # Required only if DefaultAzureCredential is used in production

Image Requirements

Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO
Size: < 20 MB
Dimensions: 50x50 to 16000x16000 pixels

Regional Availability

Caption and Dense Captions require GPU-supported regions. Check supported regions before deployment.

Trigger Phrases

"image analysis Java"
"Azure Vision SDK"
"image captioning"
"OCR image text extraction"
"object detection image"
"smart crop thumbnail"
"detect people image"

Files

ATTRIBUTION.mdAdded by AutoGineer1009 B
LICENSELicence file added by AutoGineer1.1 KiB
SKILL.md8.5 KiB
references/examples.md21.6 KiB

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Microsoft

Publisher

Microsoft (Official)

Source

microsoft/skills · .github/plugins/azure-sdk-java/skills/azure-ai-vision-imageanalysis-java · commit 0696600

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-ai-vision-imageanalysis-java.zip