AutoGineer
Generated banner for Azure AI Vision Imageanalysis Java
Generated logo for Azure AI Vision Imageanalysis Java

Azure AI Vision Imageanalysis Java

MicrosoftOfficialCreated by Microsoft

  • Skill
  • Tool integration
  • Software development
  • MIT
  • No issues found
Download ZIP

From the author

Build image analysis applications with Azure AI Vision SDK for Java. Use when implementing image captioning, OCR text extraction, object detection, tagging, or smart cropping.
— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a comprehensive guide for using the Azure AI Vision Image Analysis SDK for Java. It covers client initialization, authentication methods including API keys and DefaultAzureCredential, and detailed patterns for performing various computer vision tasks. Developers can use this SDK to extract insights from images, such as generating captions, performing OCR, detecting objects and people, and creating smart thumbnails.

What it does

  • Generates human-readable captions for images.
  • Extracts text from images using OCR capabilities.
  • Detects and locates objects and people within images.
  • Provides smart cropping regions for image thumbnails.
  • Supports both synchronous and asynchronous analysis patterns.
  • Allows for multiple visual features to be analyzed in a single request.

When to use it

  • When building Java applications that require automated image understanding.
  • When you need to extract text from documents or images (OCR).
  • When implementing object or person detection features.
  • When you need to generate descriptive metadata or tags for image libraries.
  • When optimizing image display using smart cropping.

Advantages

  • Native Java support for Azure AI Vision services.
  • Flexible authentication options including managed identity.
  • Supports both local file and URL-based image analysis.
  • Comprehensive feature set for diverse computer vision requirements.
  • Includes robust error handling patterns for production environments.

Usage examples

“How do I generate a caption for an image using the Azure AI Vision Java SDK?”
The agent provides code to initialize the ImageAnalysisClient and use the analyze method with the VisualFeatures.CAPTION parameter.
“Show me how to perform OCR on a local image file in Java.”
The agent demonstrates loading a file into BinaryData and calling the analyze method with the VisualFeatures.READ feature.
“How can I detect objects in an image and get their bounding boxes?”
The agent provides a code snippet using VisualFeatures.OBJECTS and iterating through the returned DetectedObject list to access bounding box coordinates.
“What is the recommended way to authenticate the Azure AI Vision client in production?”
The agent explains the use of DefaultAzureCredentialBuilder to securely manage credentials.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/microsoft/azure-ai-vision-imageanalysis-java/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip azure-ai-vision-imageanalysis-java.zip -d .agents/skills

Documentation by Microsoft

Azure AI Vision Image Analysis SDK for Java

Build image analysis applications using the Azure AI Vision Image Analysis SDK for Java.

Installation

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-vision-imageanalysis</artifactId>
    <version>1.1.0-beta.1</version>
</dependency>

Client Creation

With API Key
import com.azure.ai.vision.imageanalysis.ImageAnalysisClient;
import com.azure.ai.vision.imageanalysis.ImageAnalysisClientBuilder;
import com.azure.core.credential.KeyCredential;

String endpoint = System.getenv("VISION_ENDPOINT");
String key = System.getenv("VISION_KEY");

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new KeyCredential(key))
    .buildClient();
Async Client
import com.azure.ai.vision.imageanalysis.ImageAnalysisAsyncClient;

ImageAnalysisAsyncClient asyncClient = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new KeyCredential(key))
    .buildAsyncClient();
With DefaultAzureCredential
import com.azure.core.credential.TokenCredential;
import com.azure.identity.AzureIdentityEnvVars;
import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.identity.ManagedIdentityCredentialBuilder;

TokenCredential credential = new DefaultAzureCredentialBuilder()
    .requireEnvVars(AzureIdentityEnvVars.AZURE_TOKEN_CREDENTIALS)
    .build();
// Or use a specific credential directly in production:
// See https://learn.microsoft.com/java/api/overview/azure/identity-readme?view=azure-java-stable#credential-classes
// TokenCredential credential = new ManagedIdentityCredentialBuilder().build();

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(credential)
    .buildClient();

Visual Features

FeatureDescription
CAPTIONGenerate human-readable image description
DENSE_CAPTIONSCaptions for up to 10 regions
READOCR - Extract text from images
TAGSContent tags for objects, scenes, actions
OBJECTSDetect objects with bounding boxes
SMART_CROPSSmart thumbnail regions
PEOPLEDetect people with locations

Core Patterns

Generate Caption
import com.azure.ai.vision.imageanalysis.models.*;
import com.azure.core.util.BinaryData;
import java.io.File;
import java.util.Arrays;

// From file
BinaryData imageData = BinaryData.fromFile(new File("image.jpg").toPath());

ImageAnalysisResult result = client.analyze(
    imageData,
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.printf("Caption: \"%s\" (confidence: %.4f)%n",
    result.getCaption().getText(),
    result.getCaption().getConfidence());
Generate Caption from URL
ImageAnalysisResult result = client.analyzeFromUrl(
    "https://example.com/image.jpg",
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.printf("Caption: \"%s\"%n", result.getCaption().getText());
Extract Text (OCR)
ImageAnalysisResult result = client.analyze(
    BinaryData.fromFile(new File("document.jpg").toPath()),
    Arrays.asList(VisualFeatures.READ),
    null);

for (DetectedTextBlock block : result.getRead().getBlocks()) {
    for (DetectedTextLine line : block.getLines()) {
        System.out.printf("Line: '%s'%n", line.getText());
        System.out.printf("  Bounding polygon: %s%n", line.getBoundingPolygon());
        
        for (DetectedTextWord word : line.getWords()) {
            System.out.printf("  Word: '%s' (confidence: %.4f)%n",
                word.getText(),
                word.getConfidence());
        }
    }
}
Detect Objects
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.OBJECTS),
    null);

for (DetectedObject obj : result.getObjects()) {
    System.out.printf("Object: %s (confidence: %.4f)%n",
        obj.getTags().get(0).getName(),
        obj.getTags().get(0).getConfidence());
    
    ImageBoundingBox box = obj.getBoundingBox();
    System.out.printf("  Location: x=%d, y=%d, w=%d, h=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
}
Get Tags
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.TAGS),
    null);

for (DetectedTag tag : result.getTags()) {
    System.out.printf("Tag: %s (confidence: %.4f)%n",
        tag.getName(),
        tag.getConfidence());
}
Detect People
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.PEOPLE),
    null);

for (DetectedPerson person : result.getPeople()) {
    ImageBoundingBox box = person.getBoundingBox();
    System.out.printf("Person at x=%d, y=%d (confidence: %.4f)%n",
        box.getX(), box.getY(), person.getConfidence());
}
Smart Cropping
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.SMART_CROPS),
    new ImageAnalysisOptions().setSmartCropsAspectRatios(Arrays.asList(1.0, 1.5)));

for (CropRegion crop : result.getSmartCrops()) {
    System.out.printf("Crop region: aspect=%.2f, x=%d, y=%d, w=%d, h=%d%n",
        crop.getAspectRatio(),
        crop.getBoundingBox().getX(),
        crop.getBoundingBox().getY(),
        crop.getBoundingBox().getWidth(),
        crop.getBoundingBox().getHeight());
}
Dense Captions
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.DENSE_CAPTIONS),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

for (DenseCaption caption : result.getDenseCaptions()) {
    System.out.printf("Caption: \"%s\" (confidence: %.4f)%n",
        caption.getText(),
        caption.getConfidence());
    System.out.printf("  Region: x=%d, y=%d, w=%d, h=%d%n",
        caption.getBoundingBox().getX(),
        caption.getBoundingBox().getY(),
        caption.getBoundingBox().getWidth(),
        caption.getBoundingBox().getHeight());
}
Multiple Features
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(
        VisualFeatures.CAPTION,
        VisualFeatures.TAGS,
        VisualFeatures.OBJECTS,
        VisualFeatures.READ),
    new ImageAnalysisOptions()
        .setGenderNeutralCaption(true)
        .setLanguage("en"));

// Access all results
System.out.println("Caption: " + result.getCaption().getText());
System.out.println("Tags: " + result.getTags().size());
System.out.println("Objects: " + result.getObjects().size());
System.out.println("Text blocks: " + result.getRead().getBlocks().size());
Async Analysis
asyncClient.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.CAPTION),
    null)
    .subscribe(
        result -> System.out.println("Caption: " + result.getCaption().getText()),
        error -> System.err.println("Error: " + error.getMessage()),
        () -> System.out.println("Complete")
    );

Error Handling

import com.azure.core.exception.HttpResponseException;

try {
    client.analyzeFromUrl(imageUrl, Arrays.asList(VisualFeatures.CAPTION), null);
} catch (HttpResponseException e) {
    System.out.println("Status: " + e.getResponse().getStatusCode());
    System.out.println("Error: " + e.getMessage());
}

Environment Variables

VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com/ # Required for all auth methods
VISION_KEY=<your-api-key> # Only required for AzureKeyCredential auth
AZURE_TOKEN_CREDENTIALS=prod  # Required only if DefaultAzureCredential is used in production

Image Requirements

  • Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO

  • Size: < 20 MB

  • Dimensions: 50x50 to 16000x16000 pixels

Regional Availability

Caption and Dense Captions require GPU-supported regions. Check supported regions before deployment.

Trigger Phrases

  • "image analysis Java"

  • "Azure Vision SDK"

  • "image captioning"

  • "OCR image text extraction"

  • "object detection image"

  • "smart crop thumbnail"

  • "detect people image"

Files

references/examples.md

Azure AI Vision Image Analysis Java SDK - Examples

Comprehensive code examples for the Azure AI Vision Image Analysis SDK for Java.

Table of Contents

  • Maven Dependency

  • Client Creation

  • Visual Features

  • Generate Caption

  • Extract Text (OCR)

  • Detect Objects

  • Get Tags

  • Detect People

  • Smart Cropping

  • Dense Captions

  • Multiple Features

  • Async Patterns

  • Error Handling

  • Complete Application Example

Maven Dependency

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-vision-imageanalysis</artifactId>
    <version>1.1.0-beta.1</version>
</dependency>

<!-- For DefaultAzureCredential -->
<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-identity</artifactId>
    <version>1.14.2</version>
</dependency>

Client Creation

With API Key
import com.azure.ai.vision.imageanalysis.ImageAnalysisClient;
import com.azure.ai.vision.imageanalysis.ImageAnalysisClientBuilder;
import com.azure.core.credential.KeyCredential;

String endpoint = System.getenv("VISION_ENDPOINT");
String key = System.getenv("VISION_KEY");

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new KeyCredential(key))
    .buildClient();
With DefaultAzureCredential (Recommended)
import com.azure.identity.DefaultAzureCredentialBuilder;

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();
Async Client
import com.azure.ai.vision.imageanalysis.ImageAnalysisAsyncClient;

ImageAnalysisAsyncClient asyncClient = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildAsyncClient();

Visual Features

FeatureDescription
CAPTIONGenerate human-readable image description
DENSE_CAPTIONSCaptions for up to 10 regions
READOCR - Extract text from images
TAGSContent tags for objects, scenes, actions
OBJECTSDetect objects with bounding boxes
SMART_CROPSSmart thumbnail regions
PEOPLEDetect people with locations

Generate Caption

From File
import com.azure.ai.vision.imageanalysis.models.*;
import com.azure.core.util.BinaryData;
import java.io.File;
import java.util.Arrays;

// Load image from file
File imageFile = new File("photo.jpg");
BinaryData imageData = BinaryData.fromFile(imageFile.toPath());

// Analyze with caption
ImageAnalysisResult result = client.analyze(
    imageData,
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

// Get caption
CaptionResult caption = result.getCaption();
System.out.printf("Caption: \"%s\" (confidence: %.4f)%n",
    caption.getText(),
    caption.getConfidence());
From URL
String imageUrl = "https://example.com/photo.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.printf("Caption: \"%s\"%n", result.getCaption().getText());
With Language
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.CAPTION),
    new ImageAnalysisOptions()
        .setGenderNeutralCaption(true)
        .setLanguage("en"));  // Supported: en, es, fr, de, it, pt, ja, ko, zh

Extract Text (OCR)

File documentImage = new File("document.jpg");
BinaryData imageData = BinaryData.fromFile(documentImage.toPath());

ImageAnalysisResult result = client.analyze(
    imageData,
    Arrays.asList(VisualFeatures.READ),
    null);

ReadResult readResult = result.getRead();

System.out.println("=== Extracted Text ===");
for (DetectedTextBlock block : readResult.getBlocks()) {
    System.out.println("Block:");
    
    for (DetectedTextLine line : block.getLines()) {
        System.out.printf("  Line: '%s'%n", line.getText());
        
        // Get bounding polygon
        List<ImagePoint> polygon = line.getBoundingPolygon();
        System.out.printf("    Bounding polygon: [");
        for (ImagePoint point : polygon) {
            System.out.printf("(%d,%d) ", point.getX(), point.getY());
        }
        System.out.println("]");
        
        // Get individual words
        for (DetectedTextWord word : line.getWords()) {
            System.out.printf("    Word: '%s' (confidence: %.4f)%n",
                word.getText(),
                word.getConfidence());
        }
    }
}
Extract Text from URL
String documentUrl = "https://example.com/receipt.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    documentUrl,
    Arrays.asList(VisualFeatures.READ),
    null);

// Collect all text
StringBuilder fullText = new StringBuilder();
for (DetectedTextBlock block : result.getRead().getBlocks()) {
    for (DetectedTextLine line : block.getLines()) {
        fullText.append(line.getText()).append("\n");
    }
}

System.out.println("Full extracted text:");
System.out.println(fullText.toString());

Detect Objects

String imageUrl = "https://example.com/street-scene.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.OBJECTS),
    null);

System.out.println("=== Detected Objects ===");
for (DetectedObject obj : result.getObjects()) {
    // Get the primary tag (highest confidence)
    DetectedTag primaryTag = obj.getTags().get(0);
    
    System.out.printf("Object: %s (confidence: %.4f)%n",
        primaryTag.getName(),
        primaryTag.getConfidence());
    
    // Get bounding box
    ImageBoundingBox box = obj.getBoundingBox();
    System.out.printf("  Location: x=%d, y=%d, width=%d, height=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
    
    // Additional tags for this object
    if (obj.getTags().size() > 1) {
        System.out.println("  Additional tags:");
        for (int i = 1; i < obj.getTags().size(); i++) {
            DetectedTag tag = obj.getTags().get(i);
            System.out.printf("    - %s (%.4f)%n", tag.getName(), tag.getConfidence());
        }
    }
}

System.out.printf("Total objects detected: %d%n", result.getObjects().size());

Get Tags

String imageUrl = "https://example.com/nature.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.TAGS),
    null);

System.out.println("=== Image Tags ===");

// Sort by confidence
List<DetectedTag> sortedTags = new ArrayList<>(result.getTags());
sortedTags.sort((a, b) -> Double.compare(b.getConfidence(), a.getConfidence()));

for (DetectedTag tag : sortedTags) {
    System.out.printf("%-20s (confidence: %.4f)%n",
        tag.getName(),
        tag.getConfidence());
}

// Filter high-confidence tags (>80%)
System.out.println("\nHigh-confidence tags (>80%):");
for (DetectedTag tag : sortedTags) {
    if (tag.getConfidence() > 0.80) {
        System.out.println("  - " + tag.getName());
    }
}

Detect People

String imageUrl = "https://example.com/group-photo.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.PEOPLE),
    null);

System.out.println("=== Detected People ===");
System.out.printf("Number of people: %d%n", result.getPeople().size());

int personIndex = 1;
for (DetectedPerson person : result.getPeople()) {
    ImageBoundingBox box = person.getBoundingBox();
    
    System.out.printf("Person %d:%n", personIndex++);
    System.out.printf("  Confidence: %.4f%n", person.getConfidence());
    System.out.printf("  Location: x=%d, y=%d, width=%d, height=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
    
    // Calculate center point
    int centerX = box.getX() + box.getWidth() / 2;
    int centerY = box.getY() + box.getHeight() / 2;
    System.out.printf("  Center: (%d, %d)%n", centerX, centerY);
}

Smart Cropping

String imageUrl = "https://example.com/landscape.jpg";

// Request crops with specific aspect ratios
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.SMART_CROPS),
    new ImageAnalysisOptions()
        .setSmartCropsAspectRatios(Arrays.asList(1.0, 1.5, 0.75)));  // 1:1, 3:2, 3:4

System.out.println("=== Smart Crop Regions ===");
for (CropRegion crop : result.getSmartCrops()) {
    ImageBoundingBox box = crop.getBoundingBox();
    
    System.out.printf("Aspect ratio: %.2f%n", crop.getAspectRatio());
    System.out.printf("  Region: x=%d, y=%d, width=%d, height=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
}
Use Smart Crops for Thumbnails
// Get square thumbnail region
ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.SMART_CROPS),
    new ImageAnalysisOptions()
        .setSmartCropsAspectRatios(Arrays.asList(1.0)));  // Square

CropRegion squareCrop = result.getSmartCrops().get(0);
ImageBoundingBox box = squareCrop.getBoundingBox();

// Use these coordinates to crop your image
System.out.printf("Thumbnail region: x=%d, y=%d, size=%dx%d%n",
    box.getX(), box.getY(), box.getWidth(), box.getHeight());

Dense Captions

String imageUrl = "https://example.com/complex-scene.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.DENSE_CAPTIONS),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.println("=== Dense Captions ===");
int regionIndex = 1;
for (DenseCaption caption : result.getDenseCaptions()) {
    ImageBoundingBox box = caption.getBoundingBox();
    
    System.out.printf("Region %d:%n", regionIndex++);
    System.out.printf("  Caption: \"%s\"%n", caption.getText());
    System.out.printf("  Confidence: %.4f%n", caption.getConfidence());
    System.out.printf("  Location: x=%d, y=%d, width=%d, height=%d%n",
        box.getX(), box.getY(), box.getWidth(), box.getHeight());
}

Multiple Features

Analyze with multiple features in a single request.

String imageUrl = "https://example.com/photo.jpg";

ImageAnalysisResult result = client.analyzeFromUrl(
    imageUrl,
    Arrays.asList(
        VisualFeatures.CAPTION,
        VisualFeatures.TAGS,
        VisualFeatures.OBJECTS,
        VisualFeatures.PEOPLE,
        VisualFeatures.READ),
    new ImageAnalysisOptions()
        .setGenderNeutralCaption(true)
        .setLanguage("en"));

// Caption
System.out.println("=== Caption ===");
System.out.printf("\"%s\" (%.4f)%n",
    result.getCaption().getText(),
    result.getCaption().getConfidence());

// Tags
System.out.println("\n=== Tags ===");
for (DetectedTag tag : result.getTags()) {
    if (tag.getConfidence() > 0.7) {
        System.out.printf("  %s (%.2f)%n", tag.getName(), tag.getConfidence());
    }
}

// Objects
System.out.println("\n=== Objects ===");
System.out.printf("  Count: %d%n", result.getObjects().size());
for (DetectedObject obj : result.getObjects()) {
    System.out.printf("  - %s%n", obj.getTags().get(0).getName());
}

// People
System.out.println("\n=== People ===");
System.out.printf("  Count: %d%n", result.getPeople().size());

// Text
System.out.println("\n=== Text ===");
int lineCount = 0;
for (DetectedTextBlock block : result.getRead().getBlocks()) {
    lineCount += block.getLines().size();
}
System.out.printf("  Lines of text: %d%n", lineCount);

// Metadata
System.out.println("\n=== Image Metadata ===");
System.out.printf("  Dimensions: %d x %d%n",
    result.getMetadata().getWidth(),
    result.getMetadata().getHeight());
System.out.printf("  Model: %s%n", result.getModelVersion());

Async Patterns

Basic Async Analysis
ImageAnalysisAsyncClient asyncClient = new ImageAnalysisClientBuilder()
    .endpoint(endpoint)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildAsyncClient();

String imageUrl = "https://example.com/photo.jpg";

asyncClient.analyzeFromUrl(
    imageUrl,
    Arrays.asList(VisualFeatures.CAPTION, VisualFeatures.TAGS),
    new ImageAnalysisOptions().setGenderNeutralCaption(true))
    .subscribe(
        result -> {
            System.out.println("Caption: " + result.getCaption().getText());
            System.out.println("Tags: " + result.getTags().size());
        },
        error -> System.err.println("Error: " + error.getMessage()),
        () -> System.out.println("Analysis complete")
    );

// Keep application running
Thread.sleep(10000);
Parallel Analysis
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

List<String> imageUrls = Arrays.asList(
    "https://example.com/image1.jpg",
    "https://example.com/image2.jpg",
    "https://example.com/image3.jpg"
);

Flux.fromIterable(imageUrls)
    .flatMap(url -> asyncClient.analyzeFromUrl(
        url,
        Arrays.asList(VisualFeatures.CAPTION),
        null)
        .map(result -> new ImageResult(url, result.getCaption().getText())))
    .subscribe(
        imageResult -> System.out.printf("%s: %s%n", 
            imageResult.url, imageResult.caption),
        error -> System.err.println("Error: " + error.getMessage())
    );

// Helper class
class ImageResult {
    String url;
    String caption;
    
    ImageResult(String url, String caption) {
        this.url = url;
        this.caption = caption;
    }
}

Error Handling

import com.azure.core.exception.HttpResponseException;

try {
    ImageAnalysisResult result = client.analyzeFromUrl(
        "invalid-url",
        Arrays.asList(VisualFeatures.CAPTION),
        null);
        
} catch (HttpResponseException e) {
    int statusCode = e.getResponse().getStatusCode();
    System.err.println("HTTP Status: " + statusCode);
    System.err.println("Error: " + e.getMessage());
    
    switch (statusCode) {
        case 400:
            System.err.println("Bad request - check image URL or format");
            break;
        case 401:
            System.err.println("Unauthorized - check API key");
            break;
        case 404:
            System.err.println("Resource not found");
            break;
        case 415:
            System.err.println("Unsupported media type - check image format");
            break;
        case 429:
            System.err.println("Rate limited - retry with backoff");
            break;
        default:
            System.err.println("Unexpected error");
    }
} catch (Exception e) {
    System.err.println("Unexpected error: " + e.getMessage());
}

Complete Application Example

import com.azure.ai.vision.imageanalysis.ImageAnalysisClient;
import com.azure.ai.vision.imageanalysis.ImageAnalysisClientBuilder;
import com.azure.ai.vision.imageanalysis.models.*;
import com.azure.core.util.BinaryData;
import com.azure.identity.DefaultAzureCredentialBuilder;

import java.io.File;
import java.util.*;

public class ImageAnalyzer {
    
    private final ImageAnalysisClient client;
    
    public ImageAnalyzer() {
        this.client = new ImageAnalysisClientBuilder()
            .endpoint(System.getenv("VISION_ENDPOINT"))
            .credential(new DefaultAzureCredentialBuilder().build())
            .buildClient();
    }
    
    public ImageAnalysisReport analyzeImage(String imagePath) {
        File imageFile = new File(imagePath);
        BinaryData imageData = BinaryData.fromFile(imageFile.toPath());
        
        ImageAnalysisResult result = client.analyze(
            imageData,
            Arrays.asList(
                VisualFeatures.CAPTION,
                VisualFeatures.TAGS,
                VisualFeatures.OBJECTS,
                VisualFeatures.PEOPLE,
                VisualFeatures.READ),
            new ImageAnalysisOptions()
                .setGenderNeutralCaption(true)
                .setLanguage("en"));
        
        return buildReport(imagePath, result);
    }
    
    public ImageAnalysisReport analyzeImageUrl(String imageUrl) {
        ImageAnalysisResult result = client.analyzeFromUrl(
            imageUrl,
            Arrays.asList(
                VisualFeatures.CAPTION,
                VisualFeatures.TAGS,
                VisualFeatures.OBJECTS,
                VisualFeatures.PEOPLE,
                VisualFeatures.READ),
            new ImageAnalysisOptions()
                .setGenderNeutralCaption(true)
                .setLanguage("en"));
        
        return buildReport(imageUrl, result);
    }
    
    private ImageAnalysisReport buildReport(String source, ImageAnalysisResult result) {
        // Extract caption
        String caption = result.getCaption().getText();
        double captionConfidence = result.getCaption().getConfidence();
        
        // Extract high-confidence tags
        List<String> tags = new ArrayList<>();
        for (DetectedTag tag : result.getTags()) {
            if (tag.getConfidence() > 0.7) {
                tags.add(tag.getName());
            }
        }
        
        // Extract objects
        List<String> objects = new ArrayList<>();
        for (DetectedObject obj : result.getObjects()) {
            objects.add(obj.getTags().get(0).getName());
        }
        
        // Count people
        int peopleCount = result.getPeople().size();
        
        // Extract text
        StringBuilder extractedText = new StringBuilder();
        for (DetectedTextBlock block : result.getRead().getBlocks()) {
            for (DetectedTextLine line : block.getLines()) {
                extractedText.append(line.getText()).append("\n");
            }
        }
        
        return new ImageAnalysisReport(
            source,
            caption,
            captionConfidence,
            tags,
            objects,
            peopleCount,
            extractedText.toString().trim(),
            result.getMetadata().getWidth(),
            result.getMetadata().getHeight()
        );
    }
    
    // Report class
    public static class ImageAnalysisReport {
        public final String source;
        public final String caption;
        public final double captionConfidence;
        public final List<String> tags;
        public final List<String> objects;
        public final int peopleCount;
        public final String extractedText;
        public final int width;
        public final int height;
        
        public ImageAnalysisReport(String source, String caption, double captionConfidence,
                                   List<String> tags, List<String> objects, int peopleCount,
                                   String extractedText, int width, int height) {
            this.source = source;
            this.caption = caption;
            this.captionConfidence = captionConfidence;
            this.tags = tags;
            this.objects = objects;
            this.peopleCount = peopleCount;
            this.extractedText = extractedText;
            this.width = width;
            this.height = height;
        }
        
        @Override
        public String toString() {
            StringBuilder sb = new StringBuilder();
            sb.append("=== Image Analysis Report ===\n");
            sb.append(String.format("Source: %s\n", source));
            sb.append(String.format("Dimensions: %dx%d\n", width, height));
            sb.append(String.format("Caption: \"%s\" (%.2f%%)\n", caption, captionConfidence * 100));
            sb.append(String.format("Tags: %s\n", String.join(", ", tags)));
            sb.append(String.format("Objects: %s\n", String.join(", ", objects)));
            sb.append(String.format("People detected: %d\n", peopleCount));
            if (!extractedText.isEmpty()) {
                sb.append(String.format("Extracted text:\n%s\n", extractedText));
            }
            return sb.toString();
        }
    }
    
    public static void main(String[] args) {
        ImageAnalyzer analyzer = new ImageAnalyzer();
        
        // Analyze from URL
        String imageUrl = "https://raw.githubusercontent.com/Azure-Samples/cognitive-services-sample-data-files/master/ComputerVision/Images/landmark.jpg";
        
        try {
            ImageAnalysisReport report = analyzer.analyzeImageUrl(imageUrl);
            System.out.println(report);
        } catch (Exception e) {
            System.err.println("Analysis failed: " + e.getMessage());
        }
    }
}

Environment Variables

VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com/
VISION_KEY=<your-api-key>

# For DefaultAzureCredential
AZURE_CLIENT_ID=<service-principal-client-id>
AZURE_CLIENT_SECRET=<service-principal-secret>
AZURE_TENANT_ID=<tenant-id>

Image Requirements

  • Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO

  • Size: Less than 20 MB

  • Dimensions: 50x50 to 16000x16000 pixels

Regional Availability

Caption and Dense Captions require GPU-supported regions. Check supported regions before deployment.

Best Practices

  1. Use DefaultAzureCredential — Prefer managed identity over API keys

  2. Combine features — Request multiple features in one call for efficiency

  3. Check confidence scores — Filter results based on confidence thresholds

  4. Use async for batches — Process multiple images in parallel

  5. Handle regional limits — Caption features require specific regions

  6. Optimize image size — Resize large images before sending

  7. Enable gender-neutral captions — Use inclusive language in captions

  8. Implement retry logic — Handle rate limiting with exponential backoff

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Microsoft
Publisher
Microsoft (Official)
Source
microsoft/skills · .github/plugins/azure-sdk-java/skills/azure-ai-vision-imageanalysis-java · commit 0696600
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-ai-vision-imageanalysis-java.zip