Edit

Image classification on Azure

Foundry SDK
Azure Blob Storage
Azure Cosmos DB
Azure Event Grid
Azure Functions

Solution ideas

This article describes a solution idea. Your cloud architect can use this guidance to help visualize the major components for a typical implementation of this architecture. Use this article as a starting point to design a well-architected solution that aligns with your workload's specific requirements.

By using Azure services such as Azure Content Understanding and Azure Functions, you can add image classification and metadata extraction to a web or mobile application without managing servers or training your own models. This solution idea targets image classification and tagging. If you have other AI needs, see the broader Microsoft Foundry and Foundry Tools catalogs.

Architecture

Diagram that shows an intelligent image-processing pipeline that uses Azure services.

Download a Visio file of this architecture.

Data flow

This scenario covers the back-end components of a web or mobile application. The following data flow corresponds to the previous image:

  1. A user uploads an image to Azure Blob Storage, either directly or by using a web or mobile application. The upload triggers an event in Azure Event Grid.

  2. Event Grid sends a notification to Azure Functions to process the uploaded image.

  3. The function generates a time-limited, least-privilege shared access signature URL scoped to the target blob and passes it to Content Understanding. Content Understanding uses this URL to access the image directly from Blob Storage, then analyzes it by using a prebuilt analyzer.

  4. The function stores the structured output that Content Understanding returns, along with image metadata, in Azure Cosmos DB for NoSQL.

  5. A web or mobile application receives the results. This data flow returns the classification output and metadata, but not the original image file.

Components

  • Content Understanding is a Foundry Tool that uses generative AI to extract user-defined structured output from documents, images, video, and audio. In this architecture, Content Understanding analyzes each uploaded image by using a prebuilt analyzer that defines the categories, attributes, and labels that you want returned, such as product type, color, or defect class. The output is JSON that maps directly to your application's data model.

  • Azure Functions is a serverless compute platform. In this architecture, Azure Functions provides the back-end API and the event-processing layer for uploaded images. The function orchestrates the workflow. It calls Content Understanding, processes the response, and writes the result to the database. This architecture uses the Flex Consumption plan to support virtual network integration, instance memory choice, and fast scaling.

  • Azure Event Grid is a managed event-routing service that uses a publish-subscribe model. In this architecture, an Event Grid system topic on the storage account emits a Microsoft.Storage.BlobCreated event when a new image is uploaded and delivers it to the function.

  • Azure Blob Storage is an object store for unstructured data. In this architecture, it stores all uploaded images and any static assets that the web application serves. Blob Storage is the source of truth for incoming images.

  • Azure Cosmos DB for NoSQL is a managed NoSQL database. In this architecture, it stores the metadata for each image, including the structured output that Content Understanding returns.

Alternatives

  • AutoML in Azure Machine Learning supports computer vision tasks. You can train custom image classification and object detection models from your labeled data by using classic machine learning techniques. Choose AutoML when you have a labeled dataset and need a deterministic, deployable model for narrow domains where generative approaches don't fit. Examples include manufacturing defect detection or medical imaging. Microsoft recommends AutoML for customers migrating from Azure AI Custom Vision who want to keep a classic machine learning model.

  • Vision-enabled models in Foundry let you call or fine-tune multimodal models (GPT-4.1, GPT-4o, and Phi-4 multimodal) directly. Choose this path when you need fine-grained control over the prompt and model, want to fine-tune on your own data, or need visual question answering and image-grounded chat instead of structured extraction.

  • Azure AI Search indexes the metadata so that users can query and filter images by tag, caption, or other attributes. The AI enrichment skillset can call vision and generative AI services and write the results directly to a search index without a separate function.

  • Azure Logic Apps is a fit when you don't need real-time reaction to uploads. A workflow that runs on a recurring or sliding-window trigger can poll for new blobs and call Content Understanding in batch.

  • Azure Document Intelligence extracts images that are embedded in documents by using the layout model, so you can run downstream classification on embedded figures. Use custom classification models when input files contain multiple document types and you need to identify each one before further processing.

Scenario details

This scenario applies to businesses that process images at scale and want to attach structured metadata such as tags, captions, or category labels to each image without training and operating their own models.

Typical applications include classifying images on a fashion site, analyzing photos for insurance claims, and extracting context from game screenshots. Building this capability in-house traditionally requires expertise in computer vision, training data, and model lifecycle management. The architecture in this article replaces that work with managed Azure services.

Potential use cases

This solution applies to retail, e-commerce, gaming, finance, and insurance. Common use cases include:

  • Tagging images on a retail or fashion site. Sellers upload product photos. Content Understanding returns the tags, captions, and attributes that you define in the analyzer. The platform uses the returned metadata to autofill listing fields, drive visual search, and reduce manual tagging effort.

  • Categorizing products in an e-commerce catalog. A Content Understanding analyzer assigns category and subcategory metadata, such as footwear and running shoe, and visual attributes such as color and material. Buyers get more accurate search and filtering, and sellers spend less time correcting categories.

  • Classifying telemetry from game screenshots. Streaming platforms misclassify a stream when a creator forgets to update the title after switching games. A function that classifies periodic screenshots can detect the change and update the stream metadata. For narrow domains where generative classification underperforms, use AutoML for Images to train a deterministic classifier.

  • Routing insurance claim photos. Content Understanding identifies vehicle damage, natural-disaster damage, or property type from claim photos. The metadata routes the claim to the correct adjuster queue and shortens triage time.

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that you can use to improve the quality of a workload. For more information, see Well-Architected Framework.

Security

Security provides protections against deliberate attacks and the misuse of your valuable data and systems. For more information, see Design review checklist for Security.

  • Use managed identities for the function app to authenticate to Blob Storage, Azure Cosmos DB, and the Microsoft Foundry resource that hosts Content Understanding. Avoid storing connection strings or API keys in app settings.

  • Restrict the Foundry resource and Cosmos DB to private endpoints and turn off public network access when the workload runs inside a virtual network. The Flex Consumption plan supports virtual network integration.

  • Validate uploaded images before you invoke the vision service. Enforce content-type and size limits at the upload boundary, scan for malware, and store uploads in a container that public users can't read directly.

  • This architecture is only suitable for images that you decide are appropriate for a cloud solution to process. Local or offline image processing isn't supported.

Cost Optimization

Cost Optimization focuses on ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Design review checklist for Cost Optimization.

  • Limit the analyzer in Content Understanding to the fields that the application actually consumes. Each extra field increases token usage and per-call cost. For the current rates, see Foundry pricing.

  • For Azure Functions, use the Flex Consumption plan to handle spikes in event-driven workloads. The plan scales to zero and bills per second on active instances.

  • For Cosmos DB, evaluate serverless or autoscale throughput when traffic is uneven. Serverless suits low-traffic and dev/test workloads, while autoscale suits production with variable load.

Operational Excellence

Operational Excellence covers the operations processes that deploy an application and keep it running in production. For more information, see Design review checklist for Operational Excellence.

  • Send Functions, Event Grid, and Foundry diagnostics to a shared Log Analytics workspace and use Application Insights for distributed tracing across the upload-to-result flow.

  • Configure an Event Grid dead-letter destination so that events the function can't process land in a separate blob container for replay.

  • Version Content Understanding analyzer schemas as code and deploy them through the same pipeline that deploys the function. Treat schema changes as breaking changes for downstream consumers.

Contributors

Microsoft maintains this article. The following contributors wrote this article.

Principal author:

Other contributors:

To see nonpublic LinkedIn profiles, sign in to LinkedIn.

Next steps

For guided learning paths, see: