Image categorization

In addition to tags and a description, Image Analysis 3.2 can return the taxonomy-based categories detected in an image. Unlike tags, categories are organized in a parent/child hierarchy, and there are fewer of them (86, as opposed to thousands of tags). All category names are in English. Categorization can be done by itself or alongside the newer tags model.

The 86-category taxonomy

Azure AI Vision can categorize an image broadly or specifically, using the list of 86 categories in the following diagram. For the full taxonomy in text format, see Category Taxonomy.

Image categorization examples

The following JSON response illustrates what Azure AI Vision returns when categorizing the example image based on its visual features.

{
    "categories": [
        {
            "name": "people_",
            "score": 0.81640625
        }
    ],
    "requestId": "bae7f76a-1cc7-4479-8d29-48a694974705",
    "metadata": {
        "height": 200,
        "width": 300,
        "format": "Jpeg"
    }
}

The following table illustrates a typical image set and the category returned by Azure AI Vision for each image.

Image	Category
	people_group
	animal_dog
	outdoor_mountain
	food_bread

Use the API

The categorization feature is part of the Analyze Image 3.2 API. You can call this API through a native SDK or through REST calls. Include Categories in the visualFeatures query parameter. Then, when you get the full JSON response, simply parse the string for the contents of the "categories" section.

• Quickstart: Vision REST API or client libraries

Next steps

Learn the related concepts of tagging images and describing images.