Describe Image - Describe Image

Service:

Azure AI Services

API Version:

3.2

This operation generates a description of an image in human readable language with complete sentences. The description is based on a collection of content tags, which are also returned by the operation. More than one description can be generated for each image. Descriptions are ordered by their confidence score. Descriptions may include results from celebrity and landmark domain models, if applicable. Two input methods are supported -- (1) Uploading an image or (2) specifying an image URL. A successful response will be returned in JSON. If the request failed, the response will contain an error code and a message to help understand what went wrong.

POST {Endpoint}/vision/v3.2/describe

With optional parameters:

POST {Endpoint}/vision/v3.2/describe?maxCandidates={maxCandidates}&language={language}&descriptionExclude={descriptionExclude}&model-version={model-version}

URI Parameters

Name	In	Required	Type	Description
Endpoint	path	True	string	Supported Cognitive Services endpoints.
descriptionExclude	query		DescriptionExclude[]	Turn off specified domain models when generating the description.
language	query		string	The desired language for output generation. If this parameter is not specified, the default value is "en". See https://aka.ms/cv-languages for list of supported languages.
maxCandidates	query		integer (int32)	Maximum number of candidate descriptions to be returned. The default is 1.
model-version	query		string pattern: ^(latest|\d{4}-\d{2}-\d{2})(-preview)?$	Optional parameter to specify the version of the AI model. Accepted values are: "latest", "2021-04-01", "2021-05-01". Defaults to "latest".

Request Header

Name	Required	Type	Description
Ocp-Apim-Subscription-Key	True	string

Request Body

Name	Required	Type	Description
url	True	string	Publicly reachable URL of an image.

Responses

Name	Type	Description
200 OK	ImageDescription	Image description object.
Other Status Codes	ComputerVisionErrorResponse	Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Successful DescribeImage request

Sample request

HTTP

POST https://westus.api.cognitive.microsoft.com/vision/v3.2/describe?maxCandidates=1


{
  "url": "{url}"
}

Sample response

Status code:

200

{
  "description": {
    "tags": [
      "person",
      "man",
      "outdoor",
      "window",
      "glasses"
    ],
    "captions": [
      {
        "text": "Satya Nadella sitting on a bench",
        "confidence": 0.48293603002174407
      }
    ]
  },
  "requestId": "ed2de1c6-fb55-4686-b0da-4da6e05d283f",
  "metadata": {
    "width": 1500,
    "height": 1000,
    "format": "Jpeg"
  },
  "modelVersion": "2021-04-01"
}

Definitions

Name	Description
ComputerVisionError	The API request error.
ComputerVisionErrorCodes	The error code.
ComputerVisionErrorResponse	The API error response.
ComputerVisionInnerError	Details about the API request error.
ComputerVisionInnerErrorCodeValue	The error code.
DescriptionExclude	Turn off specified domain models when generating the description.
ImageCaption	An image caption, i.e. a brief description of what the image depicts.
ImageDescription	A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.
ImageMetadata	Image metadata.
ImageUrl

ComputerVisionError

Object

The API request error.

Name	Type	Description
code	ComputerVisionErrorCodes	The error code.
innererror	ComputerVisionInnerError	Inner error contains more specific information.
message	string	A message explaining the error reported by the service.

ComputerVisionErrorCodes

Enumeration

The error code.

Value	Description
InvalidRequest
InvalidArgument
InternalServerError
ServiceUnavailable

ComputerVisionErrorResponse

Object

The API error response.

Name	Type	Description
error	ComputerVisionError	Error contents.

ComputerVisionInnerError

Object

Details about the API request error.

Name	Type	Description
code	ComputerVisionInnerErrorCodeValue	The error code.
message	string	Error message.

ComputerVisionInnerErrorCodeValue

Enumeration

The error code.

Value	Description
InvalidImageFormat
UnsupportedMediaType
InvalidImageUrl
NotSupportedFeature
NotSupportedImage
Timeout
InternalServerError
InvalidImageSize
BadArgument
DetectFaceError
NotSupportedLanguage
InvalidThumbnailSize
InvalidDetails
InvalidModel
CancelledRequest
NotSupportedVisualFeature
FailedToProcess
Unspecified
StorageException

DescriptionExclude

Enumeration

Turn off specified domain models when generating the description.

Value	Description
Celebrities
Landmarks

ImageCaption

Object

An image caption, i.e. a brief description of what the image depicts.

Name	Type	Description
confidence	number (double)	The level of confidence the service has in the caption.
text	string	The text of the caption.

ImageDescription

Object

A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.

Name	Type	Description
description.captions	ImageCaption[]	A list of captions, sorted by confidence level.
description.tags	string[]	A collection of image tags.
metadata	ImageMetadata	Image metadata.
modelVersion	string pattern: ^(latest|\d{4}-\d{2}-\d{2})(-preview)?$	Version of the AI model.
requestId	string	Id of the REST API request.

ImageMetadata

Object

Image metadata.

Name	Type	Description
format	string	Image format.
height	integer (int32)	Image height, in pixels.
width	integer (int32)	Image width, in pixels.

ImageUrl

Object

Name	Type	Description
url	string	Publicly reachable URL of an image.