Describe Image - Describe Image

This operation generates a description of an image in human readable language with complete sentences. The description is based on a collection of content tags, which are also returned by the operation. More than one description can be generated for each image. Descriptions are ordered by their confidence score. Descriptions may include results from celebrity and landmark domain models, if applicable. Two input methods are supported -- (1) Uploading an image or (2) specifying an image URL. A successful response will be returned in JSON. If the request failed, the response will contain an error code and a message to help understand what went wrong.

POST {Endpoint}/vision/v3.2/describe
POST {Endpoint}/vision/v3.2/describe?maxCandidates={maxCandidates}&language={language}&descriptionExclude={descriptionExclude}&model-version={model-version}

URI Parameters

Name In Required Type Description
Endpoint
path True

string

Supported Cognitive Services endpoints.

descriptionExclude
query

DescriptionExclude[]

Turn off specified domain models when generating the description.

language
query

string

The desired language for output generation. If this parameter is not specified, the default value is "en". See https://aka.ms/cv-languages for list of supported languages.

maxCandidates
query

integer

int32

Maximum number of candidate descriptions to be returned. The default is 1.

model-version
query

string

Optional parameter to specify the version of the AI model. Accepted values are: "latest", "2021-04-01", "2021-05-01". Defaults to "latest".

Regex pattern: ^(latest|\d{4}-\d{2}-\d{2})(-preview)?$

Request Header

Name Required Type Description
Ocp-Apim-Subscription-Key True

string

Request Body

Name Required Type Description
url True

string

Publicly reachable URL of an image.

Responses

Name Type Description
200 OK

ImageDescription

Image description object.

Other Status Codes

ComputerVisionErrorResponse

Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Successful DescribeImage request

Sample request

POST https://westus.api.cognitive.microsoft.com/vision/v3.2/describe?maxCandidates=1


{
  "url": "{url}"
}

Sample response

{
  "description": {
    "tags": [
      "person",
      "man",
      "outdoor",
      "window",
      "glasses"
    ],
    "captions": [
      {
        "text": "Satya Nadella sitting on a bench",
        "confidence": 0.48293603002174407
      }
    ]
  },
  "requestId": "ed2de1c6-fb55-4686-b0da-4da6e05d283f",
  "metadata": {
    "width": 1500,
    "height": 1000,
    "format": "Jpeg"
  },
  "modelVersion": "2021-04-01"
}

Definitions

Name Description
ComputerVisionError

The API request error.

ComputerVisionErrorCodes

The error code.

ComputerVisionErrorResponse

The API error response.

ComputerVisionInnerError

Details about the API request error.

ComputerVisionInnerErrorCodeValue

The error code.

DescriptionExclude

Turn off specified domain models when generating the description.

ImageCaption

An image caption, i.e. a brief description of what the image depicts.

ImageDescription

A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.

ImageMetadata

Image metadata.

ImageUrl

ComputerVisionError

The API request error.

Name Type Description
code

ComputerVisionErrorCodes

The error code.

innererror

ComputerVisionInnerError

Inner error contains more specific information.

message

string

A message explaining the error reported by the service.

ComputerVisionErrorCodes

The error code.

Name Type Description
InternalServerError

string

InvalidArgument

string

InvalidRequest

string

ServiceUnavailable

string

ComputerVisionErrorResponse

The API error response.

Name Type Description
error

ComputerVisionError

Error contents.

ComputerVisionInnerError

Details about the API request error.

Name Type Description
code

ComputerVisionInnerErrorCodeValue

The error code.

message

string

Error message.

ComputerVisionInnerErrorCodeValue

The error code.

Name Type Description
BadArgument

string

CancelledRequest

string

DetectFaceError

string

FailedToProcess

string

InternalServerError

string

InvalidDetails

string

InvalidImageFormat

string

InvalidImageSize

string

InvalidImageUrl

string

InvalidModel

string

InvalidThumbnailSize

string

NotSupportedFeature

string

NotSupportedImage

string

NotSupportedLanguage

string

NotSupportedVisualFeature

string

StorageException

string

Timeout

string

Unspecified

string

UnsupportedMediaType

string

DescriptionExclude

Turn off specified domain models when generating the description.

Name Type Description
Celebrities

string

Landmarks

string

ImageCaption

An image caption, i.e. a brief description of what the image depicts.

Name Type Description
confidence

number

The level of confidence the service has in the caption.

text

string

The text of the caption.

ImageDescription

A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.

Name Type Description
description.captions

ImageCaption[]

A list of captions, sorted by confidence level.

description.tags

string[]

A collection of image tags.

metadata

ImageMetadata

Image metadata.

modelVersion

string

Version of the AI model.

requestId

string

Id of the REST API request.

ImageMetadata

Image metadata.

Name Type Description
format

string

Image format.

height

integer

Image height, in pixels.

width

integer

Image width, in pixels.

ImageUrl

Name Type Description
url

string

Publicly reachable URL of an image.