Describe Image - Describe Image

Reference

Service:: Cognitive Services - Computer Vision

API Version:: 3.2

This operation generates a description of an image in human readable language with complete sentences. The description is based on a collection of content tags, which are also returned by the operation. More than one description can be generated for each image. Descriptions are ordered by their confidence score. Descriptions may include results from celebrity and landmark domain models, if applicable. Two input methods are supported -- (1) Uploading an image or (2) specifying an image URL. A successful response will be returned in JSON. If the request failed, the response will contain an error code and a message to help understand what went wrong.

POST {Endpoint}/vision/v3.2/describe

With optional parameters:

POST {Endpoint}/vision/v3.2/describe?maxCandidates={maxCandidates}&language={language}&descriptionExclude={descriptionExclude}&model-version={model-version}

URI Parameters

Name	In	Required	Type	Description
Endpoint	path	True	string	Supported Cognitive Services endpoints.
descriptionExclude	query		DescriptionExclude[]	Turn off specified domain models when generating the description.
language	query		string	The desired language for output generation. If this parameter is not specified, the default value is "en". See https://aka.ms/cv-languages for list of supported languages.
maxCandidates	query		integer int32	Maximum number of candidate descriptions to be returned. The default is 1.
model-version	query		string	Optional parameter to specify the version of the AI model. Accepted values are: "latest", "2021-04-01", "2021-05-01". Defaults to "latest". Regex pattern: `^(latest\|\d{4}-\d{2}-\d{2})(-preview)?$`

Request Header

Name	Required	Type	Description
Ocp-Apim-Subscription-Key	True	string

Request Body

Name	Required	Type	Description
url	True	string	Publicly reachable URL of an image.

Responses

Name	Type	Description
200 OK	ImageDescription	Image description object.
Other Status Codes	ComputerVisionErrorResponse	Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Successful DescribeImage request

Sample Request

HTTP

POST https://westus.api.cognitive.microsoft.com/vision/v3.2/describe?maxCandidates=1


{
  "url": "{url}"
}

Sample Response

Status code:: 200

{
  "description": {
    "tags": [
      "person",
      "man",
      "outdoor",
      "window",
      "glasses"
    ],
    "captions": [
      {
        "text": "Satya Nadella sitting on a bench",
        "confidence": 0.48293603002174407
      }
    ]
  },
  "requestId": "ed2de1c6-fb55-4686-b0da-4da6e05d283f",
  "metadata": {
    "width": 1500,
    "height": 1000,
    "format": "Jpeg"
  },
  "modelVersion": "2021-04-01"
}

Definitions

Name	Description
ComputerVisionError	The API request error.
ComputerVisionErrorCodes	The error code.
ComputerVisionErrorResponse	The API error response.
ComputerVisionInnerError	Details about the API request error.
ComputerVisionInnerErrorCodeValue	The error code.
DescriptionExclude	Turn off specified domain models when generating the description.
ImageCaption	An image caption, i.e. a brief description of what the image depicts.
ImageDescription	A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.
ImageMetadata	Image metadata.
ImageUrl

ComputerVisionError

The API request error.

Name	Type	Description
code	ComputerVisionErrorCodes	The error code.
innererror	ComputerVisionInnerError	Inner error contains more specific information.
message	string	A message explaining the error reported by the service.

ComputerVisionErrorCodes

The error code.

Name	Type	Description
InternalServerError	string
InvalidArgument	string
InvalidRequest	string
ServiceUnavailable	string

ComputerVisionErrorResponse

The API error response.

Name	Type	Description
error	ComputerVisionError	Error contents.

ComputerVisionInnerError

Details about the API request error.

Name	Type	Description
code	ComputerVisionInnerErrorCodeValue	The error code.
message	string	Error message.

ComputerVisionInnerErrorCodeValue

The error code.

Name	Type	Description
BadArgument	string
CancelledRequest	string
DetectFaceError	string
FailedToProcess	string
InternalServerError	string
InvalidDetails	string
InvalidImageFormat	string
InvalidImageSize	string
InvalidImageUrl	string
InvalidModel	string
InvalidThumbnailSize	string
NotSupportedFeature	string
NotSupportedImage	string
NotSupportedLanguage	string
NotSupportedVisualFeature	string
StorageException	string
Timeout	string
Unspecified	string
UnsupportedMediaType	string

DescriptionExclude

Turn off specified domain models when generating the description.

Name	Type	Description
Celebrities	string
Landmarks	string

ImageCaption

An image caption, i.e. a brief description of what the image depicts.

Name	Type	Description
confidence	number	The level of confidence the service has in the caption.
text	string	The text of the caption.

ImageDescription

A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.

Name	Type	Description
description.captions	ImageCaption[]	A list of captions, sorted by confidence level.
description.tags	string[]	A collection of image tags.
metadata	ImageMetadata	Image metadata.
modelVersion	string	Version of the AI model.
requestId	string	Id of the REST API request.

ImageMetadata

Image metadata.

Name	Type	Description
format	string	Image format.
height	integer	Image height, in pixels.
width	integer	Image width, in pixels.

ImageUrl

Name	Type	Description
url	string	Publicly reachable URL of an image.

Share via