Describe Image - Describe Image

Service:

Cognitive Services - Computer Vision

API Version:

3.1

This operation generates a description of an image in human readable language with complete sentences. The description is based on a collection of content tags, which are also returned by the operation. More than one description can be generated for each image. Descriptions are ordered by their confidence score. Descriptions may include results from celebrity and landmark domain models, if applicable. Two input methods are supported -- (1) Uploading an image or (2) specifying an image URL. A successful response will be returned in JSON. If the request failed, the response will contain an error code and a message to help understand what went wrong.

POST {Endpoint}/vision/v3.1/describe

With optional parameters:

POST {Endpoint}/vision/v3.1/describe?maxCandidates={maxCandidates}&language={language}&descriptionExclude={descriptionExclude}

URI Parameters

Name	In	Required	Type	Description
Endpoint	path	True	string	Supported Cognitive Services endpoints.
descriptionExclude	query		DescriptionExclude[]	Turn off specified domain models when generating the description.
language	query		string	The desired language for output generation. If this parameter is not specified, the default value is "en".Supported languages:en - English, Default. es - Spanish, ja - Japanese, pt - Portuguese, zh - Simplified Chinese.
maxCandidates	query		integer int32	Maximum number of candidate descriptions to be returned. The default is 1.

Request Header

Name	Required	Type	Description
Ocp-Apim-Subscription-Key	True	string

Request Body

Name	Required	Type	Description
url	True	string	Publicly reachable URL of an image.

Responses

Name	Type	Description
200 OK	ImageDescription	Image description object.
Other Status Codes	ComputerVisionError	Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Successful DescribeImage request

Sample Request

HTTP

POST https://westus.api.cognitive.microsoft.com/vision/v3.1/describe?maxCandidates=1


{
  "url": "{url}"
}

Sample Response

Status code:

200

{
  "description": {
    "tags": [
      "person",
      "man",
      "outdoor",
      "window",
      "glasses"
    ],
    "captions": [
      {
        "text": "Satya Nadella sitting on a bench",
        "confidence": 0.48293603002174407
      },
      {
        "text": "Satya Nadella is sitting on a bench",
        "confidence": 0.4003700681542283
      },
      {
        "text": "Satya Nadella sitting in front of a building",
        "confidence": 0.38035155997373377
      }
    ]
  },
  "requestId": "ed2de1c6-fb55-4686-b0da-4da6e05d283f",
  "metadata": {
    "width": 1500,
    "height": 1000,
    "format": "Jpeg"
  }
}

Definitions

Name	Description
ComputerVisionError	Details about the API request error.
ComputerVisionErrorCodes	The error code.
DescriptionExclude	Turn off specified domain models when generating the description.
ImageCaption	An image caption, i.e. a brief description of what the image depicts.
ImageDescription	A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.
ImageMetadata	Image metadata.
ImageUrl

ComputerVisionError

Details about the API request error.

Name	Type	Description
code	ComputerVisionErrorCodes	The error code.
message	string	A message explaining the error reported by the service.
requestId	string	A unique request identifier.

ComputerVisionErrorCodes

The error code.

Name	Type	Description
BadArgument	string
CancelledRequest	string
DetectFaceError	string
FailedToProcess	string
InternalServerError	string
InvalidDetails	string
InvalidImageFormat	string
InvalidImageSize	string
InvalidImageUrl	string
InvalidModel	string
InvalidThumbnailSize	string
NotSupportedFeature	string
NotSupportedImage	string
NotSupportedLanguage	string
NotSupportedVisualFeature	string
StorageException	string
Timeout	string
Unspecified	string
UnsupportedMediaType	string

DescriptionExclude

Turn off specified domain models when generating the description.

Name	Type	Description
Celebrities	string
Landmarks	string

ImageCaption

An image caption, i.e. a brief description of what the image depicts.

Name	Type	Description
confidence	number	The level of confidence the service has in the caption.
text	string	The text of the caption.

ImageDescription

A collection of content tags, along with a list of captions sorted by confidence level, and image metadata.

Name	Type	Description
description.captions	ImageCaption[]	A list of captions, sorted by confidence level.
description.tags	string[]	A collection of image tags.
metadata	ImageMetadata	Image metadata.
requestId	string	Id of the REST API request.

ImageMetadata

Image metadata.

Name	Type	Description
format	string	Image format.
height	integer	Image height, in pixels.
width	integer	Image width, in pixels.

ImageUrl

Name	Type	Description
url	string	Publicly reachable URL of an image.