Face Detection Operations - Detect

Reference

Service:: Azure AI Services

API Version:: v1.1-preview.1

Detect human faces in an image, return face rectangles, and optionally with faceIds, landmarks, and attributes.

Important

Microsoft has retired or limited facial recognition capabilities that can be used to try to infer emotional states and identity attributes which, if misused, can subject people to stereotyping, discrimination or unfair denial of services. The retired capabilities are emotion and gender. The limited capabilities are age, smile, facial hair, hair and makeup. Email Azure Face API if you have a responsible use case that would benefit from the use of any of the limited capabilities. Read more about this decision here.

- No image will be stored. Only the extracted face feature(s) will be stored on server. The faceId is an identifier of the face feature and will be used in "Identify", "Verify", and "Find Similar". The stored face features will expire and be deleted at the time specified by faceIdTimeToLive after the original detection call.
- Optional parameters include faceId, landmarks, and attributes. Attributes include headPose, glasses, occlusion, accessories, blur, exposure, noise, mask, and qualityForRecognition. Some of the results returned for specific attributes may not be highly accurate.
- JPEG, PNG, GIF (the first frame), and BMP format are supported. The allowed image file size is from 1KB to 6MB.
- The minimum detectable face size is 36x36 pixels in an image no larger than 1920x1080 pixels. Images with dimensions higher than 1920x1080 pixels will need a proportionally larger minimum face size.
- Up to 100 faces can be returned for an image. Faces are ranked by face rectangle size from large to small.
- For optimal results when querying "Identify", "Verify", and "Find Similar" ('returnFaceId' is true), please use faces that are: frontal, clear, and with a minimum size of 200x200 pixels (100 pixels between eyes).
- Different 'detectionModel' values can be provided. The availability of landmarks and supported attributes depends on the detection model specified. To use and compare different detection models, please refer to here.
- Different 'recognitionModel' values are provided. If follow-up operations like "Verify", "Identify", "Find Similar" are needed, please specify the recognition model with 'recognitionModel' parameter. The default value for 'recognitionModel' is 'recognition_01', if latest model needed, please explicitly specify the model you need in this parameter. Once specified, the detected faceIds will be associated with the specified recognition model. More details, please refer to here.

POST {endpoint}/face/{apiVersion}/detect

With optional parameters:

POST {endpoint}/face/{apiVersion}/detect?_overload=detect&detectionModel={detectionModel}&recognitionModel={recognitionModel}&returnFaceId={returnFaceId}&returnFaceAttributes={returnFaceAttributes}&returnFaceLandmarks={returnFaceLandmarks}&returnRecognitionModel={returnRecognitionModel}&faceIdTimeToLive={faceIdTimeToLive}

URI Parameters

Name	In	Required	Type	Description
apiVersion	path	True	string	API Version
endpoint	path	True	string uri	Supported Cognitive Services endpoints (protocol and hostname, for example: https://{resource-name}.cognitiveservices.azure.com).
detectionModel	query		DetectionModel	The 'detectionModel' associated with the detected faceIds. Supported 'detectionModel' values include 'detection_01', 'detection_02' and 'detection_03'. The default value is 'detection_01'. 'detection_03' is recommended since its accuracy is improved on smaller faces (64x64 pixels) and rotated face orientations.
faceIdTimeToLive	query		integer int32	The number of seconds for the face ID being cached. Supported range from 60 seconds up to 86400 seconds. The default value is 86400 (24 hours).
recognitionModel	query		RecognitionModel	The 'recognitionModel' associated with the detected faceIds. Supported 'recognitionModel' values include 'recognition_01', 'recognition_02', 'recognition_03' or 'recognition_04'. The default value is 'recognition_01'. 'recognition_04' is recommended since its accuracy is improved on faces wearing masks compared with 'recognition_03', and its overall accuracy is improved compared with 'recognition_01' and 'recognition_02'.
returnFaceAttributes	query		FaceAttributeType[]	Analyze and return the one or more specified face attributes in the comma-separated string like 'returnFaceAttributes=headPose,glasses'. Face attribute analysis has additional computational and time cost.
returnFaceId	query		boolean	Return faceIds of the detected faces or not. The default value is true.
returnFaceLandmarks	query		boolean	Return face landmarks of the detected faces or not. The default value is false.
returnRecognitionModel	query		boolean	Return 'recognitionModel' or not. The default value is false. This is only applicable when returnFaceId = true.

Request Body

Media Types: "application/octet-stream"

Name	Type	Description
imageContent	string	The input image binary.

Responses

Name	Type	Description
200 OK	FaceDetectionResult[]	A successful call returns an array of face entries ranked by face rectangle size in descending order. An empty response indicates no faces detected.
Other Status Codes	FaceErrorResponse	An unexpected error response. Headers x-ms-error-code: string

Name

Type

Description

200 OK

FaceDetectionResult[]

A successful call returns an array of face entries ranked by face rectangle size in descending order. An empty response indicates no faces detected.

Other Status Codes

FaceErrorResponse

An unexpected error response.

Headers

x-ms-error-code: string

Security

Ocp-Apim-Subscription-Key

The secret key for your Azure AI Face subscription.

Type: apiKey
In: header

AADToken

The Azure Active Directory OAuth2 Flow

Type: oauth2
Flow: accessCode
Authorization URL: https://api.example.com/oauth2/authorize
Token URL: https://api.example.com/oauth2/token

Scopes

Name	Description
https://cognitiveservices.azure.com/.default

Examples

Detect with Image

Sample request

HTTP

POST {endpoint}/face/v1.1-preview.1/detect?_overload=detect&detectionModel=detection_01&recognitionModel=recognition_03&returnFaceId=True&returnFaceAttributes=glasses,headPose,occlusion,accessories,blur,exposure,noise,qualityForRecognition&returnFaceLandmarks=True&returnRecognitionModel=True&faceIdTimeToLive=60

"<your-image-bytes-here>"

Sample response

Status code:: 200

[
  {
    "faceId": "c5c24a82-6845-4031-9d5d-978df9175426",
    "recognitionModel": "recognition_03",
    "faceRectangle": {
      "width": 78,
      "height": 78,
      "left": 394,
      "top": 54
    },
    "faceLandmarks": {
      "pupilLeft": {
        "x": 412.7,
        "y": 78.4
      },
      "pupilRight": {
        "x": 446.8,
        "y": 74.2
      },
      "noseTip": {
        "x": 437.7,
        "y": 92.4
      },
      "mouthLeft": {
        "x": 417.8,
        "y": 114.4
      },
      "mouthRight": {
        "x": 451.3,
        "y": 109.3
      },
      "eyebrowLeftOuter": {
        "x": 397.9,
        "y": 78.5
      },
      "eyebrowLeftInner": {
        "x": 425.4,
        "y": 70.5
      },
      "eyeLeftOuter": {
        "x": 406.7,
        "y": 80.6
      },
      "eyeLeftTop": {
        "x": 412.2,
        "y": 76.2
      },
      "eyeLeftBottom": {
        "x": 413,
        "y": 80.1
      },
      "eyeLeftInner": {
        "x": 418.9,
        "y": 78
      },
      "eyebrowRightInner": {
        "x": 4.8,
        "y": 69.7
      },
      "eyebrowRightOuter": {
        "x": 5.5,
        "y": 68.5
      },
      "eyeRightInner": {
        "x": 441.5,
        "y": 75
      },
      "eyeRightTop": {
        "x": 446.4,
        "y": 71.7
      },
      "eyeRightBottom": {
        "x": 447,
        "y": 75.3
      },
      "eyeRightOuter": {
        "x": 451.7,
        "y": 73.4
      },
      "noseRootLeft": {
        "x": 428,
        "y": 77.1
      },
      "noseRootRight": {
        "x": 435.8,
        "y": 75.6
      },
      "noseLeftAlarTop": {
        "x": 428.3,
        "y": 89.7
      },
      "noseRightAlarTop": {
        "x": 442.2,
        "y": 87
      },
      "noseLeftAlarOutTip": {
        "x": 424.3,
        "y": 96.4
      },
      "noseRightAlarOutTip": {
        "x": 446.6,
        "y": 92.5
      },
      "upperLipTop": {
        "x": 437.6,
        "y": 105.9
      },
      "upperLipBottom": {
        "x": 437.6,
        "y": 108.2
      },
      "underLipTop": {
        "x": 436.8,
        "y": 111.4
      },
      "underLipBottom": {
        "x": 437.3,
        "y": 114.5
      }
    },
    "faceAttributes": {
      "glasses": "sunglasses",
      "headPose": {
        "roll": 2.1,
        "yaw": 3,
        "pitch": 1.6
      },
      "occlusion": {
        "foreheadOccluded": false,
        "eyeOccluded": false,
        "mouthOccluded": false
      },
      "accessories": [
        {
          "type": "headwear",
          "confidence": 0.99
        },
        {
          "type": "glasses",
          "confidence": 1
        },
        {
          "type": "mask",
          "confidence": 0.87
        }
      ],
      "blur": {
        "blurLevel": "medium",
        "value": 0.51
      },
      "exposure": {
        "exposureLevel": "goodExposure",
        "value": 0.55
      },
      "noise": {
        "noiseLevel": "low",
        "value": 0.12
      },
      "qualityForRecognition": "high"
    }
  }
]

Definitions

Name	Description
AccessoryItem	Accessory item and corresponding confidence level.
AccessoryType	Type of the accessory.
BlurLevel	Indicates level of blurriness.
BlurProperties	Properties describing any presence of blur within the image.
DetectionModel	The 'detectionModel' associated with the detected faceIds. Supported 'detectionModel' values include 'detection_01', 'detection_02' and 'detection_03'. The default value is 'detection_01'. 'detection_03' is recommended since its accuracy is improved on smaller faces (64x64 pixels) and rotated face orientations.
ExposureLevel	Indicates level of exposure.
ExposureProperties	Properties describing exposure level of the image.
FaceAttributes	Face attributes for the detected face.
FaceAttributeType	Analyze and return the one or more specified face attributes in the comma-separated string like 'returnFaceAttributes=headPose,glasses'. Face attribute analysis has additional computational and time cost.
FaceDetectionResult	Response for detect API.
FaceError	The error object. For comprehensive details on error codes and messages returned by the Face Service, please refer to the following link: https://aka.ms/face-error-codes-and-messages.
FaceErrorResponse	A response containing error details.
FaceLandmarks	A collection of 27-point face landmarks pointing to the important positions of face components.
FaceRectangle	A rectangle within which a face can be found.
FacialHair	Properties describing facial hair attributes.
GlassesType	Glasses type of the face.
HairColor	An array of candidate colors and confidence level in the presence of each.
HairColorType	Name of the hair color.
HairProperties	Properties describing hair attributes.
HeadPose	3-D roll/yaw/pitch angles for face direction.
LandmarkCoordinate	Landmark coordinates within an image.
MaskProperties	Properties describing the presence of a mask on a given face.
MaskType	Type of the mask.
NoiseLevel	Indicates level of noise.
NoiseProperties	Properties describing noise level of the image.
OcclusionProperties	Properties describing occlusions on a given face.
QualityForRecognition	Indicates quality of image for recognition.
RecognitionModel	The recognition model for the face.

AccessoryItem

Accessory item and corresponding confidence level.

Name	Type	Description
confidence	number	Confidence level of the accessory type. Range between [0,1].
type	AccessoryType	Type of the accessory.

AccessoryType

Type of the accessory.

Name	Type	Description
glasses	string	Glasses.
headwear	string	Head wear.
mask	string	Mask.

BlurLevel

Indicates level of blurriness.

Name	Type	Description
high	string	High blur level.
low	string	Low blur level.
medium	string	Medium blur level.

BlurProperties

Properties describing any presence of blur within the image.

Name	Type	Description
blurLevel	BlurLevel	An enum value indicating level of blurriness.
value	number	A number indicating level of blurriness ranging from 0 to 1.

DetectionModel

The 'detectionModel' associated with the detected faceIds. Supported 'detectionModel' values include 'detection_01', 'detection_02' and 'detection_03'. The default value is 'detection_01'. 'detection_03' is recommended since its accuracy is improved on smaller faces (64x64 pixels) and rotated face orientations.

Name	Type	Description
detection_01	string	The default detection model. Recommend for near frontal face detection. For scenarios with exceptionally large angle (head-pose) faces, occluded faces or wrong image orientation, the faces in such cases may not be detected.
detection_02	string	Detection model released in 2019 May with improved accuracy especially on small, side and blurry faces.
detection_03	string	Detection model released in 2021 February with improved accuracy especially on small faces.

ExposureLevel

Indicates level of exposure.

Name	Type	Description
goodExposure	string	Good exposure level.
overExposure	string	High exposure level.
underExposure	string	Low exposure level.

ExposureProperties

Properties describing exposure level of the image.

Name	Type	Description
exposureLevel	ExposureLevel	An enum value indicating level of exposure.
value	number	A number indicating level of exposure level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure.

FaceAttributes

Face attributes for the detected face.

Name	Type	Description
accessories	AccessoryItem[]	Properties describing any accessories on a given face.
age	number	Age in years.
blur	BlurProperties	Properties describing any presence of blur within the image.
exposure	ExposureProperties	Properties describing exposure level of the image.
facialHair	FacialHair	Properties describing facial hair attributes.
glasses	GlassesType	Glasses type if any of the face.
hair	HairProperties	Properties describing hair attributes.
headPose	HeadPose	3-D roll/yaw/pitch angles for face direction.
mask	MaskProperties	Properties describing the presence of a mask on a given face.
noise	NoiseProperties	Properties describing noise level of the image.
occlusion	OcclusionProperties	Properties describing occlusions on a given face.
qualityForRecognition	QualityForRecognition	Properties describing the overall image quality regarding whether the image being used in the detection is of sufficient quality to attempt face recognition on.
smile	number	Smile intensity, a number between [0,1].

FaceAttributeType

Analyze and return the one or more specified face attributes in the comma-separated string like 'returnFaceAttributes=headPose,glasses'. Face attribute analysis has additional computational and time cost.

Name	Type	Description
accessories	string	Accessories around face, including 'headwear', 'glasses' and 'mask'. Empty array means no accessories detected. Note this is after a face is detected. Large mask could result in no face to be detected.
age	string	Age in years.
blur	string	Face is blurry or not. Level returns 'Low', 'Medium' or 'High'. Value returns a number between [0,1], the larger the blurrier.
exposure	string	Face exposure level. Level returns 'GoodExposure', 'OverExposure' or 'UnderExposure'.
facialHair	string	Properties describing facial hair attributes.
glasses	string	Glasses type. Values include 'NoGlasses', 'ReadingGlasses', 'Sunglasses', 'SwimmingGoggles'.
hair	string	Properties describing hair attributes.
headPose	string	3-D roll/yaw/pitch angles for face direction.
mask	string	Whether each face is wearing a mask. Mask type returns 'noMask', 'faceMask', 'otherMaskOrOcclusion', or 'uncertain'. Value returns a boolean 'noseAndMouthCovered' indicating whether nose and mouth are covered.
noise	string	Noise level of face pixels. Level returns 'Low', 'Medium' and 'High'. Value returns a number between [0,1], the larger the noisier
occlusion	string	Whether each facial area is occluded, including forehead, eyes and mouth.
qualityForRecognition	string	The overall image quality regarding whether the image being used in the detection is of sufficient quality to attempt face recognition on. The value is an informal rating of low, medium, or high. Only 'high' quality images are recommended for person enrollment and quality at or above 'medium' is recommended for identification scenarios. The attribute is only available when using recognition models recognition_03 or recognition_04.
smile	string	Smile intensity, a number between [0,1].

FaceDetectionResult

Response for detect API.

Name	Type	Description
faceAttributes	FaceAttributes	Face attributes for detected face.
faceId	string	Unique faceId of the detected face, created by detection API and it will expire 24 hours after the detection call. To return this, it requires 'returnFaceId' parameter to be true.
faceLandmarks	FaceLandmarks	An array of 27-point face landmarks pointing to the important positions of face components. To return this, it requires 'returnFaceLandmarks' parameter to be true.
faceRectangle	FaceRectangle	A rectangle area for the face location on image.
recognitionModel	RecognitionModel	The 'recognitionModel' associated with this faceId. This is only returned when 'returnRecognitionModel' is explicitly set as true.

FaceError

The error object. For comprehensive details on error codes and messages returned by the Face Service, please refer to the following link: https://aka.ms/face-error-codes-and-messages.

Name	Type	Description
code	string	One of a server-defined set of error codes.
message	string	A human-readable representation of the error.

FaceErrorResponse

A response containing error details.

Name	Type	Description
error	FaceError	The error object.

FaceLandmarks

A collection of 27-point face landmarks pointing to the important positions of face components.

Name	Type	Description
eyeLeftBottom	LandmarkCoordinate	The coordinates of the left eye bottom.
eyeLeftInner	LandmarkCoordinate	The coordinates of the left eye inner.
eyeLeftOuter	LandmarkCoordinate	The coordinates of the left eye outer.
eyeLeftTop	LandmarkCoordinate	The coordinates of the left eye top.
eyeRightBottom	LandmarkCoordinate	The coordinates of the right eye bottom.
eyeRightInner	LandmarkCoordinate	The coordinates of the right eye inner.
eyeRightOuter	LandmarkCoordinate	The coordinates of the right eye outer.
eyeRightTop	LandmarkCoordinate	The coordinates of the right eye top.
eyebrowLeftInner	LandmarkCoordinate	The coordinates of the left eyebrow inner.
eyebrowLeftOuter	LandmarkCoordinate	The coordinates of the left eyebrow outer.
eyebrowRightInner	LandmarkCoordinate	The coordinates of the right eyebrow inner.
eyebrowRightOuter	LandmarkCoordinate	The coordinates of the right eyebrow outer.
mouthLeft	LandmarkCoordinate	The coordinates of the mouth left.
mouthRight	LandmarkCoordinate	The coordinates of the mouth right.
noseLeftAlarOutTip	LandmarkCoordinate	The coordinates of the nose left alar out tip.
noseLeftAlarTop	LandmarkCoordinate	The coordinates of the nose left alar top.
noseRightAlarOutTip	LandmarkCoordinate	The coordinates of the nose right alar out tip.
noseRightAlarTop	LandmarkCoordinate	The coordinates of the nose right alar top.
noseRootLeft	LandmarkCoordinate	The coordinates of the nose root left.
noseRootRight	LandmarkCoordinate	The coordinates of the nose root right.
noseTip	LandmarkCoordinate	The coordinates of the nose tip.
pupilLeft	LandmarkCoordinate	The coordinates of the left eye pupil.
pupilRight	LandmarkCoordinate	The coordinates of the right eye pupil.
underLipBottom	LandmarkCoordinate	The coordinates of the under lip bottom.
underLipTop	LandmarkCoordinate	The coordinates of the under lip top.
upperLipBottom	LandmarkCoordinate	The coordinates of the upper lip bottom.
upperLipTop	LandmarkCoordinate	The coordinates of the upper lip top.

FaceRectangle

A rectangle within which a face can be found.

Name	Type	Description
height	integer	The height of the rectangle, in pixels.
left	integer	The distance from the left edge if the image to the left edge of the rectangle, in pixels.
top	integer	The distance from the top edge if the image to the top edge of the rectangle, in pixels.
width	integer	The width of the rectangle, in pixels.

FacialHair

Properties describing facial hair attributes.

Name	Type	Description
beard	number	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
moustache	number	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
sideburns	number	A number ranging from 0 to 1 indicating a level of confidence associated with a property.

GlassesType

Glasses type of the face.

Name	Type	Description
noGlasses	string	No glasses on the face.
readingGlasses	string	Normal glasses on the face.
sunglasses	string	Sunglasses on the face.
swimmingGoggles	string	Swimming goggles on the face.

HairColor

An array of candidate colors and confidence level in the presence of each.

Name	Type	Description
color	HairColorType	Name of the hair color.
confidence	number	Confidence level of the color. Range between [0,1].

HairColorType

Name of the hair color.

Name	Type	Description
black	string	Black.
blond	string	Blond.
brown	string	Brown.
gray	string	Gray.
other	string	Other.
red	string	Red.
unknown	string	Unknown.
white	string	White.

HairProperties

Properties describing hair attributes.

Name	Type	Description
bald	number	A number describing confidence level of whether the person is bald.
hairColor	HairColor[]	An array of candidate colors and confidence level in the presence of each.
invisible	boolean	A boolean value describing whether the hair is visible in the image.

HeadPose

3-D roll/yaw/pitch angles for face direction.

Name	Type	Description
pitch	number	Value of angles.
roll	number	Value of angles.
yaw	number	Value of angles.

LandmarkCoordinate

Landmark coordinates within an image.

Name	Type	Description
x	number	The horizontal component, in pixels.
y	number	The vertical component, in pixels.

MaskProperties

Properties describing the presence of a mask on a given face.

Name	Type	Description
noseAndMouthCovered	boolean	A boolean value indicating whether nose and mouth are covered.
type	MaskType	Type of the mask.

MaskType

Type of the mask.

Name	Type	Description
faceMask	string	Face mask.
noMask	string	No mask.
otherMaskOrOcclusion	string	Other types of mask or occlusion.
uncertain	string	Uncertain.

NoiseLevel

Indicates level of noise.

Name	Type	Description
high	string	High noise level.
low	string	Low noise level.
medium	string	Medium noise level.

NoiseProperties

Properties describing noise level of the image.

Name	Type	Description
noiseLevel	NoiseLevel	An enum value indicating level of noise.
value	number	A number indicating level of noise level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure. [0, 0.3) is low noise level. [0.3, 0.7) is medium noise level. [0.7, 1] is high noise level.

OcclusionProperties

Properties describing occlusions on a given face.

Name	Type	Description
eyeOccluded	boolean	A boolean value indicating whether eyes are occluded.
foreheadOccluded	boolean	A boolean value indicating whether forehead is occluded.
mouthOccluded	boolean	A boolean value indicating whether the mouth is occluded.

QualityForRecognition

Indicates quality of image for recognition.

Name	Type	Description
high	string	High quality.
low	string	Low quality.
medium	string	Medium quality.

RecognitionModel

The recognition model for the face.

Name	Type	Description
recognition_01	string	The default recognition model for "Detect". All those faceIds created before 2019 March are bonded with this recognition model.
recognition_02	string	Recognition model released in 2019 March.
recognition_03	string	Recognition model released in 2020 May.
recognition_04	string	Recognition model released in 2021 February. It's recommended to use this recognition model for better recognition accuracy.

Share via

Face Detection Operations - Detect

URI Parameters

Request Body

Responses

Security

Ocp-Apim-Subscription-Key

AADToken

Scopes

Examples

Detect with Image

Sample request

Sample response

Definitions

AccessoryItem

AccessoryType

BlurLevel

BlurProperties

DetectionModel

ExposureLevel

ExposureProperties

FaceAttributes

FaceAttributeType

FaceDetectionResult

FaceError

FaceErrorResponse

FaceLandmarks

FaceRectangle

FacialHair

GlassesType

HairColor

HairColorType

HairProperties

HeadPose

LandmarkCoordinate

MaskProperties

MaskType

NoiseLevel

NoiseProperties

OcclusionProperties

QualityForRecognition

RecognitionModel

Additional resources