Face - Detect With Stream

Service:: Azure AI Services

API Version:: 1.0-preview

Detect human faces in an image, return face rectangles, and optionally with faceIds, landmarks, and attributes.

No image will be stored. Only the extracted face feature will be stored on server. The faceId is an identifier of the face feature and will be used in Face - Identify, Face - Verify, and Face - Find Similar. The stored face feature(s) will expire and be deleted at the time specified by faceIdTimeToLive after the original detection call.
Optional parameters include faceId, landmarks, and attributes. Attributes include age, gender, headPose, smile, facialHair, glasses, emotion, hair, makeup, occlusion, accessories, blur, exposure, noise, mask, and qualityForRecognition. Some of the results returned for specific attributes may not be highly accurate.
JPEG, PNG, GIF (the first frame), and BMP format are supported. The allowed image file size is from 1KB to 6MB.
Up to 100 faces can be returned for an image. Faces are ranked by face rectangle size from large to small.
For optimal results when querying Face - Identify, Face - Verify, and Face - Find Similar ('returnFaceId' is true), please use faces that are: frontal, clear, and with a minimum size of 200x200 pixels (100 pixels between eyes).
The minimum detectable face size is 36x36 pixels in an image no larger than 1920x1080 pixels. Images with dimensions higher than 1920x1080 pixels will need a proportionally larger minimum face size.
Different 'detectionModel' values can be provided. To use and compare different detection models, please refer to How to specify a detection model
Different 'recognitionModel' values are provided. If follow-up operations like Verify, Identify, Find Similar are needed, please specify the recognition model with 'recognitionModel' parameter. The default value for 'recognitionModel' is 'recognition_01', if latest model needed, please explicitly specify the model you need in this parameter. Once specified, the detected faceIds will be associated with the specified recognition model. More details, please refer to Specify a recognition model.

POST {Endpoint}/face/v1.0-preview/detect

With optional parameters:

POST {Endpoint}/face/v1.0-preview/detect?overload=stream&returnFaceId={returnFaceId}&returnFaceLandmarks={returnFaceLandmarks}&returnFaceAttributes={returnFaceAttributes}&recognitionModel={recognitionModel}&returnRecognitionModel={returnRecognitionModel}&detectionModel={detectionModel}&faceIdTimeToLive={faceIdTimeToLive}

URI Parameters

Name	In	Required	Type	Description
Endpoint	path	True	string	Supported Cognitive Services endpoints (protocol and hostname, for example: https://westus.api.cognitive.microsoft.com).
detectionModel	query		DetectionModel	Name of detection model. Detection model is used to detect faces in the submitted image. A detection model name can be provided when performing Face - Detect or (Large)FaceList - Add Face or (Large)PersonGroup - Add Face. The default value is 'detection_01', if another model is needed, please explicitly specify it.
faceIdTimeToLive	query		integer minimum: 60 maximum: 86400	The number of seconds for the faceId being cached. Supported range from 60 seconds up to 86400 seconds. The default value is 86400 (24 hours).
recognitionModel	query		RecognitionModel	Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.
returnFaceAttributes	query		FaceAttributeType[]	Analyze and return the one or more specified face attributes in the comma-separated string like "returnFaceAttributes=age,gender". The available attributes depends on the 'detectionModel' specified. 'detection_01' supports age, gender, headPose, smile, facialHair, glasses, emotion, hair, makeup, occlusion, accessories, blur, exposure, noise, and qualityForRecognition. While 'detection_02' does not support any attributes and 'detection_03' only supports mask and qualityForRecognition. Additionally, qualityForRecognition is only supported when the 'recognitionModel' is specified as 'recognition_03' or 'recognition_04'. Note that each face attribute analysis has additional computational and time cost.
returnFaceId	query		boolean	A value indicating whether the operation should return faceIds of detected faces. Detecting with this value set to true requires additional access approvals at https://aka.ms/facerecognition.
returnFaceLandmarks	query		boolean	A value indicating whether the operation should return landmarks of the detected faces.
returnRecognitionModel	query		boolean	A value indicating whether the operation should return 'recognitionModel' in response.

Request Header

Media Types: "application/octet-stream"

Name	Required	Type	Description
Ocp-Apim-Subscription-Key	True	string

Request Body

Media Types: "application/octet-stream"

Name	Type	Description
Image	object (file)	An image stream.

Responses

Name	Type	Description
200 OK	DetectedFace[]	A successful call returns an array of face entries ranked by face rectangle size in descending order. An empty response indicates no faces detected.
Other Status Codes	APIError	Error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

Examples

Detect with stream example

Sample request

HTTP

POST {Endpoint}/face/v1.0-preview/detect?overload=stream&returnFaceAttributes=age,gender,headPose,smile,facialHair,glasses,emotion,hair,makeup,occlusion,accessories,blur,exposure,noise&recognitionModel=recognition_01&returnRecognitionModel=True&detectionModel=detection_01


"{Image binary in base 64 format}"

Sample response

Status code:: 200

[
  {
    "faceId": "c5c24a82-6845-4031-9d5d-978df9175426",
    "recognitionModel": "recognition_01",
    "faceRectangle": {
      "width": 78,
      "height": 78,
      "left": 394,
      "top": 54
    },
    "faceLandmarks": {
      "pupilLeft": {
        "x": 412.7,
        "y": 78.4
      },
      "pupilRight": {
        "x": 446.8,
        "y": 74.2
      },
      "noseTip": {
        "x": 437.7,
        "y": 92.4
      },
      "mouthLeft": {
        "x": 417.8,
        "y": 114.4
      },
      "mouthRight": {
        "x": 451.3,
        "y": 109.3
      },
      "eyebrowLeftOuter": {
        "x": 397.9,
        "y": 78.5
      },
      "eyebrowLeftInner": {
        "x": 425.4,
        "y": 70.5
      },
      "eyeLeftOuter": {
        "x": 406.7,
        "y": 80.6
      },
      "eyeLeftTop": {
        "x": 412.2,
        "y": 76.2
      },
      "eyeLeftBottom": {
        "x": 413,
        "y": 80.1
      },
      "eyeLeftInner": {
        "x": 418.9,
        "y": 78
      },
      "eyebrowRightInner": {
        "x": 4.8,
        "y": 69.7
      },
      "eyebrowRightOuter": {
        "x": 5.5,
        "y": 68.5
      },
      "eyeRightInner": {
        "x": 441.5,
        "y": 75
      },
      "eyeRightTop": {
        "x": 446.4,
        "y": 71.7
      },
      "eyeRightBottom": {
        "x": 447,
        "y": 75.3
      },
      "eyeRightOuter": {
        "x": 451.7,
        "y": 73.4
      },
      "noseRootLeft": {
        "x": 428,
        "y": 77.1
      },
      "noseRootRight": {
        "x": 435.8,
        "y": 75.6
      },
      "noseLeftAlarTop": {
        "x": 428.3,
        "y": 89.7
      },
      "noseRightAlarTop": {
        "x": 442.2,
        "y": 87
      },
      "noseLeftAlarOutTip": {
        "x": 424.3,
        "y": 96.4
      },
      "noseRightAlarOutTip": {
        "x": 446.6,
        "y": 92.5
      },
      "upperLipTop": {
        "x": 437.6,
        "y": 105.9
      },
      "upperLipBottom": {
        "x": 437.6,
        "y": 108.2
      },
      "underLipTop": {
        "x": 436.8,
        "y": 111.4
      },
      "underLipBottom": {
        "x": 437.3,
        "y": 114.5
      }
    },
    "faceAttributes": {
      "age": 71,
      "gender": "male",
      "smile": 0.88,
      "facialHair": {
        "moustache": 0.8,
        "beard": 0.1,
        "sideburns": 0.02
      },
      "glasses": "sunglasses",
      "headPose": {
        "roll": 2.1,
        "yaw": 3,
        "pitch": 1.6
      },
      "emotion": {
        "anger": 0.575,
        "contempt": 0,
        "disgust": 0.006,
        "fear": 0.008,
        "happiness": 0.394,
        "neutral": 0.013,
        "sadness": 0,
        "surprise": 0.004
      },
      "hair": {
        "bald": 0,
        "invisible": false,
        "hairColor": [
          {
            "color": "brown",
            "confidence": 1
          },
          {
            "color": "blond",
            "confidence": 0.88
          },
          {
            "color": "black",
            "confidence": 0.48
          },
          {
            "color": "other",
            "confidence": 0.11
          },
          {
            "color": "gray",
            "confidence": 0.07
          },
          {
            "color": "red",
            "confidence": 0.03
          }
        ]
      },
      "makeup": {
        "eyeMakeup": true,
        "lipMakeup": false
      },
      "occlusion": {
        "foreheadOccluded": false,
        "eyeOccluded": false,
        "mouthOccluded": false
      },
      "accessories": [
        {
          "type": "headWear",
          "confidence": 0.99
        },
        {
          "type": "glasses",
          "confidence": 1
        },
        {
          "type": "mask",
          "confidence": 0.87
        }
      ],
      "blur": {
        "blurLevel": "Medium",
        "value": 0.51
      },
      "exposure": {
        "exposureLevel": "GoodExposure",
        "value": 0.55
      },
      "noise": {
        "noiseLevel": "Low",
        "value": 0.12
      }
    }
  }
]

Definitions

Name	Description
Accessory	Accessory item and corresponding confidence level.
AccessoryType	Type of an accessory
APIError	Error information returned by the API
Blur	Properties describing any presence of blur within the image.
BlurLevel	An enum value indicating level of blurriness.
Coordinate	Coordinates within an image
DetectedFace	Detected Face object.
DetectionModel	Detection model used when extracting the face.
Emotion	Properties describing facial emotion in form of confidence ranging from 0 to 1.
Error	Error body.
Exposure	Properties describing exposure level of the image.
ExposureLevel	An enum value indicating level of exposure.
FaceAttributes	Face Attributes
FaceAttributeType	Analyze and return the one or more specified face attributes in the comma-separated string like "returnFaceAttributes=age,gender". The available attributes depends on the 'detectionModel' specified. 'detection_01' supports age, gender, headPose, smile, facialHair, glasses, emotion, hair, makeup, occlusion, accessories, blur, exposure, noise, and qualityForRecognition. While 'detection_02' does not support any attributes and 'detection_03' only supports mask and qualityForRecognition. Additionally, qualityForRecognition is only supported when the 'recognitionModel' is specified as 'recognition_03' or 'recognition_04'. Note that each face attribute analysis has additional computational and time cost.
FaceLandmarks	A collection of 27-point face landmarks pointing to the important positions of face components.
FaceRectangle	A rectangle within which a face can be found
FacialHair	Properties describing facial hair attributes.
Gender	Possible gender of the face.
GlassesType	Glasses type if any of the face.
Hair	Properties describing hair attributes.
HairColor	Hair color and associated confidence
HairColorType	Name of the hair color.
HeadPose	Properties indicating head pose of the face.
Makeup	Properties describing the presence of makeup on a given face.
Mask	Properties describing the presence of a mask on a given face.
MaskType	Mask type if any of the face
Noise	Properties describing noise level of the image.
NoiseLevel	An enum value indicating level of noise.
Occlusion	Properties describing occlusions on a given face.
QualityForRecognition	Properties describing the overall image quality regarding whether the image being used in the detection is of sufficient quality to attempt face recognition on.
RecognitionModel	Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

Accessory

Object

Accessory item and corresponding confidence level.

Name	Type	Description
confidence	number minimum: 0 maximum: 1	Confidence level of an accessory
type	AccessoryType	Type of an accessory

AccessoryType

Enumeration

Type of an accessory

Value	Description
headWear
glasses
mask

APIError

Object

Error information returned by the API

Name	Type	Description
error	Error	Error body.

Blur

Object

Properties describing any presence of blur within the image.

Name	Type	Description
blurLevel	BlurLevel	An enum value indicating level of blurriness.
value	number minimum: 0 maximum: 1	A number indicating level of blurriness ranging from 0 to 1.

BlurLevel

Enumeration

An enum value indicating level of blurriness.

Value	Description
Low
Medium
High

Coordinate

Object

Coordinates within an image

Name	Type	Description
x	number	The horizontal component, in pixels.
y	number	The vertical component, in pixels.

DetectedFace

Object

Detected Face object.

Name	Type	Default value	Description
faceAttributes	FaceAttributes		Face Attributes
faceId	string (uuid)
faceLandmarks	FaceLandmarks		A collection of 27-point face landmarks pointing to the important positions of face components.
faceRectangle	FaceRectangle		A rectangle within which a face can be found
recognitionModel	RecognitionModel	recognition_01	Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

DetectionModel

Enumeration

Detection model used when extracting the face.

Value	Description
detection_01
detection_02
detection_03
detection_preview_1904
ir_detection_01
expression_01

Emotion

Object

Properties describing facial emotion in form of confidence ranging from 0 to 1.

Name	Type	Description
anger	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
contempt	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
disgust	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
fear	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
happiness	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
neutral	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
sadness	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
surprise	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.

Error

Object

Error body.

Name	Type	Description
code	string
message	string

Exposure

Object

Properties describing exposure level of the image.

Name	Type	Description
exposureLevel	ExposureLevel	An enum value indicating level of exposure.
value	number minimum: 0 maximum: 1	A number indicating level of exposure level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure.

ExposureLevel

Enumeration

An enum value indicating level of exposure.

Value	Description
UnderExposure
GoodExposure
OverExposure

FaceAttributes

Object

Face Attributes

Name	Type	Description
accessories	Accessory[]	Properties describing any accessories on a given face.
age	number	Age in years
blur	Blur	Properties describing any presence of blur within the image.
emotion	Emotion	Properties describing facial emotion in form of confidence ranging from 0 to 1.
exposure	Exposure	Properties describing exposure level of the image.
facialHair	FacialHair	Properties describing facial hair attributes.
gender	Gender	Possible gender of the face.
glasses	GlassesType	Glasses type if any of the face.
hair	Hair	Properties describing hair attributes.
headPose	HeadPose	Properties indicating head pose of the face.
makeup	Makeup	Properties describing the presence of makeup on a given face.
mask	Mask	Properties describing the presence of a mask on a given face.
noise	Noise	Properties describing noise level of the image.
occlusion	Occlusion	Properties describing occlusions on a given face.
qualityForRecognition	QualityForRecognition	Properties describing the overall image quality regarding whether the image being used in the detection is of sufficient quality to attempt face recognition on.
smile	number minimum: 0 maximum: 1	Smile intensity, a number between [0,1]

FaceAttributeType

Enumeration

Analyze and return the one or more specified face attributes in the comma-separated string like "returnFaceAttributes=age,gender". The available attributes depends on the 'detectionModel' specified. 'detection_01' supports age, gender, headPose, smile, facialHair, glasses, emotion, hair, makeup, occlusion, accessories, blur, exposure, noise, and qualityForRecognition. While 'detection_02' does not support any attributes and 'detection_03' only supports mask and qualityForRecognition. Additionally, qualityForRecognition is only supported when the 'recognitionModel' is specified as 'recognition_03' or 'recognition_04'. Note that each face attribute analysis has additional computational and time cost.

Value	Description
age
gender
headPose
smile
facialHair
glasses
emotion
hair
makeup
occlusion
accessories
blur
exposure
noise
mask
qualityForRecognition

FaceLandmarks

Object

A collection of 27-point face landmarks pointing to the important positions of face components.

Name	Type	Description
eyeLeftBottom	Coordinate	Coordinates within an image
eyeLeftInner	Coordinate	Coordinates within an image
eyeLeftOuter	Coordinate	Coordinates within an image
eyeLeftTop	Coordinate	Coordinates within an image
eyeRightBottom	Coordinate	Coordinates within an image
eyeRightInner	Coordinate	Coordinates within an image
eyeRightOuter	Coordinate	Coordinates within an image
eyeRightTop	Coordinate	Coordinates within an image
eyebrowLeftInner	Coordinate	Coordinates within an image
eyebrowLeftOuter	Coordinate	Coordinates within an image
eyebrowRightInner	Coordinate	Coordinates within an image
eyebrowRightOuter	Coordinate	Coordinates within an image
mouthLeft	Coordinate	Coordinates within an image
mouthRight	Coordinate	Coordinates within an image
noseLeftAlarOutTip	Coordinate	Coordinates within an image
noseLeftAlarTop	Coordinate	Coordinates within an image
noseRightAlarOutTip	Coordinate	Coordinates within an image
noseRightAlarTop	Coordinate	Coordinates within an image
noseRootLeft	Coordinate	Coordinates within an image
noseRootRight	Coordinate	Coordinates within an image
noseTip	Coordinate	Coordinates within an image
pupilLeft	Coordinate	Coordinates within an image
pupilRight	Coordinate	Coordinates within an image
underLipBottom	Coordinate	Coordinates within an image
underLipTop	Coordinate	Coordinates within an image
upperLipBottom	Coordinate	Coordinates within an image
upperLipTop	Coordinate	Coordinates within an image

FaceRectangle

Object

A rectangle within which a face can be found

Name	Type	Description
height	integer (int32)	The height of the rectangle, in pixels.
left	integer (int32)	The distance from the left edge if the image to the left edge of the rectangle, in pixels.
top	integer (int32)	The distance from the top edge if the image to the top edge of the rectangle, in pixels.
width	integer (int32)	The width of the rectangle, in pixels.

FacialHair

Object

Properties describing facial hair attributes.

Name	Type	Description
beard	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
moustache	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.
sideburns	number minimum: 0 maximum: 1	A number ranging from 0 to 1 indicating a level of confidence associated with a property.

Gender

Enumeration

Possible gender of the face.

Value	Description
male
female

GlassesType

Enumeration

Glasses type if any of the face.

Value	Description
noGlasses
readingGlasses
sunglasses
swimmingGoggles

Hair

Object

Properties describing hair attributes.

Name	Type	Description
bald	number minimum: 0 maximum: 1	A number describing confidence level of whether the person is bald.
hairColor	HairColor[]	An array of candidate colors and confidence level in the presence of each.
invisible	boolean	A boolean value describing whether the hair is visible in the image.

HairColor

Object

Hair color and associated confidence

Name	Type	Description
color	HairColorType	Name of the hair color.
confidence	number minimum: 0 maximum: 1	Confidence level of the color

HairColorType

Enumeration

Name of the hair color.

Value	Description
unknown
white
gray
blond
brown
red
black
other

HeadPose

Object

Properties indicating head pose of the face.

Name	Type	Description
pitch	number
roll	number
yaw	number

Makeup

Object

Properties describing the presence of makeup on a given face.

Name	Type	Description
eyeMakeup	boolean	A boolean value describing whether eye makeup is present on a face.
lipMakeup	boolean	A boolean value describing whether lip makeup is present on a face.

Mask

Object

Properties describing the presence of a mask on a given face.

Name	Type	Description
noseAndMouthCovered	boolean	A boolean value indicating whether nose and mouth are covered.
type	MaskType	Mask type if any of the face

MaskType

Enumeration

Mask type if any of the face

Value	Description
noMask
faceMask
otherMaskOrOcclusion
uncertain

Noise

Object

Properties describing noise level of the image.

Name	Type	Description
noiseLevel	NoiseLevel	An enum value indicating level of noise.
value	number minimum: 0 maximum: 1	A number indicating level of noise level ranging from 0 to 1. [0, 0.25) is under exposure. [0.25, 0.75) is good exposure. [0.75, 1] is over exposure. [0, 0.3) is low noise level. [0.3, 0.7) is medium noise level. [0.7, 1] is high noise level.

NoiseLevel

Enumeration

An enum value indicating level of noise.

Value	Description
Low
Medium
High

Occlusion

Object

Properties describing occlusions on a given face.

Name	Type	Description
eyeOccluded	boolean	A boolean value indicating whether eyes are occluded.
foreheadOccluded	boolean	A boolean value indicating whether forehead is occluded.
mouthOccluded	boolean	A boolean value indicating whether the mouth is occluded.

QualityForRecognition

Enumeration

Properties describing the overall image quality regarding whether the image being used in the detection is of sufficient quality to attempt face recognition on.

Value	Description
Low
Medium
High

RecognitionModel

Enumeration

Name of recognition model. Recognition model is used when the face features are extracted and associated with detected faceIds, (Large)FaceList or (Large)PersonGroup. A recognition model name can be provided when performing Face - Detect or (Large)FaceList - Create or (Large)PersonGroup - Create. The default value is 'recognition_01', if latest model needed, please explicitly specify the model you need.

Value	Description
recognition_01
recognition_02
recognition_03
recognition_04

Share via

Face - Detect With Stream

URI Parameters

Request Header

Request Body

Responses

Security

Ocp-Apim-Subscription-Key

Examples

Detect with stream example

Sample request

Sample response

Definitions

Accessory

AccessoryType

APIError

Blur

BlurLevel

Coordinate

DetectedFace

DetectionModel

Emotion

Error

Exposure

ExposureLevel

FaceAttributes

FaceAttributeType

FaceLandmarks

FaceRectangle

FacialHair

Gender

GlassesType

Hair

HairColor

HairColorType

HeadPose

Makeup

Mask

MaskType

Noise

NoiseLevel

Occlusion

QualityForRecognition

RecognitionModel