Edit

Face recognition

  • Article

This article explains the concept of Face recognition, its related operations, and the underlying data structures. Broadly, face recognition is the process of verifying or identifying individuals by their faces. Face recognition is important in implementing the identification scenario, which enterprises and apps can use to verify that a (remote) user is who they claim to be.

You can try out the capabilities of face recognition quickly and easily using Vision Studio.

Try Vision Studio

Face recognition operations

Caution

Face service access is limited based on eligibility and usage criteria in order to support our Responsible AI principles. Face service is only available to Microsoft managed customers and partners. Use the Face Recognition intake form to apply for access. For more information, see the Face limited access page.

PersonGroup creation and training

You need to create a PersonGroup or LargePersonGroup to store the set of people to match against. PersonGroups hold Person objects, which each represent an individual person and hold a set of face data belonging to that person.

The Train operation prepares the data set to be used in face data comparisons.

Identification

The Identify operation takes one or several source face IDs (from a DetectedFace or PersistedFace object) and a PersonGroup or LargePersonGroup. It returns a list of the Person objects that each source face might belong to. Returned Person objects are wrapped as Candidate objects, which have a prediction confidence value.

Verification

The Verify operation takes a single face ID (from a DetectedFace or PersistedFace object) and a Person object. It determines whether the face belongs to that same person. Verification is one-to-one matching and can be used as a final check on the results from the Identify API call. However, you can optionally pass in the PersonGroup to which the candidate Person belongs to improve the API performance.

The recognition operations use mainly the following data structures. These objects are stored in the cloud and can be referenced by their ID strings. ID strings are always unique within a subscription, but name fields may be duplicated.

See the Face recognition data structures guide.

Input requirements

Use the following tips to ensure that your input images give the most accurate recognition results:

  • The supported input image formats are JPEG, PNG, GIF (the first frame), BMP.
  • The image file size should be no larger than 6 MB.
  • Some faces might not be recognized because of photo composition, such as:
    • Images with extreme lighting, for example, severe backlighting.
    • Obstructions that block one or both eyes.
    • Differences in hair type or facial hair.
    • Changes in facial appearance because of age.
    • Extreme facial expressions.
  • You can utilize the qualityForRecognition attribute in the face detection operation when using applicable detection models as a general guideline of whether the image is likely of sufficient quality to attempt face recognition on. Only "high" quality images are recommended for person enrollment and quality at or above "medium" is recommended for identification scenarios.

Next steps

Now that you're familiar with face recognition concepts, Write a script that identifies faces against a trained PersonGroup.