Partilhar via


Emotion Extraction (U-SQL)

Summary

Cognitive emotion functions detect one or more human faces in an image and get back face rectangles for where in the image the faces are, along with face attributes like emotion. There are two ways in U-SQL to extract emotions from the image:

  • EmotionApplier
  • EmotionExtractor

Important

Use an Extractor rather than an Applier when you have images larger than 4 MB.

EmotionApplier

Arguments

EmotionApplier(
string imgCol = "ImgData", string numCol = "NumFaces", string indexCol = "FaceIndex", string emtCol = "Emotion", string confCol = "Confidence")

For each JPEG image provided as a byte array in the column with the default name ImgData, it returns one row per face detected in the file (column FaceIndex of type int) with additional information about the detected face's bounding box (columns RectX, RectY, Width, Height all of type float), its recognized emotion (column Emotion of type string), the confidence value (column Confidence of type float) and the overall number of faces detected in the image (column NumFaces of type int).

EmotionExtractor

Arguments

EmotionExtractor(
string numCol = "NumFaces", string indexCol = "FaceIndex", string emtCol = "Emotion", string confCol = "Confidence")

For each JPEG file it gets applied to, this U-SQL extractor returns one row per face detected in the file (column FaceIndex of type int) with additional information about the detected face's bounding box (columns RectX, RectY, Width, Height all of type float), its recognized emotion (column Emotion of type string), the confidence value (column Confidence of type float) and the overall number of faces detected in the image (column NumFaces of type int).

Examples

A. EmotionApplier

Extract tags from the image using Image tagging Applier

REFERENCE ASSEMBLY ImageCommon;       
REFERENCE ASSEMBLY ImageEmotion;

// Load images
@images =
    EXTRACT FileName string, 
            ImgData byte[]
    FROM @"/Samples/Images/{FileName}.jpg"
    USING new Cognition.Vision.ImageExtractor();

@emotions_from_applier =
    SELECT FileName,
        Details.NumFaces,
        Details.FaceIndex,
        Details.RectX, Details.RectY, Details.Width, Details.Height,
        Details.Emotion,
        Details.Confidence
    FROM @images 
    CROSS APPLY
        USING new Cognition.Vision.EmotionApplier() AS Details(
            NumFaces int, 
            FaceIndex int, 
            RectX float, RectY float, Width float, Height float, 
            Emotion string, 
            Confidence float);

OUTPUT @emotions_from_applier
TO "/ReferenceGuide/Cognition/Vision/EmotionApplier.txt"
USING Outputters.Tsv(outputHeader: true);

B. EmotionExtractor

Extract face and recognize facial expression using Emotion Extractor

REFERENCE ASSEMBLY ImageCommon;      
REFERENCE ASSEMBLY ImageEmotion;

@emotions_from_extractor =
    EXTRACT FileName string, 
            NumFaces int, 
            FaceIndex int, 
            RectX float, RectY float, Width float, Height float, 
            Emotion string, 
            Confidence float
    FROM @"/Samples/Images/{FileName}.jpg"
    USING new Cognition.Vision.EmotionExtractor();

OUTPUT @emotions_from_extractor
TO "/ReferenceGuide/Cognition/Vision/EmotionExtractor.txt"
USING Outputters.Tsv(outputHeader: true);

See Also