Text Analytics API PII masking Error in some categories in piiCategories

Fatemeh Sharifi (Avanade) 1

Hi,
I am trying to mask Quantities from a body of text using the Text Analytics PII API (as in Skillset in Azure search) and understand to do so I can use the Quantity category, but it gives me the following error while I try to pass "Quantity" in "piiCategories".
"Web Api response status: 'BadRequest', Web Api response details: '{"error":{"code":"InvalidRequest","message":"Invalid Request.","innererror":{"code":"InvalidParameterValue","message":"Invalid value for parameter 'piiCategories': 'Percentage', see valid categories at 'https://aka.ms/pii-cats'"}}}'"
However, when I pass some other categories such as "Person", "Organization" or even "Age" which is a subcategory of "Quantity" I do not get the error and the results are good.
I was wondering if you could let me know what to do so the "Quantity" parameter can become recognized.

romungi-MSFT 41,861 Reputation points Microsoft Employee

2022-09-21T09:14:33.467+00:00

@Fatemeh Sharifi (Avanade) Passing Quantity as the PII category should work with the language service API. However, I am not quite sure which version of the API is called by the cognitive search skill which might be causing this error since some previous versions of the text analytics API had fewer categories. You could try using the modelVersion to 2022-05-01 which is the latest language service API version.

romungi-MSFT 41,861 Microsoft Employee

Also, You could try calling the language service API directly to check if the same category in the request is successful. I can confirm that this works with my request when I call the language service API. For ex:

curl -v -X POST "https://<resource_name>.cognitiveservices.azure.com/language/:analyze-text?api-version=2022-05-01" -H "Content-Type: application/json" -H "Ocp-Apim-Subscription-Key: <key>" --data-ascii "{\"kind\":\"PiiEntityRecognition\",\"piiCategories\":      [\"Quantity\"],\"analysisInput\":{\"documents\":[{\"id\":\"documentId\",\"text\":\"Mateo Gomez, 28-year-old man, suffered a car accident driving near his home on Hollywood Boulevard on August 17th, 2022, and was admitted to Contoso General Hospital in Los Angeles California at 7:45 PM. The patient showed signs of chest trauma indicating possible rib fracture and had difficulty breathing. A chest CT scan and AP X-ray were performed to determine the damage to ribs and lungs. Results showed a pseudoaneurysm of the thoracic aorta with minor fracture to the first and third right ribs. Patient was kept in the ICU where treatment was initiated. A Stent was surgically placed to stabilize the hemorrhage until the blood oxygen level reached 95 percent. The patient was discharged on September 1st, 2022, under the supervision of his caretaker Nickolaus Schulz, passport number: B12345678.\",\"language\":\"en\"}]},\"parameters\":{\"domain\":\"none\"}}"

romungi-MSFT 41,861 Reputation points Microsoft Employee

2022-09-26T12:09:48.667+00:00

@Fatemeh Sharifi (Avanade) Did the direct call to the API return expected results? I think this needs an update to the skill to support all the PII categories.
Fatemeh Sharifi (Avanade) 1 Reputation point

2022-11-10T21:08:00.097+00:00

Thanks @romungi-MSFT for your reply.
Your solution (using Cognitive service) does not give error for Quantity (the one that I got in Azure search) but it does not apply it to the results neither.

I need only one category to be redacted, but this method redacts all the categories even when I have put the Quantity as the piiCategories.
I appreciate if you may have any other suggestion.

For example the following is the outcome of the code you suggested in which all piis are redacted.
romungi-MSFT 41,861 Reputation points Microsoft Employee

2022-11-11T06:44:33.753+00:00

@Fatemeh Sharifi (Avanade) I think the above request had incorrect syntax where PII categories are outside the parameters section. Here is the valid request that should work. Here is a sample to redact only DateTime category.

I see Quantity and IP are still returned as not supported although documentation says it is.