Problem with Auto-labeling in Form Recognizer (custom model)

Question

Hello! :)

I have a problem with running auto-labeling in Form Recognizer for our projects that are using custom models.

Using Azure Storage Explorer, I can connect to the blob that contains PDF files that we are using for training without any issues. I can also add new files, download, and delete them.

I can also see all the files on formrecognizer.appliedai.azure.com (custom model). I can create labels, train models, build composite models – in one word, everything works perfectly fine.

Except for one thing - when I try to auto-label a new PDF file based on the composite model that I created before, it fails.

Looking at my network tab on my browser, there's a GET request to URI https://formrecognizer.appliedai.azure.com/custom-model/projects/a154d943-9f9e-40e4-bfa0-0db910490c8f/autoLabel/operations/d0446a51-417d-47ae-b230-789bed6151e6 that fails with the response:

{
    "projectId": "a154d943-9f9e-40e4-bfa0-0db910490c8f",
    "operationId": "d0446a51-417d-47ae-b230-789bed6151e6",
    "createdTime": "2023-08-22T08:07:43.0470449Z",
    "status": "Failed",
    "error": "Failed to send analyze request to Form Recignizer service. Error: Cannot retrieve operation location. Status code: Forbidden",
    "fileList": [
        "1532347.pdf"
    ]
}

So, it seems that I don't have the necessary permissions to execute the operation for autoLabel. Honestly, I have no idea where I can find settings to change it (all firewall/access settings are managed by our admin team, so I can get access to this function; I just need some clues about where I can find the setting to change access to it).

Any pointers on how to fix this problem would be greatly appreciated.

Answer 1

I have spent all day working on this issue myself. The one thing that worked for me was to allow the IP of the Form Recognizer Studio through the firewall of the AI Document Intelligence and associated storage account. I know you mentioned that you do not have access to firewall settings so I would just send them the below link.

Here is the IP: 20.3.165.95

Here is where I found the answer: https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/faq?view=doc-intel-3.1.0#why-am-i-receiving-an-authorizationfailure-error-on-project-sharing--auto-label--or-ocr-upgrade-when-my-document-intelligence-or-storage-account-resource-is-configured-with-a-firewall-

Answer 2

We had a different experience from Matthew's answer. We did try the IP address as stated in the documentation, and referenced by Matthew above, but we also needed to add the doc intelligence service endpoint to the CORS (resource sharing) allowed origins on the storage container blob service (more on that below). We also made sure that we followed the rest of the documentation stated here: https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/quickstarts/studio-custom-project?view=doc-intel-4.0.0#custom-models

You can find your Document Intelligence endpoint by going to your document intelligence service, Expand the Resource Management section, click on 'Keys and Endpoint' and then copy the string value for 'Endpoint' on that screen.

Then, go to your related Storage Account on the Azure portal, expand Settings, click on Resource sharing (CORS), then on your blob service, paste in the endpoint string copied above into the Allowed origins field, select the 8 allowed methods and then * for both headers and finally 120 for max age (or whatever values are appropriate for your environment).

I hope this helps someone out there!