Azure AI Document Intelligence : On-premises Containers

Sidharth Chauhan 25

I want to use Azure Document Intelligence containers on-premises. For this, I created a Kubernetes file, referencing the Docker Compose file mentioned here.
The containers are running fine. Now, I need some guidance on how to use my containers to receive requests from the SDK instead of the Azure endpoint. The SDK requires two fields (endpoint and key). If I provide the Azure account keys and endpoint, the request goes to the Azure portal instead of my on-premises container. What should I pass in it so that the request goes to my on-premises containers?

Edit : My Expectations is to reduce latency , currently sdk taking processing time of 10-12s for 2MB of file for both prebuilt model and custom model. In order to reduce latency , i'm giving a try to on-premises container.

dupammi 7,750 Reputation points Microsoft Vendor

2024-05-22T06:43:53.5266667+00:00

Hi @Sidharth Chauhan

Thank you for using the Microsoft Q&A forum.

Using the latest version of the API (e.g., 2024-02-29-preview) has shown significant performance improvements, as per previous customer feedback. Try to use the latest version of the SDK and container to take advantage of these enhancements.

To answer your other question, by configuring the SDK to use the local endpoint and key, and ensuring your containers are properly set up and exposed, you should be able to direct requests to your on-premises containers and reduce latency.

I hope you understand.
Sidharth Chauhan 25 Reputation points

2024-05-22T07:13:32.85+00:00

Hi @dupammi
Please share respective node sdk version for 2024-02-29-preview .

In the SDK configuration, I provided my on-premises container endpoint and key, but it is still not working as desired. Can you help me raise a support ticket or suggest any other way that might help in this case?
dupammi 7,750 Reputation points Microsoft Vendor

2024-05-22T07:44:47.2966667+00:00

Hi @Sidharth Chauhan

Below is what I tried to repro from my side and used the localhost endpoint instead of the Azure DI endpoint. For endpoint use localhost and credential from Azure portal keys.

For more information, please refer Install and run containers.

Time:

I hope this helps. Thank you.
dupammi 7,750 Reputation points Microsoft Vendor

2024-05-23T05:53:04.4133333+00:00

Hi @Sidharth Chauhan

We haven’t heard from you on the last response and was just checking back to see if you got a chance to check my above latest suggestion.
Sidharth Chauhan 25 Reputation points

2024-05-23T05:58:20.1666667+00:00

Hello @dupammi , thanks for sharing this.
Can you please elaborate more about CPU and Wall Time. Also do you have kubernetes documentation for on-premises container if yes please share.

Basically what i'm using currently i want to use prebuilt-document model for this i have 2 pod in which 2 containers are running , one is of document model and in second its supporting container layout model. In AzureCognitiveServiceLayoutHost env property of document model i gave service endpoint of layout model . But anyhow document model is not able to make connection with layout model. Getting below error in document model while i'm trying document processing:

formrecognizerkvp[0]

{"ClassName":"Microsoft.CloudAI.Containers.VDI.Common.Errors.VdiException","Message":"HttpRequestException","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at Microsoft.CloudAI.Containers.VDI.OnPremises.Common.Handler.FrBaseHandler.HttpPostAsync(String requestUri, HttpContent content)\n at Microsoft.CloudAI.Containers.VDI.OnPremises.Common.Handler.Fr30GaUpstreamHandler.PostUpstreamInternalRequest(String modelId, Byte[] stream, ServiceMessage message)\n at Microsoft.CloudAI.Containers.VDI.OnPremises.Common.Handler.Fr30GaUpstreamHandler.HandleMessageAsync(ServiceMessage message)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"Microsoft.CloudAI.Containers.VDI.OnPremises.Common","WatsonBuckets":null} SubscriptionId='' RequestId='f8914aa3-588f-466f-936a-33abe4f6498f' Timestamp='' buildVersion='1.2.367.0-20230707.6-27dcd7ae972075713bb9e4477fe50c6e623f1596' hostId='azure-document-intelligence-cognitive-service-document-869cjvhl' serviceName='formrecognizerkvp' namespace='vdi' Namesapce='vdi' fileSizeInBytes='2139876' processedPageIndex='0' toSecondaryQueue='False' totalPageCount='1' fileType='Png' apiName='document-2022-08-31' ocrModelName='2023-07-31' scenario='fr_async_layout' resultNameEnum='UpstreamFRResult' ocrResolution='Standard' errorCode='HttpRequestException'
Sidharth Chauhan 25 Reputation points

2024-05-23T06:28:38.29+00:00

Also here on microsoft registory https://mcr.microsoft.com/en-us/product/azure-cognitive-services/form-recognizer/document-3.0/tags
Latest container image for 2024-02-29-preview api version is missing. Please share if i only missing it.
dupammi 7,750 Reputation points Microsoft Vendor

2024-05-23T07:20:52.1233333+00:00
Hi @Sidharth Chauhan

The error (HttpRequestException) suggests a communication issue between the document model and layout model containers. Ensure both containers are properly configured and running in Kubernetes. Check network configurations and policies to ensure connectivity. Verify the service endpoint for the document model container.

If preview versions are crucial for your development or testing purposes, you might need to explore alternative deployment options, such as deploying the SDK directly in your environment. This is because there were containers only for GA API versions and not the preview ones.

It seems there might be a gap in container availability for preview versions. In that case, it's advisable to utilize containers for generally available (GA) API versions for production environments, as they typically undergo more rigorous testing and are officially supported.

Regarding your question about CPU and Wall Time, here's a brief explanation:

CPU time refers to the amount of time the CPU spends executing a program or a specific piece of code. It is the actual time the CPU spends processing instructions for a specific task.

Wall time, on the other hand, refers to the actual time elapsed from the start of a program to its completion. It includes the time spent waiting for input/output operations, network latency, and other factors that can affect the program's execution time.

If issue still persists, I request you to raise a support case through Azure portal.

I hope you understand. Thank you.

Share via

Azure AI Document Intelligence : On-premises Containers