How to make Readiness and Liveness Probes Stable under Inference_Config in Azure Ai Foundry?

Question

While Deploying a custom container for production using Azure Ai Foundry's Prompt Flow, I have got error regarding Liveness and Readiness? Do I need to setup these parameters in the Docker file while making the image or in flow.dag.yaml file or at both places?, And I theoretically know that Liveness means that the container is running or not and Readiness means the container is ready to take input and provide output. But, how I can setup this in Azure Ai Foundry ( Ai Studio Earlier). It will be very helpful if someone and provide me a deep explanation and solution of this issue.

Answer 1

Hi @Lovedeep Singh,

Thank you for reaching out to Microsoft Q&A forum!

To set up readiness and liveness probes for containerized applications in Azure AI Foundry, you need to define these probes in the container configuration. The readiness probe ensures the container is only accessed when it's ready to handle traffic, while the liveness probe checks if the container is running and restarts it if unhealthy. In the container file, you can configure these probes with commands that verify the container's state, such as checking for the existence of a file or responding to an HTTP request.

For example, a readiness probe can check for a file that signals the container is ready, and a liveness probe can verify the container’s health by checking for a different file or endpoint. Once configured, the container won't receive traffic until the readiness probe passes, ensuring that only fully initialized containers handle requests. This approach helps manage container health and availability effectively in Azure AI Foundry.

For more info, please refer to: Configure liveness probes and Configure readiness probes.

Thank you.