Working Llama example for Container Apps?

Question

Working Llama example for Container Apps?

Zach Howell 45

I tried creating a Container App running Llama 3 by following this tutorial:
[https://learn.microsoft.com/en-us/azure/container-apps/serverless-gpu-nim?tabs=bash ](https://learn.microsoft.com/en-us/azure/container-apps/serverless-gpu-nim?tabs=bash

)

However my container app did not start, instead giving "Activation failed" errors. I looked at some logs but couldn't figure out an exact reason for the failure.. AI models don't give the best logging errors at the best of times, and it was not 100% clear I was getting all the right logs in ACA.

I filed https://github.com/microsoft/azure-container-apps/issues/1583 as well on the tutorial.

But I don't necessarily need that tutorial fixed; I'd just like to run Llama 3 on ACA. I tried also with a command like:
az containerapp create --name hzcontllama3 --resource-group $RESOURCE_GROUP --environment $CONTAINERAPPS_ENVIRONMENT --image $ACR_NAME.azurecr.io/$CONTAINER_AND_TAG --cpu 24 --memory 220 --target-port 8000 --ingress external --secrets hftoken=$HF_TOKEN --env-vars hftoken=secretref:hftoken --registry-server $ACR_NAME.azurecr.io --workload-profile-name LLAMA_PROFILE --query properties.configuration.ingress.fqdn --args "--model-id=meta-llama/Llama-3.2-1B-Instruct --max-concurrent-requests 1"

``where I used us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310 , a container which worked for me on Cloud Run from their tutorial: https://cloud.google.com/blog/products/ai-machine-learning/how-to-deploy-llama-3-2-1b-instruct-model-with-google-cloud-run

What am I doing wrong? Anyone have a working example using Llama (3?) for Azure Container Apps? How best should I debug erorrs?

Pashikanti Kumar 1,725 Reputation points Microsoft External Staff Moderator

2025-10-13T19:28:46.1333333+00:00
Hi Zach Howell,

Thank you for posting your question in the Microsoft Q&A forum

Allocate enough CPU (e.g., 24 vCPUs) and large memory (e.g., 220 GiB) to meet Llama 3’s model requirements

Map the correct port (usually 8000 or as specified by container).

Pass necessary secrets as environment variables securely.

az containerapp create \ --name llama3-app \ --resource-group $RESOURCE_GROUP \ --environment $CONTAINERAPPS_ENVIRONMENT \ --image us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310 \ --cpu 24 \ --memory 220Gi \ --target-port 8000 \ --ingress external \ --secrets hftoken=$HF_TOKEN \ --env-vars hftoken=secretref:hftoken \ --registry-server us-docker.pkg.dev \ --workload-profile-name Consumption-GPU-NC24-A100 \ --args "--model-id=meta-llama/Llama-3.2-1B-Instruct --max-concurrent-requests 1"

Please share your logs or error details if issues persist after verifying these steps, and I can help analyze further. This approach will get you a working Llama 3 deployment on Azure Container Apps.To run Llama 3 successfully on Azure Container Apps (ACA), follow these key steps:- Use an ACA environment configured for serverless GPU workloads (e.g., Consumption-GPU-NC24-A100), as Llama 3 requires GPU acceleration.

Use a GPU-enabled container image prebuilt for Llama inference, like us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310.

Allocate sufficient CPU (e.g., 24 cores) and memory (e.g., 220 GiB) according to Llama 3 model requirements.

Pass necessary secrets (like Hugging Face tokens) securely via ACA secrets and environment variables.

Use the Azure CLI to create the container app with appropriate flags for image, CPU, memory, ingress, secrets, environment variables, workload profile for GPUs, and command-line args to specify model ID and concurrency.

Use az containerapp logs to view container start logs to debug issues. Check for GPU driver errors, resource allocation failures, or container crashes.

Validate your container image locally or on other cloud platforms before ACA deployment to ensure container readiness.

Reference

Deploying Llama3 on Azure

Tutorial: Deploy an NVIDIA Llama3 NIM to Azure Container Apps | Microsoft Learn

Kindly let us know if the above helps or you need further assistance on this issue.

Please "Upvote" if the information helped you. This will help us and others in the community as well.

Thanks
Zach Howell 45 Reputation points

2025-10-13T19:45:56.87+00:00

This mostly looks like you formatted the command I ran (+ formatted memory as 220Gi rather than 220 ) & said "this looks right". However I appreciate I haven't given you that much to go on, and I do appreciate getting a response at all. I have some logs here: https://imgur.com/a/9nnKum8

Some possible highlights include:

Profile yadayada supports LoRA, but LoRA is disabled. Skipping profile.

Listening on :80 ``

Target resource doesn't exist

Probe of StartUp failed with status code: 1

But I don't really know what I'm looking for or see anything actionable.
Rupesh Asati 1,020 Reputation points Microsoft External Staff Moderator

2025-10-21T17:50:53.4066667+00:00

Hi Zach Howell,

I hope you checked the last response, and it was helpful. Please do let me know if you need any further assistance on this and also share the information via private message.

Thanks

1 answer

Your answer

Zach Howell 45 Reputation points

2025-10-13T19:45:56.87+00:00

This mostly looks like you formatted the command I ran (+ formatted memory as 220Gi rather than 220 ) & said "this looks right". However I appreciate I haven't given you that much to go on, and I do appreciate getting a response at all. I have some logs here: https://imgur.com/a/9nnKum8

Some possible highlights include:

Profile yadayada supports LoRA, but LoRA is disabled. Skipping profile.

Listening on :80 ``

Target resource doesn't exist

Probe of StartUp failed with status code: 1

But I don't really know what I'm looking for or see anything actionable.
Rupesh Asati 1,020 Reputation points Microsoft External Staff Moderator

2025-10-21T17:50:53.4066667+00:00

Hi Zach Howell,

I hope you checked the last response, and it was helpful. Please do let me know if you need any further assistance on this and also share the information via private message.

Thanks

Answer 1

Hi Zach Howell,

Based on your logs and the error "Probe of StartUp failed with status code: 1" plus the message "Target resource doesn't exist," here are focused troubleshooting steps and solutions for running Llama 3 on Azure Container Apps successfully

The startup probe failing usually means the container app’s readiness/liveness probe can't correctly detect your container is ready.
ACA uses probes on ports (default 80 or configured port). Make sure your container listens on the correct port and responds with a healthy HTTP status.
Action: Confirm that your app inside the container is listening on the ACA-configured target port (e.g., 80 or 8000).
If your app listens on port 8000, configure the container app ingress and probes to use port 8000 explicitly.

Mostly this is a port/probe mismatch or container startup failure. Align container listening port with ACA ingress and probes, allocate the required GPU resources, and validate logs carefully to fix activation errors.

Target resource doesn't exist

This error indicates your application is trying to access a resource (e.g., file, directory, or external service) that is unavailable or misconfigured

Probe Configuration

The startup probe failure suggests ACA is unable to confirm that your app is ready. Ensure the probe is correctly configured.

Reference

az containerapp logs | Microsoft Learn

Enable Application Insights for deeper telemetry and debugging:

Application logging in Azure Container Apps | Microsoft Learn

Kindly let us know if the above helps or you need further assistance on this issue.

Please "Upvote" if the information helped you. This will help us and others in the community as well.

Thanks

Share via

Working Llama example for Container Apps?

1 answer

Your answer