Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
With network injection, Microsoft Foundry hosted agent sessions run in micro VMs attached to the configured virtual network (VNet). If a VM can't resolve or reach an endpoint that your agent code calls, the agent can fail at runtime.
This article helps you check the deployment machine's connectivity first, then probe the hosted runtime with the diagnostic agent sample. Start with Set up private networking for Foundry Agent Service if you haven't configured network injection.
The diagnostic agent tests the hosted VM's direct outbound path. It doesn't validate tool calls through the data proxy or authenticate to your dependencies. For those separate traffic paths, see Deep dive into Foundry Agent Service networking.
Prerequisites
- A Foundry account and project configured for hosted agents with network injection.
- The network resources required by your configuration, including private endpoints and DNS zones. See Set up private networking for Foundry Agent Service.
- A Linux deployment machine with access to the private network, Git,
curl,nslookup, andopenssl. If you don't have one, follow the jump box setup in the next section. - Azure CLI and the Azure Developer CLI Foundry extensions, authenticated to the project's tenant.
- The deployment and invocation permissions in Hosted agent permissions. For container deployment, also review Azure Container Registry (ACR) permissions.
- Read access to your private DNS zones, network security groups (NSGs), route tables, and private endpoints. To create a jump box, you also need permission to create the VM and its network resources.
Check connectivity from a jump box
Validate the deployment machine's network path before you deploy the diagnostic agent. A jump box checks access from its own subnet, not from the delegated agent subnet.
Use a Linux VM in a separate, nondelegated subnet of the target VNet. Don't place it in the agent subnet delegated to
Microsoft.App/environments. If you need a VM, follow Create a Linux virtual machine in the Azure portal. Select the existing VNet and a nondelegated subnet, and don't assign a public IP address.Connect through Azure Bastion or an existing private access path. For Bastion, follow the setup and connection steps in the linked quickstart.
From the VM, check the normal service hostname that your client uses, not the
privatelinkalias. Replace<account>with your Foundry account name.nslookup <account>.services.ai.azure.comCompare the returned address with the Foundry private endpoint's IP address in the Azure portal. A private IP alone doesn't prove that the correct endpoint is selected.
Check TLS and HTTP connectivity without disabling certificate validation.
curl --verbose --connect-timeout 5 --max-time 15 \ "https://<account>.services.ai.azure.com/"A completed TLS handshake and an HTTP response show that an HTTPS responder is reachable. The root path doesn't necessarily return
200. Inspect the response before deciding whether the application request succeeds.Repeat these checks for the endpoints that your agent code calls. For container deployment, also test the registry hostname and its data endpoints. Use
/v2/for the registry HTTP check.
If these checks fail, use the layered troubleshooting steps before deploying. If they succeed, continue with the runtime probe. NSGs and routes can differ between the jump box subnet and the agent subnet.
Use the diagnostic agent
Deploy the diagnostic agent to the existing network-injected Foundry account and project. Its probes run in the hosted runtime, rather than on the machine that deploys it. It returns a structured JSON report for the hostnames you supply.
Get the sample and full instructions from diagnostic-agent in the foundry-samples repository.
The probe logic doesn't call a model or require a model deployment. Deployment and invocation still require your existing project details and authentication. The host connectivity probes use DNS queries, TCP connections, TLS handshakes, and unauthenticated HTTPS GET requests.
Deploy the diagnostic agent
Run these steps from the jump box or another machine with private access to your project.
Get the sample and open its deployment directory.
git clone https://github.com/microsoft-foundry/foundry-samples.git cd foundry-samples/samples/python/hosted-agents cd bring-your-own/invocations/diagnostic-agentCreate an
azdenvironment and select your existing project. Replace the placeholders with your resource details.azd env new hosted-agent-diagnostics azd env set AZURE_SUBSCRIPTION_ID "<subscription-id>" azd env set AZURE_LOCATION "<region>" azd ai project add --project-id "<project-resource-id>"Use the full ARM resource ID of the existing project. Don't create a new account or project for this test. For project selection details, see Set the project context.
Review the sample's
azure.yaml. The source-code configuration useslanguage: pythonandcodeConfiguration, so it doesn't require your own ACR. Review the firewall requirements for source-code deployment before deploying.Deploy to the existing project without provisioning infrastructure.
azd deploy --no-promptWait for deployment to complete before invoking the agent. If you choose a container-based configuration instead, follow Deploy with a private Azure Container Registry.
Run a probe
Invoke the agent with the normal service hostnames that your agent code needs to reach. Replace the placeholders in the following commands. Copy the callable Invocations endpoint from the deployment output or azd ai agent endpoint show into INVOCATIONS_URL.
INVOCATIONS_URL="<deployed-invocations-endpoint>"
TOKEN=$(az account get-access-token --resource https://ai.azure.com \
--query accessToken -o tsv)
curl --silent --show-error -X POST "$INVOCATIONS_URL" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"hosts": [
"<account>.services.ai.azure.com"
],
"public_hosts": [],
"include_env_dump": false
}'
The endpoint has the path /api/projects/<project>/agents/<agent>/endpoint/protocols/invocations?api-version=v1. A URL ending in /versions/1 is a management URL, not an invocation endpoint. For endpoint selection, see Invoke a hosted agent.
Add your dependency hostnames to hosts. Setting public_hosts to an empty list skips the default public endpoint checks. Setting include_env_dump to false omits the environment-variable dump.
Inspect summary and the per-probe entries in results, including status, findings, and evidence. An HTTP 200 from the diagnostic agent means it returned a report, not that every probe passed. Treat the report as sensitive network configuration data.
Follow a layered troubleshooting approach
Work through the probe results from the bottom of the network stack upward. Resolve a failure at one layer before you analyze the next, because a lower-layer failure affects the layers above it.
Layer 1: DNS resolution
Check the dns.getaddrinfo and dns.raw entries in results for each host first.
Validate private address resolution
Compare the runtime's DNS answers with the intended private endpoint's IP address, not just with the jump box results.
- If name resolution fails, inspect the DNS error and configured resolver. Missing records, incorrect forwarding, unreachable DNS servers, and transient failures can all cause resolution errors.
- If DNS returns a private IP, compare it with the intended private endpoint's IP address. Don't rely only on the address range.
- If DNS returns a public IP when a private endpoint is expected, check the private DNS zone records, VNet links, and custom DNS forwarding.
Verify the zones required for your configuration, including privatelink.services.ai.azure.com for the Foundry project endpoint. For custom DNS, follow the topology-specific guidance in Private endpoint DNS integration. On-premises resolvers must forward through an Azure DNS forwarder or Azure DNS Private Resolver, not directly to 168.63.129.16.
The dns.raw evidence includes per-resolver answers, DNS response codes, and canonical name (CNAME) chains. A DNS_OK_PUBLIC_FOR_PRIVATE finding means a name points to privatelink.* but resolves to a public IP. Check whether the finding concerns a configured resolver or an extra comparison resolver.
By default, the sample compares the runtime's configured resolvers with Azure DNS at 168.63.129.16. In a hub-and-spoke DNS configuration, the extra resolver can return a public answer. The configured hub resolver can still return the correct private endpoint address.
Set resolvers to [] to query only the configured resolvers. For RESOLVER_DISAGREE, compare each resolver's zone links and forwarding configuration with your intended DNS topology. A difference between configured and comparison resolvers doesn't always indicate a misconfiguration.
To separate DNS failure from basic network reachability, add the endpoint's IP and port to the direct_targets array, such as "<private-endpoint-ip>:443". Inspect conn.direct alongside the hostname probes. A successful direct TCP connection doesn't validate the service hostname, TLS, or authorization.
Check DNS propagation and startup timing
If resolution fails when an invocation starts, enable the dns.propagation probe to observe whether it recovers. It records a timestamped lookup timeline rather than relying on a single lookup.
Reuse
INVOCATIONS_URLand a validTOKENfrom Run a probe. Replace<account>with your account name, then send this request.curl --silent --show-error --no-buffer -X POST "$INVOCATIONS_URL" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -H "Accept: text/event-stream" \ -d '{ "hosts": ["<account>.services.ai.azure.com"], "public_hosts": [], "include_env_dump": false, "include_evidence": true, "dns_propagation_probe": true, "dns_propagation_duration_sec": 30, "dns_propagation_interval_sec": 1, "dns_propagation_threshold_sec": 15, "stream": true }'This request observes DNS for 30 seconds, with a target sampling interval of one second and a 15-second classification threshold. Server-sent events (SSE) provide heartbeats while probes run. Streaming doesn't override client or platform time limits.
Read the report in the SSE
reportevent'sreportproperty. Finddns.propagationin theresultsarray. Inspect itsfindings,metrics.first_success_after_secwhen present, andevidence.attempts.Finding code Interpretation DNS_INITIAL_INSTABILITYLookup failed at invocation start, then succeeded within the configured threshold. DNS_PROPAGATION_DELAYLookup failed at invocation start, then first succeeded after the threshold. DNS_FAILURE_PERSISTEDNo lookup succeeded during the observation window. DNS_INTERMITTENT_OVER_TIMEA lookup failed again after a successful lookup. Investigate intermittency as well as startup timing. Compare the IPs in the successful attempts with the intended private endpoint. Recovery means name resolution succeeded, not necessarily that it returned the required private address.
Increase the observation window up to 120 seconds if needed. These settings don't define a platform propagation guarantee or how long DNS records remain cached.
If DNS recovers, confirm stable private resolution before retrying dependent operations. If failures persist or recur, investigate the resolver, zone links, and forwarding path instead of only increasing the wait.
Investigate intermittent DNS failures
For failures that a single lookup doesn't reproduce, set dns_attempts and gai_attempts to 20. Compare the raw DNS timeout rate with the operating system (OS) resolver's gai_failure_rate. Repeated queries can extend the overall probe duration.
GAI_FAIL_RAW_OKmeans the OS lookup failed while raw DNS returned an answer. Inspectdns.rawfor resolver and path differences.DNS_OK_PRIVATE_INTERMITTENTmeans private answers were returned, but some attempts timed out. A successful private lookup doesn't rule out DNS-path unreliability.DNS_UDP_DROP_TCP_OKmeans UDP queries timed out while TCP DNS succeeded. Investigate UDP/53 handling and response loss.- If sequential queries succeed but OS lookups still fail, enable
parallel_probe. Inspectdns.parallelforPARALLEL_DUAL_LOSS, which reports lost replies during concurrent A and AAAA queries.
Layer 2: TCP connectivity
If DNS resolves but conn.tcp fails on port 443, investigate the network path and destination:
- A timeout can indicate an NSG rule, a user-defined route (UDR), a firewall drop, or an unavailable destination.
- A connection refusal or reset can come from the destination or an intermediate device. It doesn't identify a private endpoint's connection state.
Review the agent subnet's NSG and route table, any network virtual appliance (NVA), and the destination private endpoint. Confirm that the private endpoint connection is approved.
Layer 3: TLS handshake
If conn.tcp succeeds but conn.tls fails, inspect the certificate error and handshake details:
- Certificate validation can fail because of an untrusted issuer, an expired certificate, or a hostname mismatch. TLS inspection is one possible cause.
- Handshake failures can also involve protocol settings, Server Name Indication (SNI), or an intermediate network device. Compare the jump box and runtime results before changing firewall settings.
For a manual check from the jump box, send the original service hostname as SNI:
openssl s_client -connect <account>.services.ai.azure.com:443 \
-servername <account>.services.ai.azure.com \
-verify_hostname <account>.services.ai.azure.com \
-verify_return_error </dev/null
Don't disable certificate verification as a fix or assume that a specific issuer name is required for every Azure service.
Layer 4: HTTP response
If TLS succeeds, inspect conn.http. An HTTP response establishes reachability to an HTTPS responder, but doesn't prove authorization or correct private endpoint routing.
- For an ACR registry, a
401with aWww-Authenticate: Bearerheader on/v2/is an expected unauthenticated challenge. - A
403can reflect network restrictions, missing authorization, or service policy. Read the response body and check the service configuration rather than assigning a single cause. - Other service root paths can return
400,401,403, or404. Confirm the actual API operation with the application's identity after network checks pass.
Troubleshoot container registry access
For container-based deployments, distinguish image push from image pull. The deployment machine pushes the image; the hosted runtime pulls it with a separate identity and network path.
- From the jump box, check the registry and data endpoint hostnames. Verify DNS, port 443, and the registry
/v2/response. - If image push fails, review the deployment machine's network path and ACR permissions.
- If push succeeds but the agent can't start, check runtime registry reachability and image-pull permissions. A successful push doesn't validate the runtime pull path.
- If ACR blocks diagnostic-agent deployment, use the sample's source-code configuration to probe the runtime without your own registry. This bypasses that registry dependency; it doesn't prove the original failure was networking rather than identity, image configuration, or registry policy.
Private ACR support depends on project creation date. Projects created after June 25, 2026 support a private ACR. Earlier projects require public registry access for platform image pulls. Review the current private networking limitations before changing registry access.
Collect evidence for unresolved failures
If the jump box checks pass but runtime probes fail, collect the affected hostname, resolved IPs, probe findings, and UTC timestamps. Include the account and project resource IDs, deployment mode, and relevant NSG or route changes when you contact support. Remove credentials and review the report before sharing it.
Related content
For network setup and the complete diagnostic sample, see the following resources.