Calling Remote API Fails with HTTP 500 Error - Using Credentials Manager

Question

Hi

We're experiencing a problem connecting to remote APIs when using Azure API Gateway service (deployed in Internal VNet mode) and retrieving token from the Credentials Manager. The flow of request is this:

Azure Logic App--> Call an API configured in local APIM instance--> API Operation has an inbound policy configured to request token from Credentials Manager--> Credential Manager sends the token request to remote endpoint (shows "connected")--> local API Operation sends a request to remote API.

The flow results in HTTP 500 error (Authentication Failed because the remote party has closed the transport stream).

The remote API is front-ended by AWS CloudFront - when we resolve the FQDN of the remote API using tools such nslookup and MXToolbox, we're seeing completely different IP Addresses returned by the two - is this expected behaviour?

We have NSG and Azure Firewall along the way (when the local API Operation initiates the connection to remote API endpoint) to lock down specific destinations to which APIM can talk - NSGs being layer 3 and 4 constructs, only allow specifying specific IP Addresses, which really limits our ability to allow destinations where the IP address is not static (subject to change).

Questions / clarifications:

1. Given the scenario, what could be some of the possible causes and areas to investigate?
2. What source IP address is used by Azure APIM Credentials Manager when it connects to remote token endpoint? Is that APIMs Public IP address?

Thanks

Taranjeet Singh

Answer 1

@Taranjeet Malik

it looks like you're facing an HTTP 500 error when your Azure API Management (APIM) instance interacts with your remote API. That's definitely frustrating! Let's break it down and look into possible causes and some specific areas you might want to investigate.

Possible Causes and Areas to Investigate:

Token Generation Failure: Ensure that the Credentials Manager is correctly configured to obtain the token, and check that the endpoint it connects to is operational.

Network Configuration: As you mentioned, you have NSGs and Azure Firewall in place. Verify that they allow outbound traffic from APIM to the location of your remote API. Also, check if any network policies block the request.

Service Availability: Since you're connecting to a service fronted by AWS CloudFront, ensure that the remote API is properly configured and available. If there are issues on the AWS side, it could lead to transport stream closures.

Transport Protocol Issues: The issue you're seeing might also be related to the TLS version being used. Ensure that both your APIM instance and the remote API support the same version of TLS.

IP Address Resolution: The different IPs seen through nslookup and MXToolbox could be due to round-robin DNS or load balancing configurations of AWS CloudFront. This is often expected behavior and could induce inconsistencies if your NSG rules are too strict.

Source IP Address of APIM Credentials Manager:

When your APIM Credentials Manager connects to a remote token endpoint, it should use the public IP address of your APIM instance for outbound calls. You can confirm this by checking the outbound IP addresses of your APIM instance in the Azure portal or use the Azure CLI.