Breyta

Deila með

Backends in API Management

APPLIES TO: All API Management tiers

A backend (or API backend) in API Management is an HTTP service that implements your front-end API and its operations.

When you import certain APIs, API Management automatically configures the API backend. For example, API Management configures the backend web service when importing:

For other APIs, such as APIs from Azure services, you import an Azure resource without specifying the backend service explicitly. Examples include:

API Management also supports using other resources as an API backend, such as:

For these backends, you can create a backend entity in API Management and reference it in your APIs.

Benefits of backends

API Management supports backend entities so you can manage the backend services of your API. A backend entity encapsulates information about the backend service, promoting reusability across APIs and improved governance.

Use backends for one or more of the following:

  • Authorize the credentials of requests to the backend service
  • Take advantage of API Management functionality to maintain secrets in Azure Key Vault if named values are configured for header or query parameter authentication
  • Define circuit breaker rules to protect your backend from too many requests
  • Route or load-balance requests to multiple backends

Configure and manage backend entities in the Azure portal, or by using Azure APIs or tools.

Create a backend

You can create a backend in the Azure portal, or by using Azure APIs or tools.

Note

When you import certain APIs, such as APIs from Microsoft Foundry or other AI services, API Management automatically configures a backend entity.

To create a backend in the portal:

  1. Sign in to the portal and go to your API Management instance.
  2. In the left menu, select APIs > Backends > + Create new backend.
  3. On the Backend page, complete the following steps:
    1. Enter a Name for the backend and optional Description.
    2. Select a Backend hosting type, such as Azure resource for an Azure resource like a Function App or Logic App, Custom URL for a custom service, or a Service Fabric cluster.
    3. In Runtime URL, enter the URL of the backend service that API requests are forwarded to.
    4. Under Advanced, optionally disable certificate chain or certificate name validation for the backend.
    5. Under Add this backend service to a backend pool, optionally select or create a load-balanced pool for the backend.
    6. Under Circuit breaker rule, optionally configure a circuit breaker for the backend.
    7. Under Authorization credentials, optionally configure credentials to authorize access to the backend. Options include a request header, query parameter, client certificate, or system-assigned or user-assigned managed identity configured in the API Management instance.
    8. Select Create.

After creating a backend, you can update the backend settings at any time. For example, you can add a circuit breaker rule, change the runtime URL, or add authorization credentials.

Configure managed identity for authorization credentials

You can use a system-assigned or user-assigned managed identity configured in the API Management instance to authorize access to the backend service. To configure a managed identity for authorization credentials, complete the following steps:

  1. In the Authorization credentials section of the backend configuration, select the Managed identity tab, and select Enable.

  2. In Client identity, select either System assigned identity or a user-assigned identity that you configured in your instance.

  3. In Resource ID, enter a target Azure service or the application ID of your own Microsoft Entra application representing the backend. For example, enter https://cognitiveservices.azure.com for Azure OpenAI service.

    For more examples, see the authentication-managed-identity policy reference.

  4. Select Create.

Note

Also assign the managed identity the appropriate permissions or an RBAC role to access the backend service. For example, if the backend is an Azure OpenAI service, assign the managed identity the Cognitive Services User role.

Configure certificates for authorization credentials

You can secure gateway access to the backend service by using mutual TLS authentication with client certificates or custom certificate authority (CA) certificates.

Configure client certificate

If the backend service is secured with a certificate issued by a well-known CA, you can add a client certificate in the backend entity:

  1. Add the certificate to the API Management instance. You can reference a certificate that's managed in Azure Key Vault or upload a PFX file.
  2. In the Authorization credentials section of the backend configuration, select the Client certificates tab.
  3. In the dropdown, select the client certificate that you want to use.
  4. Select Create.

Configure CA certificate

If the backend service uses a custom CA certificate, you can reference the custom CA certificate in the backend entity. You might need to do this step to establish trust for the backend server certificate - for example, with self-signed certificates, untrusted root certificates, or partial certificate chains.

Note

Currently, you can only configure CA certificate details in a backend entity in the v2 tiers.

To add CA certificate details, follow these steps:

  1. In the Authorization credentials section of the backend configuration, select CA Certificates.
  2. Select + Add CA certificate details.
  3. In the Add CA certificate pane, select one of the following options:
    • Certificate thumbprint - Enter the thumbprint (a SHA-1, SHA-256, or SHA-512 hash) of a custom CA certificate.
    • Subject name and issuer thumbprint - Enter the subject name that uniquely identifies the CA and the thumbprint of the CA.
  4. Select Add.
  5. Select Create.

Screenshot of configuring CA certificate details for a backend in the portal.

Note

When you configure details of a custom CA certificate in the backend entity, API Management always validates the certificate name and certificate chain, regardless of whether you enable or disable validation settings in the backend's backendTlsProperties.

Reference backend using set-backend-service policy

After creating a backend, you can reference the backend identifier (name) in your APIs. Use the set-backend-service policy to direct an incoming API request to the backend. If you already configured a backend web service for an API, you can use the set-backend-service policy to redirect the request to a backend entity instead. For example:

<policies>
    <inbound>
        <base />
        <set-backend-service backend-id="myBackend" />
    </inbound>
    [...]
<policies/>

Note

Alternatively, you can use base-url. Usually, the format is https://backend.com/api. Avoid adding a slash at the end to prevent misconfigurations. Typically, the base-url and HTTP(S) endpoint value in the backend should match to enable seamless integration between frontend and backend. Note that API Management instances append the backend service name to the base-url.

You can use conditional logic with the set-backend-service policy to change the effective backend based on location, gateway that was called, or other expressions.

For example, here is a policy to route traffic to another backend based on the gateway that was called:

<policies>
    <inbound>
        <base />
        <choose>
            <when condition="@(context.Deployment.Gateway.Id == "factory-gateway")">
                <set-backend-service backend-id="backend-on-prem" />
            </when>
            <when condition="@(context.Deployment.Gateway.IsManaged == false)">
                <set-backend-service backend-id="self-hosted-backend" />
            </when>
            <otherwise />
        </choose>
    </inbound>
    [...]
<policies/>

Tip

API Management also automatically detects and uses backend entities when it receives API requests. At runtime, if there's a backend entity that matches the URL of a backend service that API Management is sending a request to, it uses the backend entity. You don't have to explicitly use set-backend-service.

Circuit breaker

API Management exposes a circuit breaker property in the backend resource to protect a backend service from being overwhelmed by too many requests.

  • The circuit breaker property defines rules to trip the circuit breaker, such as the number or percentage of failure conditions during a defined time interval and a range of status codes that indicate failures.
  • When the circuit breaker trips, API Management stops sending requests to the backend service for a defined time, and returns a 503 Service Unavailable response to the client.
  • After the configured trip duration, the circuit resets and traffic resumes to the backend.

The backend circuit breaker is an implementation of the circuit breaker pattern to allow the backend to recover from overload situations. It augments general rate-limiting and concurrency-limiting policies that you can implement to protect the API Management gateway and your backend services.

Note

  • Currently, the backend circuit breaker isn't supported in the Consumption tier of API Management.
  • Because of the distributed nature of the API Management architecture, circuit breaker tripping rules are approximate. Different instances of the gateway don't synchronize and apply circuit breaker rules based on the information on the same instance.
  • Currently, you can configure only one rule for a backend circuit breaker.

Caution

If you configure an Azure OpenAI service as a backend and the service receives too many requests, it returns a 429 Too Many Requests response status code and a Retry-After header with a value that can be large (for example, 1 day). With Azure OpenAI backends, implement circuit breaker rules to handle the 429 responses and accept the Retry-After duration.

Example

Use the Azure portal, API Management REST API, or a Bicep or ARM template to configure a circuit breaker in a backend. In the following example, the circuit breaker in myBackend in the API Management instance myAPIM trips when there are three or more 5xx status codes indicating server errors in 1 hour.

The circuit breaker in this example resets after 1 hour. If a Retry-After header is present in the response, the circuit breaker accepts the value and waits for the specified time before sending requests to the backend again.

  1. In the Azure portal, go to your API Management instance.
  2. In the left menu, select APIs > Backends > your backend.
  3. In the backend page, select Settings > Circuit breaker settings > Add new.
  4. In the Create new circuit breaker page, configure the rule:
    • Rule name: Enter a name for the rule, such as myBackend.
    • Failure count: Enter 3.
    • Failure interval: Leave the default value of 1 hour.
    • Failure status code range: Select 500 - 599.
    • Trip duration: Leave the default value of 1 hour.
    • Check 'Retry-After' header in HTTP response: Select True (Accept).

Load-balanced pool

API Management supports backend pools when you want to implement multiple backends for an API and load-balance requests across those backends. A pool is a collection of backends that are treated as a single entity for load balancing.

Use a backend pool for scenarios such as the following scenarios:

  • Spread the load to multiple backends, which might have individual backend circuit breakers.
  • Shift the load from one set of backends to another set for upgrade (blue-green deployment).

Note

  • You can include up to 30 backends in a pool.
  • Because of the distributed nature of the API Management architecture, backend load balancing is approximate. Different instances of the gateway don't synchronize and load balance based on the information on the same instance.

Load balancing options

API Management supports the following load balancing options for backend pools:

Load balancing option Description
Round-robin Requests are distributed evenly across the backends in the pool by default.
Weighted Assign weights to the backends in the pool, and distribute requests based on the relative weight of each backend. Useful for scenarios such as blue-green deployments.
Priority-based Organize backends into priority groups. Send requests to higher priority groups first; within a group, distribute requests evenly or according to assigned weights.

Note

The API Management service uses backends in lower priority groups only when all backends in higher priority groups are unavailable because circuit breaker rules are tripped.

Session awareness

With any of the preceding load balancing options, you can enable session awareness (session affinity) to ensure that all requests from a specific user during a session go to the same backend in the pool. API Management sets a session ID cookie to maintain session state. This option is useful, for example, in scenarios with backends such as AI chat assistants or other conversational agents to route requests from the same session to the same endpoint.

Note

Session awareness in load-balanced pools is being released first to the AI Gateway Early update group.

Manage cookies for session awareness

When you use session awareness, the client must handle cookies appropriately. The client needs to store the Set-Cookie header value and send it with subsequent requests to maintain session state.

You can use API Management policies to help set cookies for session awareness. For example, for the case of the Assistants API (a feature of Azure OpenAI in Microsoft Foundry Models), the client needs to keep the session ID, extract the thread ID from the body, and keep the pair and send the right cookie for each call. Moreover, the client needs to know when to send a cookie or when not to send a cookie header. These requirements can be handled appropriately by defining the following example policies:

<policies>
  <inbound>
    <base />
    <set-backend-service backend-id="APIMBackend" />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <set-variable name="gwSetCookie" value="@{
      var payload = context.Response.Body.As<JObject>();
      var threadId = payload["id"];
      var gwSetCookieHeaderValue = context.Request.Headers.GetValueOrDefault("SetCookie", string.Empty);
      if(!string.IsNullOrEmpty(gwSetCookieHeaderValue))
      {
        gwSetCookieHeaderValue = gwSetCookieHeaderValue + $";Path=/threads/{threadId};";
      }
      return gwSetCookieHeaderValue;
    }" />
    <set-header name="Set-Cookie" exists-action="override">
      <value>Cookie=gwSetCookieHeaderValue</value>
    </set-header>
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Example

Use the portal, API Management REST API, or a Bicep or ARM template to configure a backend pool. In the following example, the backend myBackendPool in the API Management instance myAPIM is configured with a backend pool. Example backends in the pool are named backend-1 and backend-2. Both backends are in the highest priority group; within the group, backend-1 has a greater weight than backend-2.

  1. In the Azure portal, go to your API Management instance.
  2. In the left menu, select APIs > Backends > your backend.
  3. In the Backends page, select the Load balancer tab.
  4. Select + Create new pool.
  5. In the Create new load-balanced pool page, enter the following information:
    • Name: Enter a name for the pool such as myBackendPool.
    • Description: Optionally enter a description.
    • Add backends to pool: Select one or more backends to add to the pool.
    • Backend weight and priority: Select Customize weight and priority to configure the weight and priority of each backend in the pool. For example, if you added two backends named backend-1 and backend-2, set the weight of backend-1 to 3 and the weight of backend-2 to 1, and set the priority of both backends to 1.
    • Select Create.

Limitations

Related content

  • Last updated on