Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,900 questions
How to build a load balancer for Cognitive Services
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 7.000000000000001%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENZ%3C/text%3E%3C/svg%3E)
Ning Zhu
131
Reputation points
We have set up two Cognitive Service Instances and we want to know whether that is possible to set up a load balancer for the two instances, so developer no need to switch between them?
{count} vote
-
romungi-MSFT 46,831 Reputation points Microsoft Employee2023-02-06T10:59:40.1633333+00:00 Ning Zhu If you are using a cognitive service resource endpoints directly in your application and looking to balance the outgoing requests to different resources then I don't think there is a ready made offering from cognitive services to redirect traffic to these endpoints. This is mostly managed on the client side and in some cases, customers are using containers for cognitive services and scaling the services with AKS or ACI.
-
Ning Zhu 131 Reputation points
2023-02-09T01:28:35.4566667+00:00 There are several layer 7 load balancer such as Front Door, Application Gateway, nothing can be used here?
-
romungi-MSFT 46,831 Reputation points Microsoft Employee
2023-02-09T13:25:38.94+00:00 I do not have much experience with load balancing applications but for scenarios where you use authentication with the subscription keys as headers for a cognitive resource, a load balancer might still have to route the request to the endpoint that is used in the original request since authentication will fail if the request is routed to the second cognitive service resource.
Sign in to comment
Sign in to answer