This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 42%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETN%3C/text%3E%3C/svg%3E)
I have a server that supports server-sent events. When running this server locally, everything works as expected.
But as soon as I deploy it in an AKS Kubernetes cluster in front of an Azure Application Gateway for Containers, the SSE connection times out after 60 seconds. After some research, I found the following documentation: https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/server-sent-events?tabs=server-sent-events-gateway-api. There it's stated that you should create a RoutePolicy and configure a timeout of 0s. But doing so doesn't change anything. The observed timeout remains 60s.
I then played around with the RoutePolicy a bit. And in case the timeout is higher than 0s and lower than 300s, the RoutePolicy works. But (as mentioned above) 0s doesn't seem to do anything at all. And any value greater than 300s leads to a 300s timeout. So the 300s appears to be the maximum.
Kubernetes version: 1.29.11.
alb-controller Helm Chart version: 1.3.7
I created a minimal example, which you can find on GitHub: https://github.com/tnotheis/SseServerTest
You can run the SSE server locally with the following command:
dotnet run dotnet run -p .\SseServerTest\SseServerTest.csproj
Afterwards, open http://localhost:5293/sse in your browser. You will receive 1 event per second, which makes it easy to see after which time a timeout occurs.
You will notice that, when running it locally, there will never be a timeout.
I further added a kubernetes.yaml file, which contains all Kubernetes resources necessary to deploy the application into a cluster with an already existing Azure Application Gateway for Containers. Just make sure to replace the name of the gateway resource as well as the hostname (both places are marked with TODO comments).
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 86%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Hi Timo Notheisen,
Thanks for update providing the information
I'm looking into it and will update on this soon
Hi Timo Notheisen,
Thank you so much for your patience while I am reviewing your query. I came across a similar issue case found posted another community member. They have shared some troubleshooting steps that might help resolve your issue. Would you mind taking a look into below link
https://learn.microsoft.com/en-us/answers/questions/2087406/increase-application-gateway-load-balancer-gateway
Could you please try and let me know if you have any issue tag me in comment will help you as needed.!
Hi @Akshay kumar Mandha ,
this unfortunately doesn't help. It's about the Azure Application Gateway, while we have problems with the Azure Application Gateway for Containers. As far as I understand, these are two completely different products.
Hi Timo Notheisen,
I hope you are doing well,
I understand your concern, and I suggest that community question and answer. It might be helpful to collect logs as input. I know it involving two different products, I got your point this one not helpful for you on my previous comment I'm working into it continuously to address the issue, and I appreciate your patience while I reviewed your query & thanks, being actively giving response on my comments
I have been looking into the timeout issue, and it appears that the default timeout is 60 seconds, as mentioned in the documentation. as per below screenshot
It seems the issue might be related to the configuration being overridden, but you have said there is no older configuration, could you please recheck the setup, including the naming conventions.
Also, regarding the logs, I noticed as you mentioned there were no error logs. Could you kindly share which logs you are already verified. Could you please let me know this will helps more quickly to resolve your issue
It seems the issue might be related to the configuration being overridden, but you have said there is no older configuration, could you please recheck the setup, including the naming conventions.
See the following output. As I said, there is not a single RoutePolicy in the whole cluster, except for the one that's causing the problems.
❯ kubectl get --all-namespaces routepolicies.alb.networking.azure.io
NAMESPACE NAME ACCEPTED AGE
default sseserver-timeout-policy True 7d
Also, regarding the logs, I noticed as you mentioned there were no error logs. Could you kindly share which logs you are already verified. Could you please let me know this will helps more quickly to resolve your issue
I already sent you the output of the kubectl describe of the RoutePolicy. I'm not aware of any other logs I could check. If you let me know here else to look, I will happily do so.
I could send you the output of kubectl describe of the gateway and the HTTPRoute as well, but they all are looking fine as well, so this would just bloat this conversation. And I don't see why they would fail anyway.
Also: why would there be error logs with a timeout below 1s and higher than 300s, but not for any value in between? That just doesn't make sense.
Did you try to deploy the sample application I sent you? (https://github.com/tnotheis/SseServerTest) Because I'm pretty sure it will fail on your end as well.
Hi Timo Notheisen,
I am currently investigating this and trying to address the issue. please allow me some time? I'm finding inputs and will provide update as soon as possible
Hi Timo Notheisen,
I'm just trying to understand how things are set up in AKS. Could you put any screenshot or details how you are accessing the application in the browser? I have tested your application locally, and it works, but I want to understand the AKS setup. Could you please provide more details on this? I'm trying to address the issue quickly.
Hi,
In your AKS, you have to
After that, you can create the Kubernetes services I added to the sample repo and access the sample app via the Application Gateway for Containers.
Hi Timo Notheisen,
I followed the documentation and set up a similar configuration. However, I’m unclear about how the custom DNS hostname is configured in the Application Gateway. and the GitHub repository only includes one YAML file u have used right? and could you please clarify how the custom DNS hostname is set up in your environment? I’m trying to understand it better in order to solve the issue.
Hello @Akshay kumar Mandha ,
I just added the Gateway resource to the kubernetes.yaml file. I also added a self signed TLS certificate. All you have to do now is create an Azure Application Gateway for Containers resource using the "bring your own deployment" strategy (see the docs), take its resource id and replace the <gateway-resource-id> placeholder in the kubernetes.yaml file with it.
I further commented out the hostnames property in the HTTPRoute. You should be able to access the gateway via its IP address instead.
Hi Timo Notheisen,
Thank you for the information. I am currently reviewing it and will update you shortly.Your patience and understanding are greatly appreciated
Hi Timo Notheisen,
I hope you're doing well. Just a few things to confirm:
Was the configuration working for you without the routeTimeout: 0s issue before, or is this the first time you're encountering it? Have you tried any other approaches to resolve it?
I'm investigating this further and need more information to find a solution.
Hi.
Thank you for your efforts.
Was the configuration working for you without the routeTimeout: 0s issue before, or is this the first time you're encountering it?
I didn't use RoutePolicys until recently. And it has never worked since I've been using it.
Have you tried any other approaches to resolve it?
All I did so far I already documented in this thread.
Have you managed to install everything from my GitHub repo on your site? If so, can you reproduce my problem or is everything working as expected?
Hi Timo Notheisen,
Due to some restrictions and limited access, I'm unable to set up in my lab. Could you please remove the default name and only mention the timeouts below? remove Default In yaml file try once
Older one
Default
timeouts:
routeTimeout: 0s
Newer One
timeouts:
routeTimeout: 0s
You mean like this?
apiVersion: alb.networking.azure.io/v1
kind: RoutePolicy
metadata:
name: sseservertest
spec:
targetRef:
kind: HTTPRoute
name: sseservertest
group: gateway.networking.k8s.io
timeouts:
routeTimeout: 0s
Because this doesn't work. It gives me the following error when trying to apply:
Error from server (BadRequest): error when creating "routepolicy.yaml": RoutePolicy in version "v1" cannot be handled as a RoutePolicy: strict decoding error: unknown field "spec.timeouts"
Hi Timo Notheisen,
Yes, this is asked for what I requested. Thank you for looking into it and keeping me informed. I will check with my team.
Hi Timo Notheisen,
Could please give it try once if it is work it could be good if not will go for another alternative method Can you mention the timeout in Kind: HTTProute not in Kind: Routepolicy
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: sseservertest
spec:
hostnames:
- <hostname> # Replace with your hostname
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: <gateway-name> # Replace with your gateway name
rules:
- backendRefs:
- kind: Service
name: sseservertest
port: 8080
matches:
- path:
type: PathPrefix
value: /sse
timeouts:
routeTimeout: 0s
There is no such field on an HTTPRoute . When trying to apply, it Kubernetes gives me the following error message:
strict decoding error: unknown field "spec.rules[0].timeouts.routeTimeout"
There is in fact a timeouts property (see https://gateway-api.sigs.k8s.io/api-types/httproute/#timeouts-optional), but it doesn't have the routeTimeout property you mentioned. Also, it is only available in the experimental channel. I tried it out anyway (with the request and backendRequest properties the docs mention), but it doesn't have any effect at all, no matter which value we configure.
I don't mean to be rude, but I don't have time for a try and error approach with random guesses. So I would appreciate it if you would only make suggestions that are at least based on correct syntax.
Thanks.
Hi Timo Notheisen,
Yeah, I got your point I'm trying on this different approach to get know whether it will be work or not I understand your things Thanks once again for your reply's will find valid document and also check with team once we get update on this will provide update on to you on this, Is that possible or not as per the documentation
Thank you for your patience
@Timo Notheisen this issue has been fixed in ALB Controller v1.6.7. https://learn.microsoft.com/azure/application-gateway/for-containers/alb-controller-release-notes
Please let us know how it goes!
Jack
thanks for the update. Unfortunately it still doesn't work as expected. After setting the timeout to 0s via RoutePolicy, the request is cancelled after 300 seconds. With the old version it would cancel after 60 seconds, so something has changed.
I took a look at the sample application and see you are sending newlines every minute as a pseudo-keep alive. Are you seeing those received by the client? We'll take a look again. Aside, feel free to reach out directly to me via email, we can take a look at your configuration a bit deeper. It's my firstname period lastname at the company I work for.
The sample application doesn't just send newlines. It sends a counter that increments every second, so you can see in the browser how long the request ran before it timed out.
But it turned out that these messages actually served as a keep alive. In my previous test I only called the SSE endpoint of our application (=> not the one of the sample application), which doesn't send events while nothing is happening. I think that's why the connection was closed after 300 seconds.
So I did something to make the SSE endpoint send something, and: it works. No more timeout. So now all I have to do is implement some automatic keep alive logic.
Thank you so much for your help with the RoutePolicy fix!
P.S. Perhaps a small note in the documentation in regards to the keep alive would help others with this problem.
Oh, and one more question out of curiosity: during my research on this topic I found out that there is a possibility to configure a timeout on an HTTPRoute: https://gateway-api.sigs.k8s.io/reference/spec/#httproutetimeouts, which sounds exactly like what the timeouts of the RoutePolicy are doing. Is there any specific reason why Azure didn't implement this and instead has its own solution?
Really appreciate the update and confirmation. Will add a note to the document.
Regarding the HTTPRouteTimeout, one of the challenges was providing backward compatibility for those wanting to use Ingress API. We are aware Gateway API has the standard for defining it, but I don't have anything further to share on that front.
@Jack Stromberg while discussing the keep alive issue with my colleagues, we couldn't find a good reason why this is required. Isn't there a setting we can make to keep "idle" connections open?
Our application runs in a Kubernetes cluster of a different cloud, and there it just works without sending keep alive messages.
Hi Timo Notheisen,
Thank you for your reply.
I understand your latest comment. Regardless of the timeout settings, it always times out after 60 seconds. If that is the case, This suggests that the timeout might be overridden, possibly due to an older configuration with a default timeout of 60 seconds in the Ingress. Could you please check the configuration for any default settings that might be causing this issue? Let me know if you find any configuration conflicts
Additionally, as you mentioned that it works locally but encounters issues when deployed in AKS, could you please provide some information on how you set up the SSE locally? It would be helpful to get an overview of the overall setup and the steps you followed for both the local environment and the AKS deployment. This will allow us to identify any potential configuration conflicts or differences
Hi Timo Notheisen,
Welcome to the Microsoft Q&A Platform! Thank you for asking your question here
Based on your query,
It is working within the limit of 300 seconds. Beyond that, it does not support. Also, when you set the timeout to 0s, it still times out after 60 seconds. Is that your issue what I have understand if yes means If there are any errors or related logs identified, if possible, could you please share the logs related to this issue? better we can assist more quickly on this thing to address the issue
Hi @Akshay kumar Mandha ,
Thank you for your answer.
Your summary is correct, yes. But unfortunately, there are no error logs - except for the ones on the client side, that say that there was a timeout.
The RoutePolicy is Accepted, no matter if the configured timeout is 0s, 300s or 3600s. There aren't even any Kubernetes events (not even after the first apply). Here's the output of kubectl describe:
Name: sseserver-timeout-policy
Namespace: default
Labels: <none>
Annotations: <none>
API Version: alb.networking.azure.io/v1
Kind: RoutePolicy
Metadata:
Creation Timestamp: 2025-02-04T06:08:02Z
Generation: 1
Resource Version: 67404617
UID: f56b2bd2-3e0e-42a1-a9f3-d47b0b20d8e1
Spec:
Default:
Timeouts:
Route Timeout: 0s
Target Ref:
Group: gateway.networking.k8s.io
Kind: HTTPRoute
Name: sseserver
Namespace: default
Status:
Conditions:
Last Transition Time: 2025-02-04T06:08:02Z
Message: Valid RoutePolicy
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Events: <none
This suggests that the timeout might be overridden, possibly due to an older configuration with a default timeout of 60 seconds
There is no other RoutePolicy anywhere in the whole cluster, if that's what you mean. And I'm not aware of any other way to configure a timeout.
could you please provide some information on how you set up the SSE locally?
I just updated my question with a link to a code example, which should answer your question.