Share via

AKS Cluster - Mutating admissions webhook does not trigger after a cluster restart

Ruokun Niu 20 Reputation points Microsoft Employee
2023-04-28T20:39:57.6533333+00:00

Hi,

For one of my projects, I am using the sidecar injection feature from dapr on a AKS cluster. If I try to restart an AKS cluster, all of the pods get de-ployed correctly after the cluster restarted, but these pods do not have sidecars injected.

I have reached out to the dapr team and they believed that this is a issue related to the mutating admission webhooks. They do not seem to trigger on a cluster restart.

I am wondering if the AKS team can indeed confirm this is an issue (that mutating admissions webhooks do not trigger on a cluster restart)? Thanks!

Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,999 questions
Accepted answer
  1. KarishmaTiwari-MSFT 19,032 Reputation points Microsoft Employee
    2023-05-31T21:15:04.8233333+00:00

    @Ruokun Niu I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

    Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer. Accepted answers show up at the top, resulting in improved discoverability for others.

    Issue: Customer shared - "AKS Cluster - Mutating admissions webhook does not trigger after a cluster restart. I was working with Dapr (https://dapr.io/) sidecars in a kubernetes cluster, where some of my pods have a sidecar injection. However, when I restart an AKS cluster, sometimes sidecar injection would fail, and I had to manually restart the pods to get it working again.

    I talked to folks in Dapr and they suspected that it might be due to Mutating admissions webhooks and suggested me to ask here."

    Solution: Cx shared - "My alternative solution is to use to the dapr injection watchdog (https://docs.dapr.io/concepts/dapr-services/operator/#injector-watchdog). It regularly checks the conditions of the sidecar and automatically triggers a restart if needed."

    If your issue remains unresolved or have further questions, please let us know in the comments how we can assist.

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Ruokun Niu 20 Reputation points Microsoft Employee
    2023-05-26T17:29:21.9766667+00:00

    Hi,

    For my scenario, I was working with Dapr (https://dapr.io/) sidecars in a kubernetes cluster, where some of my pods have a sidecar injection. However, when I restart a AKS cluster, sometimes sidecar injection would fail and I had to manually restart the pods to get it working again.

    I talked to folks in Dapr and they suspected that it might be something that is due to Mutating admissions webhooks and suggested me to ask here.

    My alternative solution is to use to the dapr injection watchdog (https://docs.dapr.io/concepts/dapr-services/operator/#injector-watchdog). It regularly checks the conditions of the sidecar and automatically trigger a restart if needed.