Troubleshoot Microsoft Defender for Containers

This article provides troubleshooting guidance for common deployment and operational issues in Microsoft Defender for Containers across all supported environments.

Azure Kubernetes Service (AKS)

Common deployment issues

• Defender sensor installation fails

  • Symptoms: kubectl get pods -n kube-system -l app=defender shows Defender sensor pods in Pending, CrashLoopBackOff, or Error.
  • Resolution:
    • Insufficient resources: Check node capacity. Use kubectl top nodes to verify if nodes have enough CPU and memory to schedule the sensor.
    • Network egress: Verify your cluster firewall or NSG allows outbound traffic to the required FQDNs.
    • Taints and Tolerations: Ensure node taints aren't preventing the pods from scheduling on specific node pools.

• Missing recommendations

  • Symptoms: Clusters show as "Healthy" but specific recommendations like "AKS clusters should have Defender profile enabled" are missing.

  • Resolution:

    • Wait time: Assessment scans can take up to 24 hours to reflect in the dashboard.
    • Exclusion tags: Check if the resource has the tag ms_defender_container_exclude_sensors = true.
    • Policy Add-on: Ensure the Azure Policy add-on is installed; without it, configuration-based recommendations will not trigger.

• Security Gating Fails to provision

  • Symptoms: Security Gating is enabled, but its admission-controller resources are missing or Admission Monitoring shows no data.

  • Diagnosis: In the affected AKS cluster's Activity log, look for a failed Microsoft.ContainerService/managedClusters/write operation with the error RequestDisallowedByPolicy. Inspect the error details to identify the blocking policy assignment. A policy assignment using the Deny effect can reject provisioning even when the existing cluster is compliant.

  • Policies enforcing the following AKS settings can be involved:

    • Disabled local authentication.

    • Customer-managed encryption keys for operating-system and data disks.

    • Encryption at host for temporary disks and agent-node-pool caches.

    • Private clusters.

  • Resolution:

    • Work with your Azure Policy administrator to resolve the identified conflict using an approved, narrowly scoped change, such as an exemption for the affected cluster.

    • Preserve the cluster's required security configuration. Don't enable local authentication or disable policy enforcement at a broader scope to work around the failure.

    • If provisioning remains incomplete, contact Microsoft Support for the supported recovery procedure.

Vulnerability scan issues

• Missing vulnerability findings for images in Azure Container Registry

  • Symptoms: Vulnerability findings don't appear for images stored in Azure Container Registry.
  • Resolution:
    • Registry scanning: Confirm that the relevant registry scanning capability is enabled for Defender for Containers. In the Azure portal, verify that Registry access is enabled for the relevant scope.
    • Further investigation: If registry scanning is enabled and findings are still missing, open a support case with the registry name, image name, image digest, and expected finding details.

• Missing vulnerability findings for images running on AKS clusters

  • Symptoms: Vulnerability findings don't appear for images that are currently running in AKS workloads.
  • Resolution:
    • Vulnerability scanning: Confirm that the relevant vulnerability scanning capability is enabled for Defender for Containers. Runtime vulnerability findings depend on available scan results for the running image, such as registry scan or disk scan results.
    • Pod inventory collection: Confirm that pod inventory collection is enabled for the cluster. For AKS, pod inventory can be collected by the Defender sensor or by agentless collection, depending on the deployment configuration.
    • Further investigation: If vulnerability scanning and pod inventory collection are enabled but findings are still missing, open a support case with the cluster name, namespace, workload name, image name, and image digest.

Amazon Elastic Kubernetes Service (EKS)

Connector and Discovery

• AWS connector disconnected

  • CloudFormation status: Check the AWS CloudFormation console to ensure the onboarding stack is in the CREATE_COMPLETE state.
  • Permissions: Verify the MDCContainersAgentlessDiscoveryK8sRole role has not been modified. This role is required for Defender for Cloud to discover your clusters via the EKS API.

• EKS clusters not appearing in inventory

  • Kubernetes API access: Ensure this component is toggled On in the AWS connector settings.
  • IAM identity mapping: Ensure the aws-auth ConfigMap has been updated to include the Defender for Cloud role, or that an EKS Access Entry has been created for the role.

Troubleshoot Defender sensor auto provisioning

If automatic provisioning is enabled but the Defender sensor isn't installed on your EKS cluster, check the following conditions with your AWS or Azure administrator.

Automatic provisioning depends on the permissions, network configuration, and resource lifecycle managed in your environment. Your administrator is responsible for maintaining the required access and removing confirmed stale resources when roles or clusters are recreated.

The Defender provisioning IAM role was recreated

Cause: Deleting and recreating the MDCContainersK8sRole IAM role changes its underlying identity, even if its name and ARN remain unchanged. An existing EKS access entry can still reference the previous identity, preventing Defender from authenticating to the cluster.

Diagnosis: Check whether your team or deployment automation recently recreated MDCContainersK8sRole. Compare the role's creation time with the corresponding EKS access entry's creation time:

aws iam get-role --role-name MDCContainersK8sRole --query "Role.CreateDate"
aws eks describe-access-entry --cluster-name <cluster-name> --principal-arn arn:aws:iam::<aws-account-id>:role/MDCContainersK8sRole --region <region> --query "accessEntry.createdAt"

Replace the placeholders with your cluster name, AWS account ID, and region. If your deployment uses a custom role name or path, use that role's name and ARN instead.

An access entry created before the confirmed role recreation supports this diagnosis. Use the entry's creation time, not the access-policy association time.

Resolution: Record the stale entry's settings and associated access policies, then remove only the EKS access entry for the recreated Defender role. Don't delete the IAM role or unrelated access entries.

Keep automatic provisioning enabled and retain the required provisioning permissions so a subsequent attempt can recreate the entry. Restore any required custom settings.

For more information, see Amazon EKS access entry considerations.

Cluster-management automation removes required Kubernetes API access

Cause: Defender provisioning requires network access to the EKS Kubernetes API endpoint. Infrastructure-management tools, such as Crossplane, can reconcile the endpoint allowlist with their declared configuration. If that configuration excludes the required Defender provisioning CIDR ranges, reconciliation can remove ranges added during provisioning and prevent installation from completing.

Diagnosis: In the Amazon EKS console, review the affected cluster's endpoint allowlist. Compare it with the configuration maintained by your infrastructure-management tool and check whether reconciliation removes Defender provisioning ranges added to the allowlist.

Your AWS administrator can review AWS CloudTrail configuration-change events to help identify these changes. If you need the required Defender provisioning ranges for your environment, contact Microsoft Support.

Resolution: Ensure that your infrastructure-management configuration retains the required Defender provisioning CIDR ranges:

• Include the ranges in the declared configuration that manages the endpoint allowlist.
• Alternatively, where your management tool supports it, configure reconciliation to preserve the required Defender entries.
• If you add the ranges manually, also ensure that subsequent reconciliation doesn't remove them. A manual change alone might be overwritten.

Preserve other required allowlist entries and restrict access to the required ranges rather than opening the endpoint to all IP addresses. Keep automatic provisioning enabled after correcting the configuration.

For endpoint configuration instructions, see Configure network access to the Amazon EKS API server endpoint.

A previous Azure Arc registration remains after the EKS cluster is recreated

Cause: Deleting an EKS cluster and creating another with the same name can leave the previous cluster's Azure Arc registration behind. The disconnected registration can interfere with onboarding the replacement cluster.

Diagnosis: Check whether your team recently deleted and recreated the EKS cluster. In Azure, review the associated Arc-enabled Kubernetes resource and its connection status. Compare the current EKS cluster's creation time with the Arc resource's creation and last-connectivity times, and use your deployment history to confirm whether the registration belongs to the deleted cluster.

Resolution: After confirming that the Arc registration belongs to the deleted cluster, your Azure administrator should review its dependent extensions and configurations, then remove the stale registration. Removing only the Defender extension doesn't remove the Arc registration.

Keep Defender automatic provisioning enabled so the replacement cluster can be onboarded. Preserve any Arc registration that is still used by an existing cluster.

For resource-removal guidance, see Clean up Azure Arc-enabled Kubernetes resources.

Vulnerability scan issues

• Missing vulnerability findings for images in Amazon Elastic Container Registry

  • Symptoms: Vulnerability findings don't appear for images stored in Amazon Elastic Container Registry.
  • Resolution:
    • Registry scanning: Confirm that the relevant registry scanning capability is enabled for Defender for Containers. In the AWS connector settings, verify that Registry access is enabled.
    • Further investigation: If registry scanning is enabled and findings are still missing, open a support case with the registry name, image name, image digest, and expected finding details.

• Missing vulnerability findings for images running on EKS clusters

  • Symptoms: Vulnerability findings don't appear for images that are currently running in EKS workloads.
  • Resolution:
    • Vulnerability scanning: Confirm that the relevant vulnerability scanning capability is enabled for Defender for Containers. Runtime vulnerability findings depend on available scan results for the running image, such as registry scan or disk scan results.
    • Pod inventory collection: Confirm that pod inventory collection is enabled for the cluster. Pod inventory can be collected by the Defender sensor or by agentless collection, depending on the deployment configuration.
    • Connector and permissions: Verify that the AWS connector and required cloud permissions are configured correctly.
    • Further investigation: If vulnerability scanning, pod inventory collection, and connector permissions are configured but findings are still missing, open a support case with the cluster name, namespace, workload name, image name, and image digest.

Runtime Alerts

• No control plane alerts generated
  • Audit logging: Audit logs must be enabled for each cluster. Run: aws eks update-cluster-config --name <cluster-name> --logging '{"clusterLogging":[{"types":["audit","authenticator"],"enabled":true}]}'
  • SQS Configuration: Verify that CloudTrail is correctly sending logs to the SQS queue used by the connector. Verify that the SQS ARN is accurate in the connector settings.

Google Kubernetes Engine (GKE)

Posture and Discovery

• GKE Autopilot limitations

  Important

  On GKE Autopilot clusters, you cannot manually configure or override resource limits for the Defender sensor. The sensor is designed to request the minimum resources required by Autopilot's specialized scheduling automatically.

• Service Account errors

  • Service Account Email: Confirm the Service Account email in the Azure portal matches the email generated in the GCP console.
  • IAM Roles: Ensure the Service Account has the container.viewer and container.clusters.update roles at the project level.

Registry assessment issues

• Missing vulnerability findings in Artifact Registry

  • Symptoms: Vulnerability findings don't appear for images stored in Google Artifact Registry.
  • Resolution:
    • Registry scanning: Confirm that the relevant registry scanning capability is enabled for Defender for Containers. In the GCP connector settings, verify that Registry access is enabled.
    • Further investigation: If registry scanning is enabled and findings are still missing, open a support case with the registry name, image name, image digest, and expected finding details.

• Missing vulnerability findings for images running on GKE clusters

  • Symptoms: Vulnerability findings don't appear for images that are currently running in GKE workloads.
  • Resolution:
    • Vulnerability scanning: Confirm that the relevant vulnerability scanning capability is enabled for Defender for Containers. Runtime vulnerability findings depend on available scan results for the running image, such as registry scan or disk scan results.
    • Pod inventory collection: Confirm that pod inventory collection is enabled for the cluster. Pod inventory can be collected by the Defender sensor or by agentless collection, depending on the deployment configuration.
    • Connector and permissions: Verify that the GCP connector and required cloud permissions are configured correctly.
    • Further investigation: If vulnerability scanning and pod inventory collection are enabled but findings are still missing, open a support case with the cluster name, namespace, workload name, image name, and image digest.

Arc-enabled Kubernetes

Arc Connectivity

• Cluster status "Disconnected"
  • Network Egress: Ensure the cluster can reach the required Azure Arc endpoints on TCP 443.
  • Time Sync: Ensure node system time is accurate and synced via NTP. If the node time drifts significantly, the Arc certificate handshake will fail, preventing components from deploying.
  • Proxy configuration: If your environment uses a proxy, ensure the Arc agents are configured with the correct http_proxy and https_proxy settings during the az connectedk8s connect step.

Extension Management

• Defender extension stuck in "Creating" or "Failed"

  • Agent logs: Check the Arc agent logs for detailed errors: kubectl logs -n azure-arc -l app.kubernetes.io/component=cluster-agent
  • Reinstallation: If the extension remains stuck, delete and recreate it via CLI: az k8s-extension delete --cluster-name <name> --resource-group <rg> --cluster-type connectedClusters --name microsoft.azuredefender.kubernetes

• Defender sensor pod "ImagePullBackOff"

  • Symptoms: Pods fail to start with an image pull error.
  • Resolution:
    • MCR Connectivity: Verify nodes can reach mcr.microsoft.com to pull the Defender sensor image.
    • Namespace conflicts: Ensure no other security solutions or admission controllers are interfering with the mdc namespace.

Verification via alert simulation

Use the Kubernetes alerts simulation tool to verify that Defender for Containers can generate alerts for your cluster and send them to Defender for Cloud.

Related content

• Deploy Defender sensor and Azure Policy to clusters using Azure CLI.

• Install Defender sensor using Helm.