Unable to delete HDInsight cluster and resource group.

Question

Unable to delete HDInsight cluster and resource group.

Abhinandan Gupta 0

Unable to delete HDInsight cluster and resource group. I am getting and error and the deletion process keeps running indefinitely.

User's image

Abhinandan Gupta 0 Reputation points

2025-04-17T21:11:05.6433333+00:00

Remove-AzHDInsightCluster: Long running operation failed with status 'Failed'. Additional Info:'ErrorCode: FailedToDeletePerClusterContainerErrorCode; ErrorDescription: Internal server error occurred while processing the request. Please retry the request or contact support.'

This is the error on CLI

2 answers

Your answer

Abhinandan Gupta 0 Reputation points

2025-04-17T21:11:05.6433333+00:00

Remove-AzHDInsightCluster: Long running operation failed with status 'Failed'. Additional Info:'ErrorCode: FailedToDeletePerClusterContainerErrorCode; ErrorDescription: Internal server error occurred while processing the request. Please retry the request or contact support.'

This is the error on CLI

Answer 1

J N S S Kasyap 2,445 Microsoft External Staff Moderator

Hi @Abhinandan Gupta
Based on the error message FailedToDeletePerClusterContainerErrorCode and the associated CLI output, it appears that the deletion process for your HDInsight cluster is encountering issues due to problems with the per-cluster default storage container. The issue arises when the cluster's associated storage container cannot be deleted automatically, causing the deletion process to fail.

To resolve HDInsight cluster deletion issues, navigate to the associated Azure Storage account used as the cluster's default storage. Identify and manually delete the container, which is often named after the. If deletion fails, check your IAM permissions to ensure that you or the system-assigned managed identity has the "Storage Blob Data Contributor" role. Additionally, verify if there are any resource locks applied to the storage account, and ensure that soft delete or immutability policies are not preventing deletion. Disable these settings temporarily if needed to allow the container to be removed.

After removing the storage container try to delete HDInsight cluster in AZURE portal

Please refer the below Microsoft threads and Documentation for to delete HDInsight cluster

https://learn.microsoft.com/en-us/azure/hdinsight/hdinsight-delete-cluster
https://learn.microsoft.com/en-us/answers/questions/2238862/i-want-to-delete-my-hdinsight-cluster
https://learn.microsoft.com/en-us/answers/questions/2200436/azure-hdinsight-cluster-not-getting-deleted

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Abhinandan Gupta 0 Reputation points

2025-04-18T00:39:48.11+00:00

Hey @J N S S Kasyap ,

I have already deleted the container which gets created by default. Now there are no containers in my storage account.
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-18T00:54:09.18+00:00
@Abhinandan Gupta

Use Azure CLI to Force Delete the HDInsight Cluster, This command should bypasses any failed attempts to delete the container:

az hdinsight delete \ --name <your-cluster-name> \ --resource-group <your-resource-group-name> \ --yes

https://learn.microsoft.com/en-us/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-delete

Sometimes ARM gets stuck delete directly using the full resource ID:

az resource delete \ --ids /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.HDInsight/clusters/<cluster-name>

https://learn.microsoft.com/en-us/cli/azure/resource?view=azure-cli-latest#az-resource-delete

If the issue persists, I will be happy to assist you further.
Abhinandan Gupta 0 Reputation points

2025-04-18T01:04:44.8766667+00:00

PS /home/abhinandan> az hdinsight delete --name clusterabhi09 --resource-group bigdatacloud --yes

(InternalServerError) ErrorCode: FailedToDeletePerClusterContainerErrorCode; ErrorDescription: Internal server error occurred while processing the request. Please retry the request or contact support.

Code: InternalServerError

Message: ErrorCode: FailedToDeletePerClusterContainerErrorCode; ErrorDescription: Internal server error occurred while processing the request. Please retry the request or contact support.
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-18T03:10:15.61+00:00
@Abhinandan Gupta
Did You try the below command as mentioned above

az resource delete \ --ids /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.HDInsight/clusters/<cluster-name>

Please let me know once the you ran the above command meanwhile I will try other possiblities to help you further
Abhinandan Gupta 0 Reputation points

2025-04-18T15:25:10.9533333+00:00

Yes I did, got the same error message as above
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-18T19:28:19.7933333+00:00
@Abhinandan Gupta
Thank you for confirming that you've already deleted the associated storage container and attempted both the standard and direct resource ID deletion methods using Azure CLI. Given that you're still encountering the error:

ErrorCode: FailedToDeletePerClusterContainerErrorCode; ErrorDescription: Internal server error occurred while processing the request. Please retry the request or contact support.

It strongly suggests an internal issue within the HDInsight platform that is preventing the resource from being properly cleaned up - even though the associated container is already removed.

Here are few recommended steps that might help you:

Check for Resource Locks:

Go to the Resource Group in the Azure Portal.

Check the "Locks" section to see if any Delete or ReadOnly locks exist. Remove them if present.

Verify Role Assignments:

Ensure your user or service principal has Owner or Contributor permissions at the Resource Group level.

Specifically,make sure there is no missing Microsoft.HDInsight/* permission that might block operations.

Use Azure REST API for Hard Delete (Optional)

If the Azure CLI fails, you can also try using the Resources - Delete - REST API (Azure Resource Management) | Microsoft Learn for deleting the resource directly, as this sometimes bypasses client-side SDK limitations.

Please refer the below Microsoft documentation for more information
https://learn.microsoft.com/en-us/rest/api/resources/resources/delete?view=rest-resources-2021-04-01
https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/lock-resources?tabs=json

If the issue persists, I will be happy to assist you further.
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-21T15:18:16.0133333+00:00

@Abhinandan Gupta
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Abhinandan Gupta 0 Reputation points

2025-04-21T15:42:56.96+00:00

Hi,

So I dont have any resource locks, and I have Owner permissions on the cluster as well.
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-21T23:25:26.3333333+00:00
@Abhinandan Gupta
Did you try Hard DELETE option via RESTAPI ?

DELETE https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.HDInsight/clusters/<cluster-name>?api-version=2021-06-01

https://learn.microsoft.com/en-us/rest/api/resources/resources/delete?view=rest-resources-2021-04-01If the REST API also fails, you're at the point where Microsoft needs to clean up the stale cluster metadata in their backend.
I will check internally to help you further on this task.
Abhinandan Gupta 0 Reputation points

2025-04-21T23:39:57.14+00:00

I tried using the hard delete using API with the following params:

It gave a 202 response, but the Deletion doesn't go through.
J N S S Kasyap 2,445 Reputation points Microsoft External Staff Moderator

2025-04-23T00:19:26.8066667+00:00

@Abhinandan Gupta
I am reaching out to internal team resolve this issue, meanwhile please check the below questions?
1.Are there any linked or dependent resources still attached to the cluster (e.g., Managed Identity, VNet, NSG, Private Endpoints, Key Vault)?

2.Are there any soft delete, immutability policies, or Azure Policy restrictions applied to the associated storage account or other related resources?
3.Are there any custom RBAC roles or deny assignments affecting the deletion permissions?
4.Have you tried deleting the cluster from a different region or Azure CLI version?
Abhinandan Gupta 0 Reputation points

2025-04-23T04:03:54.48+00:00

Hi,

I have raised a support ticket and they said that the product team would have to delete the cluster manually.
Dileep Raj Narayan Thumula 20 Reputation points Microsoft External Staff Moderator

2025-04-24T11:46:35.8+00:00

Hello @ Abhinandan Gupta,
Can you try the below and check if it helps you
To delete the resources and the resource group, you can try using the Azure Resource Explorer portal, as there might be underlying dependencies preventing the deletion.

Here's how you can do it:

Open the Resource Explorer portal.

Navigate to your Subscription.

Expand the Resource groups section.

Select the specific Resource Group you want to delete.

From there, manually attempt to delete the individual resources.
Dileep Raj Narayan Thumula 20 Reputation points Microsoft External Staff Moderator

2025-04-29T11:08:50.8866667+00:00

Hello @Abhinandan Gupta
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Dileep Raj Narayan Thumula 20 Reputation points Microsoft External Staff Moderator

2025-04-30T11:50:37.7366667+00:00

Hello @Abhinandan Gupta We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Abhinandan Gupta 0 Reputation points

2025-04-30T14:04:31.65+00:00

I was unable to do it from my end. The product team had to it manually from their end

Answer 2

Hello Abhinandan Gupta, good to hear that your issue got resolved.
Based on my research I have found that Azure Resource Explorer portal, as there might be underlying dependencies preventing the deletion. could aslo ressolve the issue. and it could help you in future if you again face such kind of ERROR.

Here's how you can do it:

Open the Resource Explorer portal.

Navigate to your Subscription.

Expand the Resource groups section.

Select the specific Resource Group you want to delete.

From there, manually attempt to delete the individual resources.

If you find this info useful Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Unable to delete HDInsight cluster and resource group.

2 answers

Your answer