Azure Service Fabric
An Azure service that is used to develop microservices and orchestrate containers on Windows and Linux.
267 questions
How can we force remove an application stuck in a deleting state with services in an unknown state
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 16%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBW%3C/text%3E%3C/svg%3E)
Ben Way
0
Reputation points
We have an application that is stuck in a deleting state in Service Fabric.
The application started out with 4 running services all in an Ok state.
1 was successfully removed, 2 are in an unknown state, and one seems to be undead and cannot be stopped.
We have tried running powershell commands to remove the application, services, and replicas with and without the -ForceRemove option. Nothing has worked.
When running Remove-ServiceFabricReplica on the one running service it does remove the service temporarily, but it immediately starts up again on a different node.
When running powershell commands against the service in the unknown status we get a "Service does not exist." error.
How can we get Service Fabric to actually force delete this application so we can redeploy a fresh application and services?
{count} votes
-
Prrudram-MSFT 24,921 Reputation points2024-09-04T05:19:57.5433333+00:00 Hi @Ben Way
Thanks for reaching out, I am checking with the product team on this behavior. Please standby while I get back. -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 40%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAJ%3C/text%3E%3C/svg%3E)
Ankit Jain 0 Reputation points2024-09-04T06:24:43.04+00:00 i am trying to put down all possible options that i could try
When an application is stuck in a deleting state in Azure Service Fabric, it can be challenging to remove it. Here are some steps you can try to force delete the application and its services:
- Use the Service Fabric Explorer:
- Open Service Fabric Explorer and connect to your cluster.
- Navigate to the application and try to delete it from the UI.
- Remove Application with PowerShell:
- Use the
Remove-ServiceFabricApplicationcmdlet with the-Forceparameter to force removal.
Remove-ServiceFabricApplication -ApplicationName 'fabric:/YourAppName' -Force - Use the
- Remove Service with PowerShell:
- If the application cannot be removed, try removing the individual services using
Remove-ServiceFabricService.
Remove-ServiceFabricService -ServiceName 'fabric:/YourAppName/YourServiceName' -Force - If the application cannot be removed, try removing the individual services using
- Remove Replicas:
- For the service that keeps restarting, try removing the replicas individually using
Remove-ServiceFabricReplica.
Get-ServiceFabricReplica -ServiceName 'fabric:/YourAppName/YourServiceName' | Remove-ServiceFabricReplica -ForceRemove - For the service that keeps restarting, try removing the replicas individually using
- Disable the Node:
- If the service keeps starting on a different node, try disabling the node temporarily.
Disable-ServiceFabricNode -NodeName 'NodeName' -Intent RemoveNode - Restart Node:
- Sometimes restarting the node where the service is stuck can help.
- You can restart the node from the Azure portal or using PowerShell.
- Clear Local State:
- If the above steps do not work, you may need to manually clear the local state on the nodes.
- Connect to each node via Remote Desktop and stop the Service Fabric host service, then manually delete the application data from the Service Fabric data folders.
- Contact Support:
- If none of the above steps work, you may need to contact Microsoft Support for assistance with your Service Fabric cluster.
- Cluster Health Check:
- Run a health check on your cluster to ensure there are no underlying issues causing the problem.
Get-ServiceFabricClusterHealth - Check for Orphaned Applications:
- Sometimes applications can become orphaned and not show up in management tools. Use the
Get-ServiceFabricApplicationcmdlet to list all applications and check for any that might be orphaned.
- Sometimes applications can become orphaned and not show up in management tools. Use the
- Use FabricClient API:
- As a last resort, you can use the FabricClient API in a .NET application to try and remove the application programmatically.
- Use the Service Fabric Explorer:
-
Ben Way 0 Reputation points
2024-09-05T21:23:53.0166667+00:00 Thanks @Prrudram-MSFT
I am anxious to hear if they have any suggestions.
-
Ben Way 0 Reputation points
2024-09-06T01:58:01.13+00:00 I truly appreciate you outlining all of these options. Unfortunately, I have tried all of them and none have worked to actually remove the services or application.
-
Prrudram-MSFT 24,921 Reputation points
2024-09-06T09:53:05.1333333+00:00 Hi @Ben Way
I haven't heard from the team yet, I believe this requires product team intervention to force remove this from backend/needs deeper investigation. I would recommend you open an azure support case. If you don't have the ability to open a technical support ticket, please let me know and I can help you further with this.
Sign in to comment
Sign in to answer