rename of pipeline failed part-way, now I have both source and destination pipeline and cant remove them

Robert Vaughan 0

Hi we have a pipeline A that was being renamed to pipeline B, and the rename has failed. now we have pipeline A with an EntityState of "Deleting", and Pipeline B with an EntityState of "Creating". they are linked by Source and Target values and an OperationId. e.g.

       "EntityState": "Deleting",
        "RenameOperationDetails": {
            "Source": "Pipeline A",
            "Target": "Pipeline B",
            "OperationId": "7aed9c51-9377-4a25-954e-682587898b41"

We can't rename. remove or update these pipelines, in any way, the message being "The Operation failed because the entity 'Pipeline A' is being renamed to 'Pipeline B'. A retry of rename from exact source to target may work"

It doesn't work, now it is blocking our deployment via GitHub as they were created directly in our Live environment and not via Git.

phemanth 6,885 Reputation points Microsoft Vendor

2024-05-09T08:20:23.2633333+00:00
@robert vaughan

Thanks for using MS Q&A platform and posting your query.

I understand that you’re facing issues with renaming pipelines in Azure Synapse Analytics. This seems to be a common issue when pipelines are renamed directly in the live environment, especially when the workspace is Git configured

Here are a few steps you can try to resolve this issue:

Revert to the Original Name: Change the pipeline back to the original name and republish

Use a New Local Branch: Create a new local branch, rename the pipeline there, and then raise a PR to merge the changes into the collaboration branch

Clone the Pipeline: If directly changing the pipeline name is not working, clone the pipeline and provide the desired name for the new pipeline. Then, delete the older pipeline and merge the changes.

Hope this helps. Do let us know if you any further queries.
Robert Vaughan 0 Reputation points

2024-05-09T10:09:36.64+00:00

Hi @phemanth Thanks for your suggestions, however, our Prod Synapse instance isn't Git configured, only the dev/test one, so we are unable to create a branch. (we use CI/CD to push changes to our Prod synapse instance)

we are unable to rename either of the pipelines (A & B) , I keep getting the 'rename is in progress' message, and the publish fails.

I can clone the existing pipeline, but am still unable to remove it, getting the same error mentioned above.

We have tried using the CLI -both versions (az Synapse pipeline delete & remove-AzSynapsePipeline) but neither work. we've also tried updating the code both in the JSON editor in studio, and via the CLI.

Does anyone know of a 'backdoor' method to remove these objects, since the front-end tools aren't working?
phemanth 6,885 Reputation points Microsoft Vendor

2024-05-10T09:17:35.15+00:00
@Robert Vaughan I understand that you’re facing challenges with renaming and removing pipelines in your Azure Synapse Analytics environment. Let’s explore some potential solutions to address this issue:

Cancel Running Pipelines:

Sometimes, a pipeline might be stuck in an in-progress state due to various reasons.

In the Synapse Studio, navigate to the Monitor tab.

Identify the pipeline run that is causing the issue (e.g., the one with the “rename is in progress” message).

Look for an option to cancel the run (usually represented by a “Cancel” button or similar UI element).

Confirm the cancellation of the run.

Check for Recovery Runs:

Occasionally, recovery runs can cause pipelines to hang.

Identify any recovery runs that might be interfering with your pipelines.

Cancel any such recovery runs to free up your pipeline for reruns.

Use PowerShell or Azure CLI:

Since the front-end tools aren’t working, consider using PowerShell or Azure CLI to manage your pipelines.

You mentioned trying the CLI commands az Synapse pipeline delete and remove-AzSynapsePipeline. Ensure that you have the necessary permissions and are using the correct syntax.

If these commands still don’t work, try using the following PowerShell script to remove the pipelines:

# Replace with your specific details $resourceGroupName = "YourResourceGroup" $workspaceName = "YourSynapseWorkspace" $pipelineNameA = "Pipeline A" $pipelineNameB = "Pipeline B" # Remove Pipeline A Remove-AzSynapsePipeline -ResourceGroupName $resourceGroupName -WorkspaceName $workspaceName -PipelineName $pipelineNameA # Remove Pipeline B Remove-AzSynapsePipeline -ResourceGroupName $resourceGroupName -WorkspaceName $workspaceName -PipelineName $pipelineNameB

Execute the script in your PowerShell environment, replacing the placeholders with your actual resource group, workspace, and pipeline namesUse

If none of the above methods work,please do let us know
Robert Vaughan 0 Reputation points

2024-05-10T14:30:57.8+00:00

@phemanth

Still no joy with those suggestions. thanks for trying though.

Rob.
phemanth 6,885 Reputation points Microsoft Vendor

2024-05-13T10:59:27.6966667+00:00
@Robert Vaughan

Azure Resource Explorer:

Azure Resource Explorer (ARE) is a powerful tool for directly interacting with Azure resources.

Open ARE and navigate to your Synapse workspace.

Locate the pipelines (A and B) and try deleting them from there.

Use the following URL format to access ARE:
https://resources.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}

Replace {subscriptionId}, {resourceGroupName}, and {workspaceName} with your actual values.

Azure REST API:

If ARE doesn’t work, consider using the Azure Synapse REST API.

You can programmatically delete pipelines using the API.

Refer to the official documentation for details on how to construct the request.

If none of the above methods work, please do let us know we will further help to resolve this issue.
Robert Vaughan 0 Reputation points

2024-05-14T08:23:26.24+00:00

@phemanth I couldn't find the pipeline via the ARE, even when using the URL you provided (also used Resource Explorer in the azure portal)

We also tried removing them via the REST API, but no luck there either.
phemanth 6,885 Reputation points Microsoft Vendor

2024-05-15T11:07:36.53+00:00
@Robert Vaughan

finally check these steps below

Azure CLI with Additional Flags:

While the az Synapse pipeline delete command didn't work previously, consider trying it again with the --force flag. This flag might bypass some checks and force the deletion, but use it with caution as it's a more aggressive approach.

Command:

az Synapse pipeline delete --resource-group $resourceGroupName --workspace-name $workspaceName --name $pipelineName --force

Important: Replace placeholders with your actual values and ensure you have the necessary permissions to use the --force flag.

Check for Locks:

In rare cases, resource locks might prevent deletion. Use Azure Portal or Azure CLI to check for any locks on the pipelines or the Synapse workspace. Remove any irrelevant locks to allow deletion.

if none of this work let us know we will check with our internal team

Share via

rename of pipeline failed part-way, now I have both source and destination pipeline and cant remove them