This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 31%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
I keep getting this error in random pipelines I'm running. How can I retry it automatically?
{
"errorCode": "145",
"message": "Internal Server Error in Synapse batch operation: 'Cluster creation failed due to unexpected error. Please retry.'.",
"failureType": "SystemError",
"target": "FactWvActivity",
"details": []
}
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Hello Rhuan Samary Barreto,
This issue has been reported, and PG is actively working on it. I will provide updates once they have fixed the issue
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 61%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAV%3C/text%3E%3C/svg%3E)
I have been getting this error repeatedly and not able to resolve this. Pipeline reads data out of CDC connector connection via ODP and dumping data in ADLS storage.
Detailed Error:
SAP2DeltaDataflow
{ "error Code": "145", "message": "Internal Server Error in Synapse batch operation: '[plugins.AZSYNUSCETSTBIDATA001.General-16-1-31a3da26-63e8-44aa-a1e9-46a8fa87b9d9.155 Workspace Type:<Synapse> CCID:<3e2024e6-4734-47a1-a4ad-385c8d7721ce>] Attempt=[1]/[3]Cluster was in terminal state=[Cancelled] before it reached 'Ready' state. Cluster job has Workspace Name=[AZSYNUSCETSTBIDATA001], Spec Name=[General-16-1-31a3da26-63e8-44aa-a1e9-46a8fa87b9d9], and Job Id=[dfd15566-093a-4b34-8769-3c99a0e295b8].'.", "failure Type": "System Error", "target": "SAP2DeltaDataflow", "details": [] }
Hello Rhuan Samary Barreto,
This issue has been mitigated, so you should not see these errors anymore.
Please let us know if you still encounter any internal server errors.
I hope this helps.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 16%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
It appears this error can reoccur per: 'Issues executing spark jobs for Azure Synapse Analytics in East US' (Tracking ID: PT10-5VZ).
It looks like ADF MDF's are impacted by the same event that causes Synapse outtages.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 32%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Hi, there. I am seeing the same error from my dataflow. It does not look like the mitigation you applied worked.
May be check CMK keys if expired, you have CMK keys set on workspace and come from some keyvault. MSFT should give you more detailed logs.
Thanks,
Vinod
The pipeline runs daily and it has multiple child pipelines loading using the same reusable dataflow to load different tables in parallel. I re-ran the child pipeline for the table load that failed and it succeeded.
The error ("Cluster was in terminal state=[Cancelled] before it reached 'Ready' state") did not occur when the pipeline ran today.
I will keep an eye on its run over the next few days but wonder what the root cause of this error was and what mitigation MS applied at the time.
Quick fix to the first sentenced in my previous comment: "The pipeline runs daily and it has multiple child pipelines using the same reusable dataflow to load different tables in parallel"
Hi, MS. The same error occurred again today. Any idea as to what the root cause could be ?
This has reoccurred today for the Azure East US region, the issue is flagged in the Azure Service Health Monitoring Tool: https://azure.microsoft.com/en-us/get-started/azure-portal/service-health
The current incident PT10-5VZ (Issues executing spark jobs for Azure Synapse Analytics in East US) is linked to Azure Synapse Analytics, however there is a clear link to ADF Mapping Data Flows which also run on the Spark Clusters.
Not so much a solution but worth checking for Service Health events if you see similar error.