@heta desai - Thanks for the question and using MS Q&A paltform.
The error message indicates that the target Spark pool specified in the Spark job definition is not in a succeeded state. The current state is Delete Error. This means that the Spark pool may have been deleted or is in the process of being deleted.
To resolve this issue, you can try the following steps:
- Check if the Spark pool still exists in your Synapse workspace. If it has been deleted, you will need to create a new Spark pool and update the Spark job definition to use the new pool.
- If the Spark pool still exists, check if there are any ongoing operations on the pool, such as scaling up or down. Wait for these operations to complete and check the state of the pool again.
- If the pool is not in a succeeded state, you can try deleting the pool and creating a new one. Make sure to update the Spark job definition to use the new pool.
- Try to delete the existing spark pool and create a new spark pool and retry.
If none of these steps work, please share more details stack trace of the error message along with the debug logs of the spark cluster for further assistance.
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click Accept Answer
and Yes
for was this answer helpful. And, if you have any further query do let us know.