The logs did not give any additional information. But i was able to resolve the issue by moving the synapse notebook to databricks. The pipeline gave the exact error information after doing this. After fixing the code issue i moved the notebook back to synapse and it worked.
Thank you for the support.