Azure Synapse - All the tasks are completed but still running the job in Syanapse notebook
I am storing the data into dedicated pool through scala.
%%spark
val scala_fact_summary_df = spark.sqlContext.sql("select td.dateid,tsm.spid,tsm.mailerid, "+
"case when tec.errorid is null then 0 else tec.errorid end as errorid, "+
"tet.errortypeid,count(tec.errorid) as TotalOfError from "+
"tblFileData tfd left join tblServiceMailerAccount tsm on "+
"tfd.SubMid=tsm.MIDValue inner join tblDate td on "+
"td.DateId=tfd.Date left join tblErrorCode tec on "+
"upper(trim(tec.message))=upper(trim(tfd.errormessage)) inner join tblErrorType tet on "+
"tet.ErrorType=tfd.ErrorType group by td.dateid,tsm.spid,tsm.mailerid,case when tec.errorid is null then 0 else tec.errorid end, " +
" tet.errortypeid ")
scala_fact_summary_df.write.mode("append").synapsesql("syndp_test.dbo.Fact_ErrorWarning", Constants.INTERNAL )
When I am running this code all the tasks are completed but job is still running very long time.
Kindly help me.