Azure Synapse - All the tasks are completed but still running the job in Syanapse notebook

Syed Mohammed Yusuf 1 Reputation point

I am storing the data into dedicated pool through scala.

val scala_fact_summary_df = spark.sqlContext.sql("select td.dateid,tsm.spid,tsm.mailerid, "+
"case when tec.errorid is null then 0 else tec.errorid end as errorid, "+
"tet.errortypeid,count(tec.errorid) as TotalOfError from "+
"tblFileData tfd left join tblServiceMailerAccount tsm on "+
"tfd.SubMid=tsm.MIDValue inner join tblDate td on "+
"td.DateId=tfd.Date left join tblErrorCode tec on "+
"upper(trim(tec.message))=upper(trim(tfd.errormessage)) inner join tblErrorType tet on "+
"tet.ErrorType=tfd.ErrorType group by td.dateid,tsm.spid,tsm.mailerid,case when tec.errorid is null then 0 else tec.errorid end, " +
" tet.errortypeid ")

scala_fact_summary_df.write.mode("append").synapsesql("syndp_test.dbo.Fact_ErrorWarning", Constants.INTERNAL )

When I am running this code all the tasks are completed but job is still running very long time.

Kindly help me.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,112 questions
{count} votes