Databricks Synapse Spark connector fails to capture exception in preActions/postActions

AC 1 Reputation point
2022-06-30T15:12:40.853+00:00

We are trying to make use of the preActions and postActions options of Databricks Synapse Spark connector which fails to catch exception.

We are using a DBR 10.4 LTS high concurrency cluster.

Following code is expected to fail, however it succeeds -

df = spark.sql('select 1 as id')
df.write \
.mode('overwrite') \
.format('com.databricks.spark.sqldw') \
.option('url', synURL) \
.option('tempDir', synTmpDir) \
.option('enableServicePrincipalAuth', True) \
.option('maxStrLength', 4000) \
.option('preActions', 'select 1 / 0; select cast(\'a\' as int)') \
.option('postActions', 'select 1 / 0; select cast(\'a\' as int)') \
.option('dbTable', 'dbo.t1111') \
.save()

Please review and let us know if -

  1. we are missing any option to trap the error
  2. there is a workaround
  3. this is a product bug

216636-image.png

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,172 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,325 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,506 Reputation points Microsoft Employee
    2022-07-04T06:52:16.61+00:00

    Hi @AC ,

    Thank you for posting query in Microsoft Q&A Platform.

    You mean you are expecting statements select 1/0 or select cast('a' as int) to throw error while running? If yes, then It will not happen actually. These both statements will return you back null. Please check below screenshots.
    217128-image.png

    Hope this helps. Please let us know if any further queries.

    --------
    Please consider hitting Accept Answer button. Accepted answers help community as well.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.