Error while executing azure mapping data flows on azure synapse serverless views

Question

we are trying to create mapping data flows in azure data factory. The data source for mapping data flow is synapse serverless views. The data for the views is stored in Azure Blob Storage. On previewing, we are able to see the data from the views. but when we are executing the data flow in debug mode, we are getting the following error:

{"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: at Source 'CurrencyView': shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near 'HADOOP'.","Details":"shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near 'HADOOP'.\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1632)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:872)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:767)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7418)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3272)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247)\n\tat shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222)\n\tat shaded.msd"}

there is no reference to HADOOP in view definition. Could you please let us know the reason for the error.

Thanks

Prashant

Answer 1

Hello @Prashant Malik ,

Thanks for the question and using MS Q&A platform.

As per my understanding while executing mapping data flows on Azure Synapse serverless views as source you were able to preview the data but it fails when you debug run or trigger the pipeline. Please correct if I am not accurate.

Could you please confirm if you have enabled staging under your data flow source options?

If yes, that could be the reason you are seeing this behavior as Serverless pool doesn't support staged data loading, so 'enable staging' is not supported. Hence Please try to uncheck enable staging and see if that helps to resolve the issue.

Ref doc: Serverless pool (SQL on-demand) related issues

Here is another thread where a similar issue has been discussed: Azure Synapse: Dataflow runs in preview but not from Pipeline

Hope this info helps. Do let us know how it goes.

Thank you