Merge operation fails on Azure Synapse

Atseg R 0 Reputation points
2023-08-18T09:51:53.8633333+00:00

I was trying to merge two tables with 634 columns each but was receiving “CREATE TABLE failed because column 'Column388' in table 'QTable_7d03ea86ac8f410d8456ac0783b25172_17' exceeds the maximum of 1024 columns.'” error.

And it seems like there is a limitation in Azure Synapse when the MERGE statement performs UPDATE and INSERT at the same time in such large tables: https://learn.microsoft.com/en-us/answers/questions/794197/merge-command-in-azure-synapse-fails-with-create-t

So, I split the merge statement in two and they were successful:

1. MERGE INTO <dest_table> USING <source_table> ON <col1=col2> WHEN not matched then INSERT …  

User's image

 2. MERGE INTO <> USING <> ON <> WHEN matched then UPDATE …

User's image

Till today, when the  'MERGE INTO <> USING <> ON <> WHEN matched then UPDATE' query is failing with the above-mentioned error (please note that nothing was changed on my environment, table’s columns, or query).

Even when executing the following, it fails:

User's image

However, the other query is still successful:

User's image

Not sure what may be causing this behavior?

Is there any workaround? How can we execute Merge operation on such large tables?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,841 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.