Merge operation fails on Azure Synapse
I was trying to merge two tables with 634 columns each but was receiving “CREATE TABLE failed because column 'Column388' in table 'QTable_7d03ea86ac8f410d8456ac0783b25172_17' exceeds the maximum of 1024 columns.'” error.
And it seems like there is a limitation in Azure Synapse when the MERGE statement performs UPDATE and INSERT at the same time in such large tables: https://learn.microsoft.com/en-us/answers/questions/794197/merge-command-in-azure-synapse-fails-with-create-t
So, I split the merge statement in two and they were successful:
1. MERGE INTO <dest_table> USING <source_table> ON <col1=col2> WHEN not matched then INSERT …
2. MERGE INTO <> USING <> ON <> WHEN matched then UPDATE …
Till today, when the 'MERGE INTO <> USING <> ON <> WHEN matched then UPDATE' query is failing with the above-mentioned error (please note that nothing was changed on my environment, table’s columns, or query).
Even when executing the following, it fails:
However, the other query is still successful:
Not sure what may be causing this behavior?
Is there any workaround? How can we execute Merge operation on such large tables?