Using Azure Polybase data copy- empty string becomes NULL, NULL becomes empty string #sqldatawarehouse #polybase #datafactory #usetypedefault

Bharat Thatipally 21 Reputation points
2023-04-05T08:53:17.87+00:00

Hi I see an issue while loading data from source to target using polybase in ADF copy activity. issue detail:

  1. Nulls in the source are converted to empty strings
  2. I tried the option #usetypedefault in the sink and now Blanks in the source are converted to Nulls.

Also I found a relevant thread for the same issue in the forum where it was suggested to go with DFT but i cannot since we have hundreds of tables to copy the data. reference link: https://social.msdn.microsoft.com/Forums/en-US/a3cb56e4-2851-4c51-88d2-01d4efccddc8/polybase-empty-string-becomes-null-sqldatawarehouse-polybase-datafactory-usetypedefault?forum=AzureDataFactory Thank you, Bharat

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. HimanshuSinha 19,567 Reputation points Microsoft Employee Moderator
    2023-04-06T21:20:55.41+00:00

    Hello @Bharat Thatipally , Thanks for the question and using MS Q&A platform. I did test the scenario and if sink is SQL it does works fine . User's image

    If the sink is blob then you will have to set the NULL value property User's image

    and this gives the correct output. User's image

    Thanks Himanshu
    Please accept as "Yes" if the answer provided is useful , so that you can help others in the community looking for remediation for similar issues. 


Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.