Share via

ADF load empty strings as null using Polybase

FionaHu 1 Reputation point
2021-01-19T08:33:42.207+00:00

{
"name": " ",
"properties": {
"activities": [
{
"name": " ",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "SqlServerSource",
"queryTimeout": "02:00:00",
"partitionOption": "None",
"treatEmptyAsNull": false
},
"sink": {
"type": "SqlDWSink",
"allowPolyBase": true,
"polyBaseSettings": {
"rejectValue": 0,
"rejectType": "value",
"useTypeDefault": false,
"treatEmptyAsNull": false
}
},
"enableStaging": true,
"stagingSettings": {
"linkedServiceName": {
"referenceName": "LS_BlobStorage",
"type": "LinkedServiceReference"
},
"enableCompression": true
},
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"name": "nvc_city",
"type": "String"
},
"sink": {
"name": "nvc_city",
"type": "String"
}
}
]
}
},
"inputs": [
{
"referenceName": " ",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": " ",
"type": "DatasetReference"
}
]
}
],
"folder": {
"name": "Scratch"
},
"annotations": [],
"lastPublishTime": "2021-01-18T05:39:39Z"
},
"type": "Microsoft.DataFactory/factories/pipelines"
}

Azure Synapse Analytics
Azure Synapse Analytics

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.


1 answer

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,161 Reputation points
    2021-01-19T20:26:47.48+00:00

    Hello @FionaHu and welcome to Microsoft Q&A.

    If you want to load empty strings as null, then there is one change you need to do in the code your shared:

    "treatEmptyAsNull": false  
    

    change to:

    "treatEmptyAsNull": true  
    

    If this solves your issue, please mark as accepted answer. If not, please explain better your ask.

    Was this answer helpful?

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.