Azure data factory data flow NULLing date column

Oscar Garcia 20 Reputation points
2023-05-23T15:02:15.8466667+00:00

Hi Microsoft community.

I having an issue reading all date fields , the issue is following

The SalesHeader file have the following data, let’s focus on the column 4 (Image 1)

Image 1

Here is the SalesHeader metadata definition (Image 2)

Image 2

Here is DF projection (image 3)

Image 3

In the DF preview (image 4), the column is null

Image 4

I’ve added a derived column trying to convert the value, but the NULL column persist.

Here are some links that I’ve tested

https://learn.microsoft.com/en-us/answers/questions/401414/csv-file-loads-dates-as-null-values-azure-data-fac

https://learn.microsoft.com/en-us/answers/questions/1192318/datetime-source-values-showing-as-null-in-azure-da?cid=kerryherger

https://stackoverflow.com/questions/63625287/azure-data-factory-data-flow-silently-nulling-date-column

Could you help me to fix this issue before?

image 1.jpg

image 2.jpg

image 3.jpg

image 4.jpg

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,616 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,133 questions
{count} votes

Accepted answer
  1. QuantumCache 20,261 Reputation points
    2023-05-23T17:06:22.76+00:00

    Hello @Oscar Garcia,

    Could you please attach sample files (Input & Expected Output Format) to the question, so that the community members can try to repro the situation and help better?

    I was able to reproduce your issue with a sample Source Data!

    User's image

    Then I have used the Derived Column Transformation+ Expression Builder to convert the my Source String Data to the required format!

    If your Source DateTime is Read as a string then the below expression works, I hope this helps with your initial query!

    toTimestamp(LastProcessedChangeDateTIme, 'yyyy-MM-dd\'T\'HH:mm:ss\'Z\'')
    

    User's image

    Please do let us know if you need further help in this matter.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful