ADF Data Flow source transformation is not reading data types correctly while working with complex data types.

Question

I have a json file which looks like below. Though all attributes in both objects are in

string( '' ''). Data flow is reading them as boolean in case of "1" and as short in case of "2","3" etc.

Under OccurrenceSpans, object 1 elements are read as (date,short,bool,date) but in object 2 it is being read as (str,str,str,str) - though data is of same type in both objects.

JSON File format Screenshot

Data Flow Screenshot

When tried to Overwrite schema, it is not allowing me to define data types for sub elements. I am getting option only to define main object data type only.

Can you help me in solving this ?

Answer 1

Hi @Varun Chandra ,

Thank you for the ask.

I think you are trying to change the schema from the DataFlow >> Source. You can change the Dataset-Schema by going into the Dataset >> code, and then modify the TYPE. Now, if you go in the DataFlow >> Source >> Projection and do reset Schema, it will show you the updated one. Thanks!

Answer 2

@Nasreen Akter Thank you for the response.

Found a way to define datatypes for Inline Datasets. We have an option in Source>Projection > Enable overwrite schema >Define Complex type where we can change sub elements data type.