Facing issue in ADF Data Flow

Question

Facing issue in ADF Data Flow

ARUP DEBBURMAN 1

I have a xml file where few fields available which is basically Boolean data type and value given as like "Y" and "N". Now I am trying to develop the mapping and during that used flatten and when trying to see data preview for those Boolean field its showing value different way like attached screenshot .Could you please have a look and let me know why it's coming that way.

Yitzhak Khabinsky 27,091 Reputation points

2021-05-23T05:49:59.557+00:00

@ARUP DEBBURMAN ,

Please provide more details.
ARUP DEBBURMAN 1 Reputation point

2021-05-23T06:42:06.333+00:00

I have source xml file which is contain one of the field called ISPAYMENT(Please refer the attached source file screenshot).

Now I am developing ADF data flow ....where I have used flatten which we are generally used for xml source files. Now I am trying to mapped aforesaid xml source and preview the data output using dat flow debug. I am getting ISPAYMENT field as well as other same kind of field which value showing Y OR N in xml structure which you can see in source file screenshot. But after using flatten I have seen value is showing as tick mark or cross mark as I have already given that screenshot. Now my query is why its showing as different compare to source file.
Mark Kromer MSFT 1,146 Reputation points

2021-05-23T18:14:46.37+00:00

In data preview, boolean values are represented in the UI as check for true, X for false.
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2021-05-26T06:17:44.07+00:00

Hi @@ARUP DEBBURMAN ,

Thanks for reaching out. Just wanted to check if the above information from Mark Kromer answers your query? Do let us know if need further assistance? And also I haven't seen any screenshot attached in the original post about the data preview, please consider to add it for better understanding of the scenario.

Thank you
ARUP DEBBURMAN 1 Reputation point

2021-05-26T14:54:52.6+00:00

Thanks for all of your support. I have handled the situation with different approach now its working fine.But I am getting a different issue now as like below

The following column(s) have a complex structure which can only be written to JSON, AVRO, and Azure Cosmos DB datasets: 'HEADER'. Please remove the columns or update the sink dataset to JSON, AVRO, or Azure Cosmos DB.

I have a XML file structure attached .
ARUP DEBBURMAN 1 Reputation point

2021-05-26T15:01:45.59+00:00

Sorry Missed the attachment in previous
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2021-05-28T07:14:31+00:00

Hi @ARUP DEBBURMAN ,

Thanks for your response and glad to know you were unblocked with the initial issue. As this issue is different than the original query, we request you to please open a new thread so that the community can better visibility on this issue and will be able to assist better.

But looking at the data types and the error message it seems like you have complex data types and hence your sink data format should be JSON, AVRO, or Azure CosmosDB as they support complex data types. If you are still blocked please open a new thread with the error message, sample data and what is your expected sink.

Thank you

Your answer

Yitzhak Khabinsky 27,091 Reputation points

2021-05-23T05:49:59.557+00:00

@ARUP DEBBURMAN ,

Please provide more details.
ARUP DEBBURMAN 1 Reputation point

2021-05-23T06:42:06.333+00:00

I have source xml file which is contain one of the field called ISPAYMENT(Please refer the attached source file screenshot).

Now I am developing ADF data flow ....where I have used flatten which we are generally used for xml source files. Now I am trying to mapped aforesaid xml source and preview the data output using dat flow debug. I am getting ISPAYMENT field as well as other same kind of field which value showing Y OR N in xml structure which you can see in source file screenshot. But after using flatten I have seen value is showing as tick mark or cross mark as I have already given that screenshot. Now my query is why its showing as different compare to source file.
Mark Kromer MSFT 1,146 Reputation points

2021-05-23T18:14:46.37+00:00

In data preview, boolean values are represented in the UI as check for true, X for false.
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2021-05-26T06:17:44.07+00:00

Hi @@ARUP DEBBURMAN ,

Thanks for reaching out. Just wanted to check if the above information from Mark Kromer answers your query? Do let us know if need further assistance? And also I haven't seen any screenshot attached in the original post about the data preview, please consider to add it for better understanding of the scenario.

Thank you
ARUP DEBBURMAN 1 Reputation point

2021-05-26T14:54:52.6+00:00

Thanks for all of your support. I have handled the situation with different approach now its working fine.But I am getting a different issue now as like below

The following column(s) have a complex structure which can only be written to JSON, AVRO, and Azure Cosmos DB datasets: 'HEADER'. Please remove the columns or update the sink dataset to JSON, AVRO, or Azure Cosmos DB.

I have a XML file structure attached .
ARUP DEBBURMAN 1 Reputation point

2021-05-26T15:01:45.59+00:00

Sorry Missed the attachment in previous
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2021-05-28T07:14:31+00:00

Hi @ARUP DEBBURMAN ,

Thanks for your response and glad to know you were unblocked with the initial issue. As this issue is different than the original query, we request you to please open a new thread so that the community can better visibility on this issue and will be able to assist better.

But looking at the data types and the error message it seems like you have complex data types and hence your sink data format should be JSON, AVRO, or Azure CosmosDB as they support complex data types. If you are still blocked please open a new thread with the error message, sample data and what is your expected sink.

Thank you

Share via

Facing issue in ADF Data Flow

Your answer