ADF - dataflow - xml parsing with optional tags

Bhanuprasad 121 Reputation points
2021-02-03T03:14:38.977+00:00

I have xml file, if the optional elements are present able to use flatten activity and parse the xml it works fine.
When the optional elements (or tags) not present the process fails.

With tag:
<EDocument DVersion="1.0" Client="100" Id="5" Created="1/30/2021">
<Export_List>
<Export BDate="5/29/2020" Amount="57.45" ANumber="119" PName="1/2021" Year="2021" />
</Export_List>
</EDocument>

Without tag:
<EDocument DocVersion="1.0" Client="100" Id="5" Created="1/30/2021">
</EDocument>

Tried with filter, byName(column), also referred the following logic with no avail
https://learn.microsoft.com/en-us/answers/questions/91259/issue-with-adf-data-flow-xml-source-and-optional-n.html

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,693 questions
{count} votes

2 answers

Sort by: Most helpful
  1. HarithaMaddi-MSFT 10,136 Reputation points
    2021-02-05T07:46:22.883+00:00

    Thanks @Bhanuprasad for confirming the details. Its not possible using a direct option in dataflow but the only possibility I can think of is to use multiple dataflows and by using lookup and IF Condition activities, trigger the dataflow (either with flatten or without flatten). Disabling an activity with a condition is unavailable in data factory today. I understand this is adding more code, I would recommend you to upvote an existing idea in feedback forum which will be closely monitored by data factory product team and considered for future implementations.

    64462-image.png

    IF Condition Expression:

    @empty(activity('Lookup1').output.value[0].Edocument)  
    

    Hope this helps! Please let us know for further queries and we will be glad to assist.

    --

    • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification.

  2. Bhanuprasad 121 Reputation points
    2021-02-12T07:24:54.063+00:00

    @HarithaMaddi-MSFT , thank you for your suggestions. just checking if there are any alternatives (or) work around we could validate in the xml file tags present or not?