how to exclude json inside the json using data flow in adf

Veera Mummidi 20 Reputation points
2023-12-20T05:16:53.74+00:00

Hi Team,

I have to exclude json inside the json if it is exists in dataflow using azure data factory.

In the below json file i have to exclude the "exclude" if it exists in the formdata json using dataflow in adf.

"formdata": {

  "a":1,

  "b": 2,

  "include": {

                 "a1":1,

                 "date" : ""

    },

 "exclude": {

             "name": "john"

             "country" : "us"

}

}

Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,750 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,212 questions
{count} votes

1 answer

Sort by: Most helpful
  1. AnnuKumari-MSFT 34,011 Reputation points Microsoft Employee
    2023-12-20T06:36:57.8833333+00:00

    Hi Veera Mummidi ,

    Welcome to Microsoft Q&A platform and thanks for posting your query here.

    As per my understanding you are trying to remove one of the jsons from your data and load it in new json file using mapping dataflow in ADF .

    First of all, the json you shared is slightly invalid due to absence of comma , I have modified it to make valid json:

    {"formdata": {
    
      "a":1,
    
      "b": 2,
    
      "include": {
    
                     "a1":1,
    
                     "date" : ""
    
        },
    
     "exclude": {
    
                 "name": "john",
    
                 "country" : "us"
    }
    
    }
    }
    
    
    1. Point the Source dataset to your input json file and select the document form as 'Single document'
    2. Use Aggregate transformation . Leave the 'Group by' tab as blank and go to 'Aggregate' tab . Select 'formdata' column and provide expression as 'collect(formdata)'. This will convert the complex datatype into array
    3. Now use Flatten transformation and select 'Formdata[]' array to unroll by and choose 3 columns 'a','b' and 'include'
    4. Use Sink transformation and select json connector to load the output. removejson

    Hope it helps. Kindly accept the answer by clicking on Accept answerbutton


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.