ADF - Sending data to REST API Sink as nested objects

Question

Hello everyone

I am trying to use ADF to push data obtained from a SQL Azure table into a REST API. I know most of the information around is the total opposite, but in this case, I am looking to .... "unflatten" the structure to be posted as the endpoints require it to be nested. So far I created a pipeline and a dataflow. In the dataflow, I am using a source and sink, which I am getting errors when trying to post data as it is not expected in the way is being sent.

Let say the current mapping is pushing this [ { "objectID" : "1" }] when it should be pushing this [ { "objectID" { "value" : " 1" } } ]; so, I need to create a child/nested object to contain the value for ID retrieved from the database and in some other cases it can get deeper as in { "EmployeeSettings" { "Level2" { "Attribute" { "value" : "A" } } }

As there is no "Unflatten" formatter in the data flow panel, what can be done here ? Is it even possible or do I need to call an external transformation ?

Thank you in advance.

PS - Verb used will be PUT, in case it makes any difference.

Answer 1

Hi @Rene Rugerio ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

As I understand your ask , you want to convert SQL rows and columns into JSON key value pair with the help of mapping dataflow. Please let me know if my understanding is incorrect.

Your understanding is correct , there is no option to unflatten the data , however, in your case, you can use derive column transformation or aggregate transformation to create sub column and collect function to convert them as item of an array.

Please watch this video for the implementation: Convert records into NESTED JSON using mapping dataflow

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators