Azure Data Factory - Reading JSON Array and Writing to Individual CSV files

Question

Hi all!

I have many JSON files that lands in my data lake that all have the same format like this:

[
  {
    "AzureContainer": "someContainer",
    "AzureFolderName": "someFolder",
    "AzureFilename": "myFilename",
    "Timestamp: "123123213",
    "ItemName": "myItem",
    "Value": 567
  },
  {
    "AzureContainer": "someContainer",
    ...
  },
  ...
] 

What I want to do, is to use the information in these items to append to CSV files in my Data Lake in the format:
[Timestamp],[ItemName],[Value]
[Timestamp],[ItemName],[Value]
....

What I've done thus far is create a lookup that reads all of the files within my landing blob. I can then extract each of these individual JSON objects. Now, I need to use the information in the JSON object to write to the specified location in my Azure storage account (using AzureContainer, AzureFolderName and AzureFilename), and I need to create a CSV entry using Timestamp, ItemName and Value.

How can I do this? I'm trying to find something that allows me to write to a CSV without having to assign a Source datasource? Thus I want to format my CSV content, and use the Azure parameters to append the CSV file in the specified location.

Answer 1

Hi @Cornel Verster ,

Thanks for using Microsoft Q&A forum and posting your query.
The same question was answered by one of our engineer in Stackoverflow forum.

Here is the stackoverflow link with detailed step by step explanation of the implementation to achieve this requirement: Azure Data Factory - Reading JSON Array and Writing to Individual CSV files

Sharing it here as it would be beneficial for others reading this thread.
Hope this helps. Please feel free to let us know if you have further query

----------

• Please accept an answer and up-vote if it helps. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.