Azure functions, web jobs or Azure Data Factory ADF - example scenarios ?

Question

Trying to choose between Azure functions, web jobs or looking to something like Azure Data Factory
Anyone

Which Azure services fit this need - moving 1-10MB data files with transformations

• Data files must be imported, read and the data "shaped" into a hieararchical JSON structure to feed another import process
• During the process of reading/extracting the data in each file, lookups must be performed to fill in missing/required values
• The transformations will also involve shifting different collections around within each record/object
• This activity needs to be scueduled
• As suspected, one or more steps might take more than the 5 min limit imposed on Azure functions

Since I am a .net coder, I tend to look for solutions that are code-centric. Hence, I am drawn to Azure functions but im not sure this is the right choice.
I have done a fair amount of work with SSIS and have started looking at ADF (Azure Data Factory) but I would tend to lean toward the code-centric approach.
The thing with functions that cause me to pause a bit is when I consider just how much processing must be done with each file, which will contain 100+ orders and their details.

Answer 1

Hello @bitshift ,

Thanks for the ask and using the Microsoft Q&A platform .
From all in the inputs which you have provided I see that you are more inclined towards Azure Functions . I am pretty confident that you will be able to achieve the ask with that also . Since you are more code centric , this may be more helpful.

I could suggest you to evaluate the Mapping data flow (MDF ) as it does provide the the transformation option and its also lets you play with the JSON object . For MDF you will not need any coding ( just some expression ) as most of the thing are drag and drop .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members