Gathering Analytics from txt files from Blob container with complex file paths

HarriS 1

I want to be able to perform analytics on several txt files that are stored in a blob container within a storage account.

The file path looks something like

shoppingcenter\location\2020-05\shop\myfile.txt

there's several location folders, several date folders, and several shop folders.

I want to be able to analyse all the txt files that start with myfile_.txt together.

What would be the best approach to produce analytics (graphs and statistics) from all the data combined?

I assume this is an azure data factory question.

1 answer

Ashley Mallia 1 Reputation point

2021-01-26T06:37:15.537+00:00

Hi Harri,

The file connector accepts wildcards and has options like recursive and flatten, which will allow you to match files of a specific name at any point in the hierarchy.

Refer to the docs here:
https://learn.microsoft.com/en-us/azure/data-factory/connector-file-system

Best Regards,
Ashley
Please sign in to rate this answer.
HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2021-02-02T17:30:55.997+00:00

Thanks anonymous userMallia-6560 for the help .
@HarriS : We have not received a response from you. Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Thanks
Himanshu

HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2021-02-03T18:24:26.813+00:00

hello @HarriS

We have not received a response from you yet . Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Thanks
Himanshu
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Your answer