Logging file names in cosmos db using ADF

PC 6

Hello ,

I have one scenario to implement: I am reading files in ADLS using ADF and copying into another location. While doing that, I need to capture file names and store in cosmos db. Does anybody has done such scenario, please suggest. TIA.

1 answer

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2021-03-23T19:21:31.787+00:00

Hi @purushothamchanda-2222,

Thanks for your query.

After copying the files, once the initial copy activity is succeeded, you can use a getMetadata activity to get the list of file names from source and use the output value to copy to your desired sink (in this case to the db collection in cosmos database).

Hope this helps. Do let us know if you have further query.

Thanks

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2021-03-25T16:17:18.4+00:00

Hi @purushothamchanda-2222,

Just checking in to see if the above suggestion was helpful. If this answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thanks

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2021-03-30T01:21:52.033+00:00

Hi @purushothamchanda-2222,

We still have not heard back from you. Just wanted to check if the above suggestion was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

PC 6 Reputation points

2021-04-05T08:16:43.363+00:00

Hi Kranthi,
sorry for late response. I did use the same already earlier but used GeMetadata and foreach so i can pick individual file names along with some of derived columns. Any suggestion to avoid foreach and get all the data along with derived columns. Thanks

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2021-04-06T18:22:16.937+00:00

Hi @PurushothamChanda-2222,

Thanks for your response. If you would like to pick individual file names and some derived columns from the list then you will have to use For Each activity.

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.
Sign in to comment