A Block blob file is getting generated for each new folder when doing copy activity

Question

A Block blob file is getting generated for each new folder when doing copy activity

K, Aswathi 31

Trying to copy a folder (JSON data) to another ADLS container using Azure data lake storage Gen2. But a file with same name is auto generated along with new folders created. Kindly help on this.

Subashri Vasudevan 5,026 Reputation points

2022-08-25T12:48:33.73+00:00

Hi @K, Aswathi ,

Can you please let us know what is the driver you are using in the copy activity? (What option was selected while creating sink data set , blob or data lake?)
K, Aswathi 31 Reputation points

2022-08-25T12:57:36.063+00:00

Hi @Subashri Vasudevan ,

Thank you for the response. We are using Azure data lake storage Gen2 for both source and sink data set.

Please let me know if you need any other details.
Subashri Vasudevan 5,026 Reputation points

2022-08-25T13:06:01.33+00:00

Hi @K, Aswathi ,

Please verify if you have enabled hierarchical namespace in the storage account you are seeing this issue in. You may find the property in the Azure portal storage account blade.
K, Aswathi 31 Reputation points

2022-08-25T13:33:21.283+00:00

I could see the namespace disabled @Subashri Vasudevan
K, Aswathi 31 Reputation points

2022-08-25T14:48:44.853+00:00

@Subashri Vasudevan , you are correct. In copy activity I have used Data lake Gen 2 but the storage account itself is a blob storage. That is causing the issue I guess.
Subashri Vasudevan 5,026 Reputation points

2022-08-26T07:15:13.64+00:00

Hi @K, Aswathi ,

Did you get a chance to check on the issue? Were you able to make it work? Lookong forward to your response.

Thanks
K, Aswathi 31 Reputation points

2022-08-26T09:33:13.653+00:00

Yes @Subashri Vasudevan . Changed to storage account and enabled hierarchical namespace, we are able to do the copy activity as expected. Thank you

Subashri Vasudevan 5,026 Reputation points

2022-08-25T12:48:33.73+00:00

Hi @K, Aswathi ,

Can you please let us know what is the driver you are using in the copy activity? (What option was selected while creating sink data set , blob or data lake?)
K, Aswathi 31 Reputation points

2022-08-25T12:57:36.063+00:00

Hi @Subashri Vasudevan ,

Thank you for the response. We are using Azure data lake storage Gen2 for both source and sink data set.

Please let me know if you need any other details.
Subashri Vasudevan 5,026 Reputation points

2022-08-25T13:06:01.33+00:00

Hi @K, Aswathi ,

Please verify if you have enabled hierarchical namespace in the storage account you are seeing this issue in. You may find the property in the Azure portal storage account blade.
K, Aswathi 31 Reputation points

2022-08-25T13:33:21.283+00:00

I could see the namespace disabled @Subashri Vasudevan
K, Aswathi 31 Reputation points

2022-08-25T14:48:44.853+00:00

@Subashri Vasudevan , you are correct. In copy activity I have used Data lake Gen 2 but the storage account itself is a blob storage. That is causing the issue I guess.
Subashri Vasudevan 5,026 Reputation points

2022-08-26T07:15:13.64+00:00

Hi @K, Aswathi ,

Did you get a chance to check on the issue? Were you able to make it work? Lookong forward to your response.

Thanks
K, Aswathi 31 Reputation points

2022-08-26T09:33:13.653+00:00

Yes @Subashri Vasudevan . Changed to storage account and enabled hierarchical namespace, we are able to do the copy activity as expected. Thank you

Answer 1

Subashri Vasudevan 5,026

Then please create a new storage account and make hierarchical namespace enabled and use that in the linked service and try.

Let us know how it goes.

K, Aswathi 31 Reputation points

2022-08-26T07:09:16.75+00:00

Sure @Subashri Vasudevan . Thank you.
Subashri Vasudevan 5,026 Reputation points

2022-08-26T10:04:37.587+00:00

Hi @K, Aswathi ,

Thanks for your confirmation, request you to accept the answer as it will be helpful to the community too.

Best Regards.
Subashri Vasudevan 5,026 Reputation points

2022-08-27T07:57:11.203+00:00

Hi @K, Aswathi ,

Just checking to see if you have any other doubt with respect to the blob formation. If all good, request you to please accept the above as answer for the benefit of the community.

Thanks
Subashri
K, Aswathi 31 Reputation points

2022-08-29T06:52:28.153+00:00

Sure @Subashri Vasudevan , thanks for your help. Accepting the answer.

A Block blob file is getting generated for each new folder when doing copy activity

0 additional answers

Activity