Data Files Not Copying to Azure Blob Storage

Bathini Harshitha 345 Reputation points
2024-10-07T09:34:24.98+00:00

Why are some data files not copying to Azure Blob Storage even though other files copy successfully without errors?

PS - Based on common issues that we have seen from customers and other sources, we are posting these questions to help the Azure community.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Bathini Harshitha 345 Reputation points
    2024-10-07T09:35:51.4766667+00:00

    The issue you are experiencing, where certain data files are not copying to Azure Blob Storage while others do, is likely due to the length of the file names. In your case, two specific files with longer names were not copying. When the file names were shortened, they copied successfully.

    To resolve this issue, ensure that the file names adhere to the limitations set by Azure Blob Storage. The maximum path length and blob name length should not exceed 256 characters. If a file name is too long, it may prevent the file from being copied successfully.

    Here are the steps to troubleshoot and resolve the issue:

    Check File Name Length: Review the length of the file names that are not copying. Ensure they do not exceed the 256-character limit.

    Shorten File Names: If the file names are too long, shorten them and attempt to copy the files again.

    Test with Shortened Names: Perform a test by copying files with shortened names to confirm that the issue is resolved.

    Review Documentation: Refer to the official Azure documentation on naming and referencing containers, blobs, and metadata to understand the limitations and best practices.

    Resources:

    Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

    Please do not forget to "Accept the answer" and "up-vote" wherever the information provided helps you, as this can be beneficial to other community members.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.