Invalid property name error in ADF Copy Data Activity

Question

We have created pipelines for copying data from sharepoint. We have configured our copy data activity to store the file with some metadata also like author name, title, link, last modified, etc. Now for some files I'm getting below error:-

ErrorCode=AdlsGen2OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'BadRequest'. Account: 'rmsdatalakestracc1mqa'. FileSystem: 'rmsadlscontainer'. Path: 'sharepoint/documents/RMS-MMY/ITBS-Apps/Unique User Experiences – Leading US Based Credit.pdf'. ErrorCode: 'InvalidPropertyName'. Message: 'The property name contains invalid characters.'. RequestId: '71491bd1-201f-0048-74c2-88e256000000'. TimeStamp: 'Sun, 07 Apr 2024 08:05:53 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'BadRequest',Source=Microsoft.DataTransfer.ClientLibrary,'

The file is download in our adls container but its without metadata. And due to this I'm not able to move to the next phase of my pipeline. Please guide me to solve this issue.

Answer 1

Azure Data Lake Storage Gen2 only allows alphanumeric characters and the dash (-) in property name :

• Verify that all metadata property names comply with Azure's naming rules. They should contain only alphanumeric characters and dashes (-).
• You can use the "Fault Tolerance" feature in the Copy Activity. This feature allows the copy operation to proceed even if some rows are incompatible, skipping over the invalid metadata.
• Enable session logging within the copy activity to log any skipped data for further review.
• If the issue continues, consider using a Data Flow activity to separate compatible and incompatible rows before executing the copy operation.

If the problem persists, please let us know.