ADF copy activity fileextension option in delimited dataset as sink is not changing the extension. Any solution for this?

Anvy Elizabeth 1 Reputation point
2023-05-10T05:19:44.3+00:00

I have some files in blob storage in .csv format. I was copying this to ADLS folder. I need to append date format as postfix to file copied. I have not added filename in output dataset. But this is not changing file extension.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,036 questions
{count} votes

1 answer

Sort by: Most helpful
  1. AnnuKumari-MSFT 31,911 Reputation points Microsoft Employee
    2023-05-11T10:52:17.75+00:00

    Hi Anvy Elizabeth ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here.

    As per my understanding , you are trying to concatenate the sink filename with date as a postfix along with the extension. Please let me know if that is not what you are looking for .

    I tried using fileExtension option to create the postfix to store date with the extension for the sink file. It worked for a file but didn't for another. Assuming the requirement is to append current date at the end of the filename, I added dynamic content in file extension as below expression:

    @concat(formatDateTime(utcnow(),'yyyy-MM-dd'),'.csv')
    

    file_extension

    Although it works , but the file extension is meant to detect the output file name and the expected values are '.csv','.txt' etc. I would recommend you to use filename option in the sink dataset to create the dynamic filenames.

    For more details on the implementation, kindly checkout this video: How to provide dynamic name to the target file based on pipelinerundate In azure data factory

    Hope it helps. Kindly accept the answer by clicking the Accept answer button. Thankyou.