FTPS Linked Service - FEAT

Question

Hi All, I hope you are all doing well. We are in the final phase of migrating our Data Warehouse systems from on-prem to Azure. We are using Data Factory to pull data from the different third parties. The requirement with one of the providers is to use FTPS with client cert authentication. I could add the cert of a Key Vault and reference to it from the FTP linked service, however it keeps failing out. I noticed in the requirements that we need to sent the FEAT control command before and after login and wondering how could I do it via the linked service? Thank you in advance :)

Answer 1

Unfortunately, Azure Data Factory (ADF) does not provide an option to explicitly send the FEAT control command when using the FTPS-linked service. ADF's FTP connector is designed to handle the standard FTP and FTPS connections and does not expose advanced FTP command configurations. As a workaround, you can use an Azure Function or an Azure Logic App to create a custom FTP client that can handle the FEAT control command before and after login. Then, you can use a Web Activity or an Azure Function Activity in your ADF pipeline to trigger the Azure Function or the Logic App. Example Approach

• Create an Azure Function or an Azure Logic App with the necessary code to connect to the FTPS server using a custom FTP client (e.g., using an FTP library that supports the FEAT control command).
• Implement the required FTPS operations (e.g., retrieving files) in the Azure Function or the Logic App, ensuring that you send the FEAT command before and after login as required by your provider.
• Store the retrieved files in a staging location, such as an Azure Blob Storage account or an Azure Data Lake Storage account.
• In your ADF pipeline, use a Web Activity or an Azure Function Activity to trigger the Azure Function or the Logic App.
• After the Azure Function or the Logic App completes, use ADF activities like Copy or Mapping Data Flow to process the files in the staging location and load them into your Data Warehouse.

By using this approach, you can handle the FEAT control command requirements for your FTPS connection while still leveraging the power of Azure Data Factory for your data movement and transformation needs.