File transfer from on-premise file system to the SFTP

Question

Hi Team,

we have a specific requirement where we have to copy a .csv file from an on-premise file system to the SFTP server. File size is currently 30 MB but in future it can be more.
Note: This file contains sensitive information.

we tried to follow below approaches:

1. Tried logic apps to connect file system through on-premise data gateway via FileSystem connector. But there is a limitation for on-premise data gateway which has a restriction limit for 2-MB payload limit. so it won't be suitable for our requirement.
2. Tried Azure Data Factory which has a connector for both FileSystem and SFTP. In this approach we have to allow ADF IPs into the destination SFTP server. But our client is not willing to open firewall for these many IP addresses.
3. since we have already established connection with destination SFTP server via Logic app. we tried to use ADF to copy file from on-premise file system and store it in Azure Blob and then through web activity call logic app which will read the blob and copy the file to the destination SFTP server. But in this approach our client does not want to have a staging area on the blob because of GDPR constraints even for temporary purpose.

can you guys please suggest some effective approach which we can use in this use-case. Is there a way where we can call logic app as a sink for copy activity and pass the content of file in the request body. I am suspecting whether this is possible as the file size may be greater than 30 MB.

Please suggest some solution where we can replace the on-premise data gateway in logic app through any other service to connect on-premise file system.

Thanks & Regards
Shashank Mishra

Answer 1

Hey,
you can create a Self hosted IR and use it to access the SFTP, meaning SFTP linked service leverage the self IR.
That way you can whitelist only 1 IP in SFTP since the IR would contain only 1 public IP.