Finding solution for my SFTP with File Shares and Container accounts scenario in Azure

Question

Hello experts,

trying to get some advise on what I've been dealing with recently...

Scenario:

I have an SFTP server running on VM in Azure. There are files coming from different sources to this SFTP. These files needs to be processed and distributed do appropriate folders where they are processed further. After 7 days, they are deleted. I also need these files to keep for long period time so they are copied to another storage at the time they are distributed to folders as mentioned above. This ling term storrage needs to preserve files for 7 years.... and nothing should be deleted from it.

My set up is:

I have SFTP server running on VM in Azure.
I have File Shares that are mapped to the SFTP Server (to run workloads on them)
I have Blob Container to archvie data (to stora files for 7 years). It is a DATA LAKE storage

Now, there are files that are beining processed in SFTP server during a day (copy files to appropriate folders, decrypt encrypted files received, etc). Files are received from different SFTP sources and then distributed do folders (File Shares) as required. Files older than 7 days are deleted automatically from File Shares. There is a software that is taking care of moving/deleting files.

Also, I have a Blob container that is used to preserve those files for longer period (requirement is 7 years). So when files are moved as described above, there is a task that copies those files also to this container. There should be nothing deleted from this container.

Data Protection:
File Shares
-I'm taking backup to store for 30 days and also have soft-delete enabled for File Shares for 7 days

Blob Container
-I have soft-delete configured for 365 days
-I have a police to move files from HOT tier to ARCHIVE tier after 30 days

I have few questions now:
-is there a better way to achieve what I need? My set up seems to be working fine but maybe there is more effective way
-when blobs are moved from HOT to ARCHIVE, is the file modified somehow? I mean - I have a tool that is checking files on File Share and Archive to see whether a file is missing and if is, it will copy it over to Container - is the file moved to archive different in any way so that the tool will not find it properly? When I was testing this few weeks ago, I've found out that blobs were deleted form time to time and have not been able to identify the issue. So changing tier was one of thoughts that could cause that. Since then, I've recreated blob storage from scratch and trying again...
-how can I restrict deleting blobs? I used recently released Legal Hold, but it block also updating/modifying blobs. I need to restrict deletion of blobs.

thanks in advance ;)

Answer 1

@michal Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

The new blob support SFTP: SFTP support for Azure Blob Storage | Microsoft Learn. Blob storage now supports the SSH File Transfer Protocol (SFTP). This support lets you securely connect to Blob Storage via an SFTP endpoint, allowing you to use SFTP for file access, file transfer, and file management.

Are you open to migrate from their VM based SFTP solution with file shares to SFTP for Azure Blob Storage?

If so, then the below answers could help

1. Better way could be to use SFTP with blob containers instead of VM + file shares + blob container
2. Yes, the blob’s metadata is modified. Instead of some custom tool checking for this, customer could use lifecycle management policy to transition the blobs to different access tier when a condition is met: Hot, cool, and archive access tiers for blob data - Azure Storage | Microsoft Learn
3. For restricting only deletes, customer could configure a local user without delete permissions. Ref: SFTP support for Azure Blob Storage | Microsoft Learn

Currently SFTP doesn't supports Azure File Share

Additional information: How-To create an on-demand SFTP Server with a new Azure Files persistent storage

How to Protect your Azure blob storage from accidental deletion

Please let us know if you have any further queries. I’m happy to assist you further.

----------

Please do not forget to and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.