Azure Data Lake Storage Gen2 Security Features and Backup/Recovery Recommendation

Question

Can someone please let me what security features are available on Azure Data Lake Storage Gen2 and how to plan/implement Backup/Recovery on Adls gen2

Answer 1

Hi @KranthiPakala-MSFT ,
Is there any update regarding Azure Data Lake Storage Gen2 automatic Backup/Recovery capability, similar to Azure Storage Account blobs?
I saw that there is a feature request that was in the roadmap two years ago.
I'm wondering if there is any progress on this.

Answer 2

Hi @Akash Verma ,

Thanks your reaching out and using Microsoft Q&A forum.

Security Features
Firstly to talk about Azure Data Lake Storage Gen2 Security features, Data Lake Storage provides six different layers of security: authentication, access control, network isolation, data protection, advanced threat protection, and auditing.

2. Authentication:
   ADLS Gen2 supports three different authentication methods: a) Azure Active Directory is the ideal way to verify a user’s identity. The only potential issue is that users must be defined in azure active directory before they can
   access data.
   b) SAS - Shared Access Signature : You can create a SAS that only has access to specific data and has an expiry date and time, after which it is no longer valid
   c) Shared Access Keys - The caller effectively gains 'super-user' access, meaning full access to all operations on all resources, including setting owner and changing
   ACLs.
   To more about these authentication methods please refer to this doc: Shared Key and Shared Access Signature (SAS) authentication
3. Access Controls:
   For access control, Azure Data Lake Storage Gen2 implements an access control model that supports both Azure role-based access control (Azure RBAC) and POSIX-like access control lists (ACLs).
   Here is an article which summarizes the basics of the access control model for Data Lake Storage Gen2. Please refer to it: Access control in Azure Data Lake Storage Gen2
4. Isolation:
   The third layer of security is network isolation. You can actually set up a firewall just for your data lake. Select Firewalls and virtual networks in the Settings menu. The default is to allow access from all networks. If you click Selected networks, then a whole bunch of other configuration options appear.
   i) First, you can enable access from specific virtual networks. Second, you can allow access from particular IP addresses.
   ii) If you want to access your data lake using other Azure services, such as Azure Backup, then you can make an exception by checking this box. Another couple of
   possible exceptions are if you want to allow read access to storage logging and metrics from any network.
5. Data Protection:
   The fourth layer of security is data protection. ADLS supports encryption of data both at rest and in transit. Data in transit is encrypted using HTTPS by default. Data at rest is also encrypted automatically.
6. Advanced Security:
   The fifth layer of security is Advanced Threat Protection. If you enable this, it will watch for attempts to access or exploit your storage accounts. If any suspicious activities are detected, then it will send you alerts through Azure Security Center.
7. Auditing:
   The sixth layer of security is auditing. ADLS logs all account management activities. To see them, click “Activity log”.

For more info please refer to this document: ADLS Gen2 Security recommendations

Backup/Recovery:
Coming to the second ask in the original query, i.e., regarding Backup/Recovery, as per the latest information from internal sources, ADLS Gen2 Backup integration is in the roadmap but no concrete ETA at the moment. Soft delete is in the current plan and you can expect this feature to be landed in the near future.

I would recommend you to please subscribe to Azure updates to know about the latest updates on Azure products and features.

Here is an existing feature request thread regarding ADLS Gen2 Backup feature, I would encourage you to please comment and/or up-vote as it would help to increase the priority of the feature request : ADLS Gen2 Backup and Point-in-time restore

Additional info : Since ADLS Gen2 doesn't have native support for Backup, as a workaround I would like to share an article/blog (Disclaimer: Not an MSFT document/article) which was found online, please have a look at it and see if the meets your requirement: Custom Backup Azure Data Lake Gen2 using Azure Data Factory

Hope the above information helps.

Thank you

----------

Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

Answer 3

@Voloshin, Denis Last I heard from our Azure CSM was that there is still not a concrete date, but it is coming.