Central ADLS vs Federated architecture

Anshal 2,251 Reputation points
2023-04-15T15:29:16.9966667+00:00

Hi friends, I lack clarity on the ADLS approach of central and federated architectures. Federated ADLS consists of multiple ADLS and my questions are: a. are these ADLS separately created individually? b. how one adls interact with other adls c. how security and integrity are maintained? d. implementation challenges e. costs and challenges Please help.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,562 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,652 questions
{count} votes

Accepted answer
  1. Boris Von Dahle 3,221 Reputation points
    2023-04-15T15:44:37.67+00:00

    Hello,

    Are these ADLS separately created individually? Yes, in a federated architecture, each ADLS account is created separately. This allows organizations to distribute their data storage across multiple accounts for improved scalability and flexibility.

    How does one ADLS interact with other ADLS? Interaction between ADLS accounts can be achieved using data movement tools like Azure Data Factory, Azure Databricks, or custom applications using Azure SDKs. You can read data from one ADLS account and write it to another, allowing for data processing, transformation, or aggregation.

    How is security and integrity maintained? Each ADLS account can have its own security configurations. You can use Azure AD integration for authentication, and RBAC (Role-Based Access Control) for authorization. Data can be encrypted at rest using Azure Storage Service Encryption and in transit using SSL/TLS. To maintain data integrity, ADLS supports Azure Storage Service redundancy options like LRS, ZRS, GRS, or RA-GRS.

    Implementation challenges:

    • Ensuring consistent security configurations across all ADLS accounts.
    • Managing access controls and permissions for multiple accounts.
    • Data movement and orchestration between accounts can add complexity.
    • Monitoring and managing multiple ADLS accounts can be challenging.

    Costs :

    • Having multiple ADLS accounts may increase storage and data transfer costs.
    • Data movement between accounts may lead to additional costs for data egress.
    • Managing multiple accounts can require more administrative effort and resources. The right choice beetwen central or federated architecture for your ADLS deployment will depend on factors like data size, scalability requirements, security, data access patterns, and cost considerations.
    1 person found this answer helpful.
    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.