Provisioning New Component Vs Using the same Component

Question

I am building a Unified Data Lake Solution using Azure component like.

Azure Event hub, Stream analytics, ADLS Gen 2 and Azure SQL.

There are 5 source. SAP, SALESFORCE, CIAM, ADOBE and FB.

I have only 1 Question:

Do I need to use seperate ADF and ADLS Gen2 for each source.

How the cost will be affected if we are using new instance of the component VS utilising the same.

when to use new instance of the component vs reuse the existing one.

Please share your thoughts.

Answer 1

Hi @Relay
When building a unified data lake solution using Azure Event Hub, Stream Analytics, Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS Gen2), and Azure SQL it's important to consider cost, security, manageability, and scalability when deciding whether to provision separate components or reuse existing ones across multiple sources like SAP, Salesforce, CIAM, Adobe, and Facebook.

Azure Data Factory (ADF)

You can reuse a single ADF instance for all sources by:

• Creating separate pipelines per source
• Using parameterized linked services and datasets
• Applying naming conventions to organize and manage workflows

This approach is recommended unless:

• You have strict isolation or regulatory requirements
• Separate teams manage different data pipelines and require independent deployments

https://learn.microsoft.com/en-us/azure/data-factory/how-to-expression-language-functions

Azure Data Lake Storage Gen2 (ADLS Gen2)

You can also reuse one ADLS Gen2 account and separate data by:

• Creating distinct containers (e.g., sap/, salesforce/)
• Organizing with folder structures (e.g., /raw/sap/, /curated/ciam/)
• Applying RBAC or POSIX ACLs to manage secure access

This approach keeps cost low and simplifies governance.

Provision separate ADLS accounts only if you need:

• Strict data residency compliance
• Billing isolation
• Blast-radius containment

https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-best-practices

Cost Considerations

Provisioning new ADF and ADLS instances per source increases cost:

• Each ADF instance has separate Integration Runtime and monitoring overhead.
• Each ADLS account incurs minimum throughput and storage costs.

Reusing shared components is more cost-efficient and easier to manage, especially when unified governance is required.

https://azure.microsoft.com/en-us/pricing/details/data-factory/data-pipeline/?cdn=disable
https://azure.microsoft.com/en-us/pricing/details/storage/blobs/?cdn=disable

Provision new instances only if:

• Isolation is needed for compliance or organizational boundaries
• You have a multi-tenant architecture
• Teams require independent deployments
• You expect high ingestion volume per source requiring scalability

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.