Azure Archive Solution

Question

Hi Team,

We are planning to migrate approximately 2 PiB of data from our on-premises storage to the Microsoft Azure cloud platform, and we are evaluating the best architectural approach for long-term archiving.

As part of the requirement, we need:

A cost-optimized archiving solution on Azure

A cost comparison between different Azure storage tiers (Hot, Cool, Archive, and Cold)

Support for quarterly data recall, as a portion of the archived data will be retrieved once every three months based on business needs

Automated lifecycle management policies to classify and move data across tiers based on retention, access frequency, and compliance

A central dashboard or monitoring view to track data growth, storage tier alignment, recall trends, and operational costs

Please suggest which service will suits for this requirement.

Answer 1

Hi @ Saravana M

Welcome to Microsoft Q&A Platform.

For your 2 PiB long-term archive with quarterly recall requirements, we recommend using Azure Blob Storage (GPv2) with Hot/Cool/Cold/Archive tiers, ingesting data via Azure Data Box Heavy, and enabling Blob lifecycle management to automatically classify and transition data.

Monitoring and cost governance can be centrally managed using Azure Monitor Storage Insights, Azure Workbooks, and Azure Cost Management. This architecture provides the most cost-optimized, scalable, and operationally efficient solution for long-term archival workloads on Azure.

You can also set up alerts for real-time governance and automation to monitor and secure your storage account.

Refer: Access tiers for blob data and Azure Blob Storage lifecycle management overview and Azure Data Box

Kindly let us know if the above helps or you need further assistance on this issue.

 

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

Answer 2

For your requirement of migrating approximately 2 PiB of data to Azure for long-term archiving, the following Azure services and strategies would be suitable:

1. Azure Blob Storage: This is a cost-effective solution for long-term data archiving. You can utilize the different access tiers (Hot, Cool, and Archive) to optimize costs based on how frequently the data is accessed. Since you mentioned quarterly data recall, the Cool or Archive tiers would be appropriate, as they are designed for infrequent access.
   • Hot Tier: Best for data that is accessed frequently.
   • Cool Tier: Suitable for data that is infrequently accessed and stored for at least 30 days.
   • Archive Tier: Ideal for data that is rarely accessed and can tolerate retrieval times of several hours.
2. Automated Lifecycle Management: Implement data lifecycle policies to automatically move data between the Hot, Cool, and Archive tiers based on predefined criteria such as access frequency and compliance needs. This will help in managing costs effectively while ensuring compliance with retention policies.
3. Monitoring and Management: Use Azure Monitor and Azure Advisor to create a central dashboard for tracking data growth, storage tier alignment, recall trends, and operational costs. Azure Monitor provides insights into your storage usage and performance, helping you optimize your architecture over time.
4. Cost Comparison: The costs associated with each tier vary:
   • Hot Tier: Higher storage costs but lower access costs.
   • Cool Tier: Lower storage costs than Hot but higher access costs.
   • Archive Tier: Lowest storage costs but highest access costs and longer retrieval times.
   • You can utilize Azure Pricing Calculator to get a detailed cost estimate based on your specific usage patterns.

In summary, Azure Blob Storage with appropriate lifecycle management policies and monitoring tools will meet your requirements for a cost-optimized archiving solution.

---

References:

• Optimize data management
• Define storage needs