Azure Environment Creation / Deployment

Question

Hi

I am currently working for a client where were are migrating their IT estate to Azure. This involves migrating database and re-engineering numerous integrations using Azure Function / Logic Apps / Data Factory. All in All we have nearly 200 resources contained within a Resource Group. We have also created VMs, containers, NSG, DB, API, Vnets etc.

We have developed the integration using a combination of Visual Studio and the Azure Portal. All code has been written using C#. The analysis to re-develop the existing integrations has been large and the current integration used a different and old technologies. I did consider creating the dev resources using IaC but the build task has been and ongoing process and integration have been delivered on an ongoing basis.

We are now at the end of build and I need to recreate the resources in SIT, UAT and eventually production. I had envisaged I could export the ARM resource group template and refactor. However, this does not seem possible as I receive an error stating 200 resources exceeded.

Could somebody recommend how this can be achieved instead of manually creating resources in each environment. I thougth Azure would be agile and would provide a capability to recreate resources are speed. If this is not possible then I will have to look at not considering Azure for future clients and bids.

Also could somebody recommend (as a lesson learnt) how we could have created a repeatable process at the outset of the project. I was considering developing Terrafrom scripts but due to timescales in build and the need to analyse the integration this was dropped.

Thanks

Mike

Answer 1

Both ARM and Terraform are valid ways to handle automated Azure deployments at this scale. You could also use the Azure CLI or management API, but since you can't simply export the scripts, using one of these alone isn't going to be the best choice.

While ARM can handle large deployments, there are limits on what a single template can hold, as you have discovered. I also want to acknowledge that, while exporting templates from the portal is the simplest way to create them, it does not handle every situation. This is something that we will continue to improve on. We provide feedback from Q&A to the product teams to help guide their product planning, but you can also post your requests to UserVoice or vote on existing suggestions. The forum is actively monitored, and the voting system helps the product teams prioritize new features.

----------

You can handle this situation by essentially breaking your templates into deployment stages. By selecting specific resources inside the resource group, you will be able to export the templates for just those resources. One way to keep track of which resources are in which template would be to add tags to the resources with that information. You can then filter the list by tag value when exporting templates and ensure that all the resources are defined in an exported template.

There are a few different ways to put the templates back together into a single deployment. It isn't possible to export the piece that links the templates, so you will need to create that part yourself.

1. Linked templates. In this scenario, you would create a main template that handles gathering parameters and passing them to each of your resource templates in turn. The main template is also able to pass values between the templates where you need to do so.
2. Make use of the Azure CLI or management API. While I would suggest against using one of these tools to define your entire deployment at this stage, both tools can deploy ARM templates. These tools are the best option if you want to modify your services after they are deployed without changing your templates.
3. Use the Deployment Manager service that is currently in public preview. The service works similarly to linked templates, but lets you define more clearly how your services are linked together and how they should be deployed. This is the most robust option for large scale deployments. However, it is important to know that preview services aren't recommended for production deployments as bugs and breaking changes are going to be more common.