ARM deployment completes successfully while DSC configuration is still working

Question

I have a DSC extension intended to bootstrap and harden my Azure VMs. It includes several steps which require a reboot. The LCM is configured to reboot automatically and to continue configuration after the reboot. It appears that the DSC extension doesn't realise that the DSC is still in progress after the first reboot.

The problem with this is that if I want to apply further customisation through the custom script extension it will go ahead an attempt to deploy that script while the DSC configuration is still going and the server will reboot several more times.

As an aside, I can't remove the reboots since some of the DSC components which install software via MSI fail if there is a pending reboot.

Is there a way to handle this or is it a bug in the current version (2.83.5) of the DSC extension?

Answer 1

Hi jeremyahagan,

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

Issue: When you’re using Azure DSC (Desired State Configuration) to set up and secure your VMs, some steps require the VM to reboot. The Local Configuration Manager (LCM) is set to automatically reboot and continue the configuration after each reboot. However, the DSC extension doesn’t always recognize that the configuration is still in progress after the first reboot. The server might end up in an unstable state because both DSC and the Custom Script Extension are trying to make changes simultaneously. Multiple reboots can disrupt the Custom Script Extension’s execution, causing it to fail or produce unexpected results.

Suggested Workarounds: To avoid this, you need a way to ensure that the Custom Script Extension only runs after the DSC configuration is completely finished. This can be achieved by using coordination mechanisms like flag files, registry keys, or breaking down the DSC configuration into smaller, sequential steps.

1. Using Flag Files: You can create a flag file at the end of your DSC configuration to signal completion. The Custom Script Extension can then check for this file before proceeding.
https://learn.microsoft.com/en-us/powershell/dsc/reference/resources/windows/fileresource?view=dsc-1.1

2. Using Registry Keys: You can use the DSC Registry Resource to set a registry key once the configuration is complete. The Custom Script Extension can check this registry key before running. Documentation: https://learn.microsoft.com/en-us/powershell/dsc/reference/resources/windows/registryresource?view=dsc-1.1

3.Breaking Down DSC Configurations: Consider breaking down your DSC configuration into smaller, sequential steps. Each step can set a flag or registry key that the next step or the Custom Script Extension can check.

General DSC Documentation: https://learn.microsoft.com/en-us/powershell/dsc/reference/microsoft/windows/registry/overview?view=dsc-3.0

This article provides information about each version of the Azure DSC VM extension, what environments it supports, and comments and remarks on new features or changes.
https://learn.microsoft.com/en-us/azure/automation/automation-dsc-extension-history

If the above suggestion is helpful, please click "Upvote it."

Answer 2

I have the answer to this question. The TL;DR version is that the DSC extension handler is supposed to detect, handle the reboot and resume the configuration until it is done, HOWERVER this can be ineffective if something else (i.e. the LCM) reboots the node before the handler can detect what happens. It also requires you to include a LocalConfigurationManager block inside your DSC config that configures the reboots.

For the longer version, read on. Thanks to this excellent deep dive it showed the precise configuration you need to ensure everything works smoothly. The sample config from the linked blog is listed below:

Configuration DSCPush {
    Node localhost {
        LocalConfigurationManager {
            ConfigurationMode = 'ApplyAndMonitor'
            RebootNodeIfNeeded = $true
            ActionAfterReboot = 'ContinueConfiguration'
        }
        File SomeRandomFile {
            DestinationPath = "C:\Temp\RandomFile.txt"
            Contents = "Hello World!"
            Type = "File"
        }
        Script ForceReboot {
            GetScript = {
                Try{
                    Get-ItemProperty -Path "HKLM:\Software\DSCPushProcess\" -Name Reboot -ErrorAction Stop
                    Return @{"Result"=$true}
                }
                Catch{
                    Return @{"Result"=$false}
                }
            }
            TestScript = {
                Try{
                    Get-ItemProperty -Path "HKLM:\Software\DSCPushProcess\" -Name Reboot -ErrorAction Stop
                    Return $true
                }
                Catch{
                    Return $false
                }
            }
            SetScript = {
                New-Item -Path "HKLM:\Software\DSCPushProcess" -Force
                New-ItemProperty -Path "HKLM:\Software\DSCPushProcess\" -Name Reboot -PropertyType Dword -Value 1
                $Global:DSCMachineStatus = 1
            }
            DependsOn = "[File]SomeRandomFile"
        }
        File AnotherRandomFile {
            DestinationPath = "C:\Temp\AnotherRandomFile.txt"
            Contents = "Hello World 2!"
            Type = "File"
            DependsOn = "[Script]ForceReboot"
        }
    }
}

This will create the first file, reboot and then create the second file and only THEN will the execution of the DSC extension terminate.

If you omit this:

LocalConfigurationManager {
	ConfigurationMode = 'ApplyAndMonitor'
	RebootNodeIfNeeded = $true
	ActionAfterReboot = 'ContinueConfiguration'
}

Then the DSC script will either complete and be in a pending reboot state or reboot itself and the extension will terminate prematurely (depending on how your LCM is configured)