SQLIaasExtension does not work with Azure Disk Encryption and DomainJoin/Reboot

Question

I am not sure where to post this, but there is an incompatibility with the Sql Server for Virtual Machines extension and the Azure Disk Encryption Extension and the JsonADDomainExtension.

If you use all three, the SQL extension fails to create the storage configuration with the following error:

Error: 'Failed to get all physical disks in the same storage pool.'

I believe the issue is because the Azure Disk Encryption extension creates a new BEK Volume and then the Domain Join Extension reboots the vm. Once those two things have happened, the SQLIaasExtension is no longer able to configure the storage.

The following Bicep template replicates. I have used variables rather than parameters as I have been testing this over 4 days, so if you need to test, you need to change the variables at the top.

You also need a working test domain in order to replicate. It might be easier to replicate with just a reboot.

   // param deploymentId string = take(newGuid(), 5)  
   param location string = resourceGroup().location  
     
   var serverName = 'vm-test'  
   var osDiskName = '${serverName}_osdisk'  
   var computerName = 'hub-test'  
   var adminUsername = 'adminuser'  
   var adminPassword = 'AdminUser123'  
   var domainName = 'contoso.local'  
   var domainAdminUser = 'user'  
   var domainAdminPassword = 'pass'  
   var OS = 'Windows'  
   var sku = 'Standard_B2s'  
   var keyVaultName = 'existingKeyVaultName'  
   var plan = {}   
   var image = {  
     publisher: 'microsoftsqlserver'  
     offer: 'sql2019-ws2022'  
     sku: 'web-gen2'  
     version: 'latest'  
   }  
     
   var dataDisks = [{  
     name: '${serverName}-datadisk_1'  
     diskSizeGB: 64  
     managedDisk: {  
       storageAccountType: 'Premium_LRS' // Must be premium disks to use the SQL Extensions  
     }  
     lun: 0  
     createOption: 'Empty'  
     caching: 'ReadOnly'  
   }]  
     
   var logDisks = [{  
     name: '${serverName}-logdisk_1'  
     diskSizeGB: 64  
     managedDisk: {  
       storageAccountType: 'Premium_LRS' // Must be premium disks to use the SQL Extensions  
     }  
     lun: 1  
     createOption: 'Empty'  
     caching: 'None'  
   }]  
     
   var dbDataDisksLuns = [0]  
   var dbLogDiskLuns = [1]  
     
   // Existing Keyvault  
     
   resource keyvault 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {  
     name: keyVaultName  
   }  
     
   var allDisks = concat(dataDisks, logDisks)  
     
   // Data Disks  
   resource dataDiskResource 'Microsoft.Compute/disks@2022-03-02' = [for dataDisk in allDisks: {  
     name: dataDisk.name  
     location: location  
     sku: {      
       name: dataDisk.managedDisk.storageAccountType  
     }  
     properties: {  
       creationData: {  
         createOption: 'Empty' // could use dataDisk.createOption    
       }  
       diskSizeGB: dataDisk.diskSizeGB  
     }  
   }]  
     
   // Network Interface  
     
   resource networkInterface 'Microsoft.Network/networkInterfaces@2021-03-01' = {  
     name: 'nic-isams-hosting-hub-brett-dev'  
     location: location  
     properties: {  
       enableIPForwarding: false  
       enableAcceleratedNetworking: false  
       ipConfigurations: [  
         {  
           name: 'ipconfig1'  
           properties: {  
             privateIPAllocationMethod: 'Dynamic'  
             privateIPAddress: ''  
             subnet: {  
               id: resourceId('Microsoft.Network/virtualNetworks/subnets', 'vnet-isams-hosting-hub-dev', 'snet-isams-hosting-hub-db-dev-01')  
             }  
           }  
         }  
       ]  
     }  
   }  
     
   // Virtual Machine  
     
   resource server 'Microsoft.Compute/virtualMachines@2022-03-01' = {  
     name: serverName  
     location: location  
     identity: {  
       type: 'SystemAssigned'  
     }  
     properties: {  
       licenseType: (OS == 'Windows') ? 'Windows_Server' : null // hybrid license  
       hardwareProfile: {  
         vmSize: sku  
       }  
       osProfile: {  
         computerName: computerName  
         adminUsername: adminUsername  
         adminPassword: adminPassword  
       }  
       storageProfile: {  
           imageReference: image  
           osDisk: {            
               name: osDiskName  
               createOption: 'FromImage'  
               managedDisk: {  
                 storageAccountType: 'StandardSSD_LRS'  
               }  
               osType: 'Windows'            
           }  
           dataDisks: [for dataDisk in allDisks: {  
             createOption: 'Attach'  
             managedDisk: {  
               id: resourceId('Microsoft.Compute/disks', dataDisk.name)  
             }  
             lun: dataDisk.lun  
             caching: empty(dataDisk.caching) ? null : dataDisk.caching  
           }]  
         }      
       networkProfile: {  
         networkInterfaces: [  
           {  
             id: networkInterface.id  
           }  
         ]  
       }  
       diagnosticsProfile: {  
         bootDiagnostics: {  
           enabled: false  
         }  
       }  
     }  
     plan: !empty(plan) ? plan : null  
   }  
     
   // domain join  
     
   resource domainJoin 'Microsoft.Compute/virtualMachines/extensions@2022-03-01' = {  
     name: '${serverName}/domainjoin'  
     location: location  
     properties: {  
       publisher: 'Microsoft.Compute'  
       type: 'JsonADDomainExtension'  
       typeHandlerVersion: '1.3'  
       autoUpgradeMinorVersion: true  
       settings: {  
         Name: domainName  
         OUPath: ''  
         User: domainAdminUser  
         Restart: true  
         Options: 3  
       }  
       protectedSettings: {  
         Password: domainAdminPassword  
       }  
     }  
     dependsOn: [  
       server  
     ]  
   }  
     
   resource azureDiskEncryption 'Microsoft.Compute/virtualMachines/extensions@2021-11-01' = if (OS == 'Windows') {  
     parent: server  
     name: 'AzureDiskEncryption'  
     location: location  
     properties: {  
       publisher: 'Microsoft.Azure.Security'  
       type: 'AzureDiskEncryption'  
       typeHandlerVersion: '2.2'  
       autoUpgradeMinorVersion: true  
       settings: {  
         EncryptionOperation: 'EnableEncryption'  
         KeyVaultURL: keyvault.properties.vaultUri  
         KeyVaultResourceId: keyvault.id  
         VolumeType: 'All'  
       }  
     }  
   }  
     
     
   resource sqlVmAssociation 'Microsoft.SqlVirtualMachine/sqlVirtualMachines@2022-02-01' = {  
     name: serverName  
     location: location  
     properties: {  
       virtualMachineResourceId: server.id  
       sqlImageOffer: 'SQL2019-WS2022'  
       sqlServerLicenseType: 'PAYG'  
       sqlManagement: 'Full'  
       sqlImageSku: 'Web'  
       autoPatchingSettings: {  
         enable: false  
       }  
       storageConfigurationSettings: {  
         diskConfigurationType: 'NEW'  
         storageWorkloadType: 'OLTP'  
         sqlDataSettings: {  
           luns: dbDataDisksLuns  
           defaultFilePath: 'F:\\SQLData'  
         }  
         sqlLogSettings: {  
           luns: dbLogDiskLuns  
           defaultFilePath: 'G:\\SQLLog'  
         }  
         sqlTempDbSettings: {  
           dataFileCount: 4  
           dataFileSize: 512  
           dataGrowth: 512  
           logFileSize: 512  
           logGrowth: 512  
           defaultFilePath: 'D:\\SQLTemp'  
         }  
         sqlSystemDbOnDataDisk: true  
       }  
       serverConfigurationsManagementSettings: {  
         sqlInstanceSettings: {  
           isOptimizeForAdHocWorkloadsEnabled: true          
         }  
         // sqlWorkloadTypeUpdateSettings: {  
         //   sqlWorkloadType: 'OLTP'  
         // }  
         sqlConnectivityUpdateSettings:{  
           connectivityType: 'PRIVATE'  
           port: 1433  
         }  
       }  
     }  
     dependsOn:[  
       domainJoin  
     ]  
   }  

Answer 1

@Richard - We solved this simply by changing the order of the tasks. Its pretty simple to get it working. I would need to check, but I suspect you need to do the disk encryption last and have it depend on the SQLIaasExtension.