Azure Compute https://docs.microsoft.com/archive/blogs/mckittrick/feed.xml Micah McKittrick Sun, 17 Feb 2019 18:38:23 GMT en-US hourly 1 Unable to RDP to Virtual Machine: CredSSP Encryption Oracle Remediation https://docs.microsoft.com/archive/blogs/mckittrick/unable-to-rdp-to-virtual-machine-credssp-encryption-oracle-remediation Fri, 11 May 2018 10:12:50 GMT https://blogs.technet.microsoft.com/mckittrick/?p=2005 This blog is being migrated to www.solveazure.com and will be decommissioned once the content has been moved. ETA 2/28/2019

Please use www.solveazure.com for updated and continuous content. 

================================================================================================================= This specific post has moved to https://solveazure.com/2018/05/11/unable-to-rdp-to-virtual-machine-credssp-encryption-oracle-remediation/  =================================================================================================================]]> Windows Update Issues - Clearing the Windows Update Cache https://docs.microsoft.com/archive/blogs/mckittrick/windows-update-issues-clear-windows-update-cache Mon, 07 Aug 2017 13:58:25 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1935 This blog is being migrated to www.solveazure.com and will be decommissioned once the content has been moved. ETA 2/28/2019

Please use www.solveazure.com for updated and continuous content. 

  1. To start, we need to stop the Windows update service before we can clear the update cache. To do that, search for “Services” in the Start menu and open it. If you are using your system as a standard user, then open it as an administrator using the right-click menu.
  2. Once the Services window has been opened, find the service “Windows Update,” right-click on it and then select the option “Stop.” This action stops the Windows update service.
  3. After stopping the service, press “Win + R,” enter the below path and press the Enter button. This is the folder where Windows stores all the files related to Windows updates.C:\Windows\SoftwareDistribution\
  4. Once you are in this folder, open the “Download” folder, select all the files in it and just press the Delete key on your keyboard to delete all the files. You might be prompted for administrator permissions; simply click on the “Continue” button to complete the procedure.
  5. Open the Services window again and start the “Windows Update service again by right-clicking on it and then select the option “Start.”
  6. Check for and install updates
]]> Troubleshoot a Broken Azure VM using Nested Virtualization in Azure (Managed Disk) https://docs.microsoft.com/archive/blogs/mckittrick/troubleshoot-a-broken-azure-vm-using-nested-virtualization-in-azure-managed-disk Thu, 03 Aug 2017 16:23:53 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1866 Please note that 3rd party virtualization software(s) for Windows 2016 Nested Virtualization feature is not supported. Only Hyper-V is currently supported. Also note that as of right now, Nested Virtualization is only available in the following geographic locations: US -West 2 -East Europe -West Asia Pacific -Southeast Reference Links: Nested Virtualization in Azure New DV3 and EV3 Series Virtual Machines
  1. Delete the Broken Virtual machine
  2. Create a new Recovery VM
    1. OS: Windows Server 2016 Datacenter
    2. Select the same Location and Resource Group as the Broken VM
    3. Size: Any V3 series with at least 2 cores
    4. Select to use Managed Disks
  3. Once the Recovery VM is created select it in the Portal
  4. Select Disks
  5. Add Data Disk
  6. Select the Broken VHDs OS disk from the drop down list
  7. Save the Changes
  8. Once the disk has successfully attached RDP to the Recovery VM
  9. In Server Manger select Manage -> Add Roles and Features
  10. For Installation Type select Role-based or feature-based installation
  11. Ensure the Recovery VM is selected
  12. Select the Hyper-V role
  13. Select Next on the Features Screen.
    1. If a Virtual Switch is available, select it and continue. Otherwise select Next and continue without a switch
  14. On the Migration Page select Next
  15. On the Default Stores Page select Next
  16. Check the box to restart the server automatically if required
  17. Select Install
  18. Allow the Server to install the Hyper-V role. This will take a few mins and the server will reboot automatically. You can monitor the reboot cycle using the Boot Diagnostics Tab in the Azure Portal
  19. Once the VM starts back up RDP back into it
  20. Open Disk Management and ensure the VHD of the Broken VM is set to Offline
  21. In Server Manger select the Hyper-V role
  22. Right click on the server and select the Hyper-V Manger
  24. In the Hyper-V manager right click on the Recovery VM and select New -> Virtual Machine
  25. Select Next
  26. Name the Virtual Machine and select Next
  27. Select Generation 1
  28. Leave the startup memory at 1024MB unless your VM requires more.
  29. If applicable select the Hyper-V Network Switch that was created. Else just move on to the next page
  30. Chose the option to Attach a Virtual Hard Disk Later
  31. Select Finish and the VM will be created
  32. Right click on the VM we just created and select Settings
  33. Select IDE Controller 0
  34. Select Hard Drive and click Add
  35. Under Physical Hard Disk select the VHD of the broken VM we attached to the Recovery VM
    1. If you do not see any disks listed it is because the VHD attached to the Recovery VM is set to Online. Refer back to step 22
  36. Select Apply and OK
  37. Double Click on the VM and Start it
  38. Depending on your scenario, apply any troubleshooting/ mitigations steps now that you have console access
  39. Once you get the VM back online shutdown the VM in the Hyper-V manager
  40. Go back to the Azure Portal
  41. Select the Recovery VM
  42. Select Disks
  43. Select Edit and remove the now fixed VHD from the Recovery VM and save the changes
  44. Proceed to rebuild the now fixed VM. Refer to How to Rebuild an Azure VM from an Existing Managed Disk
]]> Troubleshoot a Broken Azure VM using Nested Virtualization in Azure (RDFE) https://docs.microsoft.com/archive/blogs/mckittrick/troubleshoot-broken-azure-vm-using-nested-virtualization-in-azure-rdfe Thu, 03 Aug 2017 14:54:38 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1786 Please note that 3rd party virtualization software(s) for Windows 2016 Nested Virtualization feature is not supported. Only Hyper-V is currently supported. Also note that as of right now, Nested Virtualization is only available in the following geographic locations: US -West 2 -East Europe -West Asia Pacific -Southeast Reference Links: Nested Virtualization in Azure New DV3 and EV3 Series Virtual Machines
  1. Create a new Recovery VM
    1. OS: Windows Server 2016 Datacenter
    2. Size: Any V3 series with at least 2 cores
    3. Same Location, Storage Account and Resource Group as the Broken VM
    4. Select the same storage type as the broken VM (Standard or Premium)
  2. Delete the Broken VM. Ensure to KEEP all attached disks
  3. Once the Broken VM is deleted navigate to the Recovery VM
  4. Select Disks
  5. Select Attach Existing
  6. Browse to the location of the Broken VMs VHD
  7. Select the Broken VMs VHD and click OK
  8. Once the disk has successfully attached RDP to the Recovery VM
  9. In Server Manger select Manage -> Add Roles and Features
  10. For Installation Type select Role-based or feature-based installation
  11. Ensure the Recovery VM is selected
  12. Select the Hyper-V role
  13. Select Next on the Features Screen.
    1. If a Virtual Switch is available, select it and continue. Otherwise select Next and continue without a switch
  14. On the Migration Page select Next
  15. On the Default Stores Page select Next
  16. Check the box to restart the server automatically if required
  17. Select Install
  18. Allow the Server to install the Hyper-V role. This will take a few mins and the server will reboot automatically. You can monitor the reboot cycle using the Boot Diagnostics Tab in the Azure Portal
  19. Once the VM starts back up RDP back into it
  20. Open Disk Management and ensure the VHD of the Broken VM is set to Offline
  21. In Server Manger select the Hyper-V role
  22. Right click on the server and select the Hyper-V Manger
  23. In the Hyper-V manager right click on the Recovery VM and select New -> Virtual Machine
  24. Select Next
  25. Name the Virtual Machine and select Next
  26. Select Generation 1
  27. Leave the startup memory at 1024MB unless your VM requires more.
  28. If applicable select the Hyper-V Network Switch that was created. Else just move on to the next page
  29. Chose the option to Attach a Virtual Hard Disk Later
  30. Select Finish and the VM will be created
  31. Right click on the VM we just created and select Settings
  32. Select IDE Controller 0
  33. Select Hard Drive and click Add
  34. Under Physical Hard Disk select the VHD of the broken VM we attached to the Recovery VM
    1. If you do not see any disks listed it is because the VHD attached to the Recovery VM is set to Online. Refer back to step 22
  35. Select Apply and OK
  36. Double Click on the VM and Start it
  37. Depending on your scenario, apply any troubleshooting/ mitigations steps now that you have console access
  38. Once you get the VM back online shutdown the VM in the Hyper-V manager
  39. Go back to the Azure Portal
  40. Select the Recovery VM
  41. Select Disks
  42. Detach the now fixed VHD  from the Recovery VM
  43. Proceed to rebuild the now fixed VM. Refer to How to Rebuild an Azure VM from an Existing Disk OS disk
]]> Troubleshoot a Broken Azure VM using Nested Virtualization in Azure (ARM) https://docs.microsoft.com/archive/blogs/mckittrick/troubleshoot-broken-azure-vm-using-nested-virtualization-in-azure-arm Thu, 03 Aug 2017 14:49:06 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1587 Please note that 3rd party virtualization software(s) for Windows 2016 Nested Virtualization feature is not supported. Only Hyper-V is currently supported.
  1. Delete the Broken Virtual machine
  2. Create a new Recovery VM
    1. OS: Windows Server 2016 Datacenter
    2. Size: Any V3 series with at least 2 cores
    3. Same Location, Storage Account and Resource Group as the Broken VM
    4. Select to not use Managed Disks
  3. Once the Recovery VM is created select it in the Portal
  4. Select Disks
  5. Add Data Disk
  6. Select Source Type Existing Blob
  7. Browse to the location of the Broken VMs VHD
  8. Select the Broken VMs VHD and click OK
  9. Save the Changes
  10. Once the disk has successfully attached RDP to the Recovery VM
  11. In Server Manger select Manage -> Add Roles and Features
  12. For Installation Type select Role-based or feature-based installation
  13. Ensure the Recovery VM is selected
  14. Select the Hyper-V role
  15. Select Next on the Features Screen.
    1. If a Virtual Switch is available, select it and continue. Otherwise select Next and continue without a switch
  16. On the Migration Page select Next
  17. On the Default Stores Page select Next
  18. Check the box to restart the server automatically if required
  19. Select Install
  20. Allow the Server to install the Hyper-V role. This will take a few mins and the server will reboot automatically. You can monitor the reboot cycle using the Boot Diagnostics Tab in the Azure Portal
  21. Once the VM starts back up RDP back into it
  22. Open Disk Management and ensure the VHD of the Broken VM is set to Offline
  23. In Server Manger select the Hyper-V role
  24. Right click on the server and select the Hyper-V Manger
  26. In the Hyper-V manager right click on the Recovery VM and select New -> Virtual Machine
  27. Select Next
  28. Name the Virtual Machine and select Next
  29. Select Generation 1
  30. Leave the startup memory at 1024MB unless your VM requires more.
  31. If applicable select the Hyper-V Network Switch that was created. Else just move on to the next page
  32. Chose the option to Attach a Virtual Hard Disk Later
  33. Select Finish and the VM will be created
  34. Right click on the VM we just created and select Settings
  35. Select IDE Controller 0
  36. Select Hard Drive and click Add
  37. Under Physical Hard Disk select the VHD of the broken VM we attached to the Recovery VM
    1. If you do not see any disks listed it is because the VHD attached to the Recovery VM is set to Online. Refer back to step 22
  38. Select Apply and OK
  39. Double Click on the VM and Start it
  40. Depending on your scenario, apply any troubleshooting/ mitigations steps now that you have console access
  41. Once you get the VM back online shutdown the VM in the Hyper-V manager
  42. Go back to the Azure Portal
  43. Select the Recovery VM
  44. Select Disks
  45. Select Edit and remove the now fixed VHD from the Recovery VM and save the changes
  46. Proceed to rebuild the now fixed VM. Refer to How to Rebuild an Azure VM from an Existing Disk OS disk
]]> Windows Boot Error 0xC0000001 (ARM) https://docs.microsoft.com/archive/blogs/mckittrick/windows-boot-error-0xc0000001-arm Wed, 31 May 2017 16:33:55 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1535 Reason for TSG:  The VM screenshot shows error 0xC0000001.
This is due to a BCD corruption Suggested Fix: 1) Delete the broken VM and attach it to a recovery machine 2) Open up an elevated CMD and gather the current booting setup info and document it on the case. You will see that on the windows partition (the other partition than {bootmgr} ), both the DEVICE and OSDEVICE values are missing. Furthermore, we will use this step to take note of the identifier on the active partition
bcdedit /store <drive letter>:\boot\bcd /enum
3) Then run the following commands:
bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {bootmgr} device partition=<BCD FOLDER - DRIVE LETTER> bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {bootmgr} integrityservices enable bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {<IDENTIFIER>} device partition=<WINDOWS FOLDER - DRIVE LETTER> bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {<IDENTIFIER>} integrityservices enable bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {<IDENTIFIER>} recoveryenabled Off bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {<IDENTIFIER>} osdevice partition=<WINDOWS FOLDER - DRIVE LETTER> bcdedit /store <BCD FOLDER - DRIVE LETTER>:\boot\bcd /set {<IDENTIFIER>} bootstatuspolicy IgnoreAllFailures
Note: The identifier could be Default or a GUID such as 32ebc729-6296-41b0-9750-5d275c7f5439 Here is a sample of what it should look like:
bcdedit /store G:\boot\bcd /set {bootmgr} device partition=G: bcdedit /store G:\boot\bcd /set {bootmgr} integrityservices enable bcdedit /store G:\boot\bcd /set {32ebc729-6296-41b0-9750-5d275c7f5439} device partition=F: bcdedit /store G:\boot\bcd /set {32ebc729-6296-41b0-9750-5d275c7f5439} integrityservices enable bcdedit /store G:\boot\bcd /set {32ebc729-6296-41b0-9750-5d275c7f5439} recoveryenabled Off bcdedit /store G:\boot\bcd /set {32ebc729-6296-41b0-9750-5d275c7f5439} osdevice partition=F: bcdedit /store G:\boot\bcd /set {32ebc729-6296-41b0-9750-5d275c7f5439} bootstatuspolicy IgnoreAllFailures
5) Detach the disk from the recovery machine 6) Recreate the VM from now the fixed OS disk]]> Linux Boot Error: Emergency Mode! (RDFE) https://docs.microsoft.com/archive/blogs/mckittrick/linux-boot-error-emergency-mode-rdfe Wed, 31 May 2017 15:16:39 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1465 Reason for TSG: Boot diagnostics screen shows VM booting into Emergency Mode   Suggested Fix: 1) Before proceeding with Step 2 ensure you have an recovery VM located in the same Cloud Service, region and the same version (Or distribution) as the impacted VM. If you do not, create that new machine now. 2) Once you have the recovery machine created proceed to delete the affected virtual machine from the Azure Portal. Ensure to opt to KEEP all attached disks

3) Once the machine is deleted wait about 3 mins for the disk lease to expire 4) Select the recovery VM you made and select Disks -> Attach Existing
5) Navigate to the location of the VHD of the broken VM we just deleted. The path will be something like Storage Account Name -> vhds -> VHDname 6) Select the affected VHD and attach it to the troubleshooting VM. 7) Once the VHD is attached proceed to SSH to the recovery machine. 8) From now on lets refer machines using the following naming convention:
  A = Original VM (Inaccessible VM)   B = New VM (New Troubleshooting VM)
9) Once you have connected to VM B locate the drive name to mount, each Linux distribution is slightly different.
For Ubuntu/debian: grep SCSI /var/log/kern.log For Centos/Suse/Oracle/Redhat: grep SCSI /var/log/messages
10) Mount the attached disk onto mountpoint /rescue
df -h mkdir /rescue For Red Hat 7.2+ mount -o nouuid /dev/sdc2 /rescue For CentOS 7.2+ mount -o nouuid /dev/sdc1 /rescue For Debian 8.2+, Ubuntu 16.04+, SUSE 12 SP4+ mount /dev/sdc1 /rescue
11) Change into /etc directory where the original OS disk from resides
cd /rescue/etc/ cp fstab fstab_orig
12) Now that you have made a backup of you fstab you can proceed to make the changes you require using vi, nano or your favorite text editor, this may include commenting out entries by appending a # at the start of the line.
vi fstab cd / umount /rescue
13) Detach the disk from VM B via the Azure portal 14) Recreate the original VM A from the repaired VHD 15) To do this navigate to OS Disks (Classic)
16) Select the now fixed VHD and select + Create VM
17) Recreate the VM putting it back into the original Cloud Service and using the same name 18) Once it finishes creating attempt to SSH to it]]> How to delete a VM and attach the OS disk as a Data Disk to a Recovery VM (RDFE) https://docs.microsoft.com/archive/blogs/mckittrick/how-to-delete-a-vm-and-attach-the-os-disk-as-a-data-disk-to-a-recovery-vm-rdfe Fri, 26 May 2017 15:47:15 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1425 5) Select Configure Required Settings  -> Storage Account of VHD -> Container of VHD 6) Select the VHD of the broken VM 7) Wait for the operation to complete and login to your VM 8) Open File Explorer to ensure the disk is showing up 9) If not, bring the disk online via the Disk Management Application inside the OS]]> How to delete a VM and attach the OS disk as a Data Disk to a Recovery VM (ARM) https://docs.microsoft.com/archive/blogs/mckittrick/how-to-delete-a-vm-and-attach-the-os-disk-as-a-data-disk-to-a-recovery-vm-arm Fri, 26 May 2017 15:45:16 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1386 5) Change the Source type to Existing Blog 6) Browse to the Source Blog 7) Locate the VHD of the broken VM in the correct Storage Account and Container 8) Select the VHD and click OK 9) Wait for the operation to complete and login to your VM 10) Open File Explorer to ensure the disk is showing up 11) If not, bring the disk online via the Disk Management Application inside the OS]]> How to Perform an In-Place-Upgrade in Azure (Windows) https://docs.microsoft.com/archive/blogs/mckittrick/how-to-perform-an-in-place-upgrade-in-azure-windows Fri, 26 May 2017 15:42:10 GMT https://blogs.technet.microsoft.com/mckittrick/?p=1375 not supported.   In order to upgrade the machine to a new version of Windows it is required that a new machine is created using the desired Operating System and the data migrated. If an update is initialized from inside the OS the result will be a complete loss of RDP access.   In order to restore RDP access from that point console access is needed. Since console access is not possible in Azure the VHD will need to be downloaded on premises and spun up in an on premise virtual environment such as Hyper-V. Once the OS upgrade is completed the machine will need to be configured and uploaded back to Azure. Please see KB2721672 for additional information regarding in-place upgrades inside of Azure Virtual Machines.   EDIT 9/6/17: As of today, you can now gain console access using Nested Virtualization. Please see the following articles on how to achieve this:
  • https://blogs.technet.microsoft.com/mckittrick/troubleshoot-broken-azure-vm-using-nested-virtualization-in-azure-arm/
  • https://blogs.technet.microsoft.com/mckittrick/troubleshoot-a-broken-azure-vm-using-nested-virtualization-in-azure-managed-disk/
  • https://blogs.technet.microsoft.com/mckittrick/troubleshoot-broken-azure-vm-using-nested-virtualization-in-azure-rdfe/
This post will be updated shortly to reflect all these changes.]]>