Troubleshoot issues in Azure Stack Hub
This document provides troubleshooting information for Azure Stack Hub integrated environments. For help with the Azure Stack Development Kit, see ASDK Troubleshooting or get help from experts on the Azure Stack Hub MSDN Forum.
Frequently asked questions
These sections include links to docs that cover common questions sent to Microsoft Support.
Purchase considerations
Updates and diagnostics
- How to use diagnostics tools in Azure Stack Hub
- How to validate Azure Stack Hub system state
- Update package release cadence
- Verify and troubleshoot node status
Supported operating systems and sizes for guest VMs
Azure Marketplace
Manage capacity
Memory
To increase the total available memory capacity for Azure Stack Hub, you can add additional memory. In Azure Stack Hub, your physical server is also referred to as a scale unit node. All scale unit nodes that are members of a single scale unit must have the same amount of memory.
Retention period
The retention period setting lets a cloud operator to specify a time period in days (between 0 and 9999 days) during which any deleted account can potentially be recovered. The default retention period is set to 0 days. Setting the value to 0 means that any deleted account is immediately out of retention and marked for periodic garbage collection.
Security, compliance, and identity
Manage RBAC
A user in Azure Stack Hub can be a reader, owner, or contributor for each instance of a subscription, resource group, or service.
If the built-in roles for Azure resources don't meet the specific needs of your organization, you can create your own custom roles. For this tutorial, you create a custom role named Reader Support Tickets using Azure PowerShell.
Manage usage and billing as a CSP
Choose the type of shared services account that you use for Azure Stack Hub. The types of subscriptions that can be used for registration of a multi-tenant Azure Stack Hub are:
- Cloud Solution Provider
- Partner Shared Services subscription
Get scale unit metrics
You can use PowerShell to get stamp utilization information without help from Microsoft Support. To obtain stamp utilization:
Create a PEP session.
Run the following command:
Test-AzureStack
Exit PEP session.
Run the following using an Invoke-Command call:
Get-AzureStackLog -FilterByRole SeedRing
Extract the seedring .zip. You can obtain the validation report from the ERCS folder where you ran
Test-AzureStack
.
For more information, see Azure Stack Hub Diagnostics.
Troubleshoot virtual machines (VMs)
Reset Linux VM password
If you forget the password for a Linux VM and the Reset password option is not working due to issues with the VMAccess extension, you can perform a reset following these steps:
Choose a Linux VM to use as a recovery VM.
Sign in to the User portal:
- Make a note of the VM size, NIC, Public IP, NSG and data disks.
- Stop the impacted VM.
- Remove the impacted VM.
- Attach the disk from the impacted VM as a data disk on the recovery VM (it may take a couple of minutes for the disk to be available).
Sign in to the recovery VM and run the following command:
sudo su - mkdir /tempmount fdisk -l mount /dev/sdc2 /tempmount /*adjust /dev/sdc2 as necessary*/ chroot /tempmount/ passwd root /*substitute root with the user whose password you want to reset*/ rm -f /.autorelabel /*Remove the .autorelabel file to prevent a time consuming SELinux relabel of the disk*/ exit /*to exit the chroot environment*/ umount /tempmount
Sign in to the User portal:
- Detach the disk from the Recovery VM.
- Recreate the VM from the disk.
- Be sure to transfer the Public IP from the previous VM, attach the data disks, etc.
You may also take a snapshot of the original disk and create a new disk from it rather than perform the changes directly on the original disk. For more information, see these topics:
License activation fails for Windows Server 2012 R2 during provisioning
In this case, Windows will fail to activate and you will see a watermark on the bottom-right corner of the screen. The WaSetup.xml logs located under C:\Windows\Panther contains the following event:
<Event time="2019-05-16T21:32:58.660Z" category="ERROR" source="Unattend">
<UnhandledError>
<Message>InstrumentProcedure: Failed to execute 'Call ConfigureLicensing()'. Will raise error to caller</Message>
<Number>-2147221500</Number>
<Description>Could not find the VOLUME_KMSCLIENT product</Description>
<Source>Licensing.wsf</Source>
</UnhandledError>
</Event>
To activate the license, copy the Automatic Virtual Machine Activation (AVMA) key for the SKU you want to activate.
Edition | AVMA Key |
---|---|
Datacenter | Y4TGP-NPTV9-HTC2H-7MGQ3-DV4TW |
Standard | DBGBW-NPF86-BJVTX-K3WKJ-MTB6V |
Essentials | K2XGM-NMBT3-2R6Q8-WF2FK-P36R2 |
On the VM, run the following command:
slmgr /ipk <AVMA_key>
For complete details, see VM Activation.
Default image and gallery item
A Windows Server image and gallery item must be added before deploying VMs in Azure Stack Hub.
I've deleted some VMs, but still see the VHD files on disk
This behavior is by design:
- When you delete a VM, VHDs aren't deleted. Disks are separate resources in the resource group.
- When a storage account gets deleted, the deletion is visible immediately through Azure Resource Manager. But the disks it may contain are still kept in storage until garbage collection runs.
If you see "orphan" VHDs, it's important to know if they're part of the folder for a storage account that was deleted. If the storage account wasn't deleted, it's normal that they're still there.
You can read more about configuring the retention threshold and on-demand reclamation in manage storage accounts.
Troubleshoot storage
Storage reclamation
It may take up to 14 hours for reclaimed capacity to show up in the portal. Space reclamation depends on different factors including usage percentage of internal container files in block blob store. Therefore, depending on how much data is deleted, there's no guarantee on the amount of space that could be reclaimed when garbage collector runs.
Azure Storage Explorer not working with Azure Stack Hub
If you're using an integrated system in a disconnected scenario, it's recommended to use an Enterprise Certificate Authority (CA). Export the root certificate in a Base-64 format and then import it in Azure Storage Explorer. Make sure that you remove the trailing slash (/
) from the Resource Manager endpoint. For more information, see Prepare for connecting to Azure Stack Hub.
Troubleshoot App Service
Create-AADIdentityApp.ps1 script fails
If the Create-AADIdentityApp.ps1 script that's required for App Service fails, be sure to include the required -AzureStackAdminCredential
parameter when running the script. For more information, see Prerequisites for deploying App Service on Azure Stack Hub.
Troubleshoot Azure Stack Hub updates
The Azure Stack Hub patch and update process is designed to allow operators to apply update packages in a consistent, streamlined way. While uncommon, issues can occur during patch and update process. The following steps are recommended should you encounter an issue during the patch and update process:
Prerequisites: Be sure that you have followed the Update Activity Checklist and enable proactive log collection.
Follow the remediation steps in the failure alert created when your update failed.
If you have been unable to resolve your issue, create an Azure Stack Hub support ticket. Be sure you have logs collected for the time span when the issue occurred. If an update fails, either with a critical alert or a warning, it's important that you review the failure and contact Microsoft Customer Support Services as directed in the alert so that your scale unit does not stay in a failed state for a long time. Leaving a scale unit in a failed update state for an extended period of time can cause additional issues that are more difficult to resolve later.
Common Azure Stack Hub patch and update issues
Applies to: Azure Stack Hub integrated systems
PreparationFailed
Applicable: This issue applies to all supported releases.
Cause: When attempting to install the Azure Stack Hub update, the status for the update might fail and change state to PreparationFailed
. For internet-connected systems this is usually indicative of the update package being unable to download properly due to a weak internet connection.
Remediation: You can work around this issue by clicking Install now again. If the problem persists, we recommend manually uploading the update package by following the Install updates section.
Occurrence: Common
Update failed: Check and Enforce external key protectors on CSVs
Applicable: This issue applies to all supported releases.
Cause: The baseboard management controller (BMC) password is not set correctly.
Remediation: Update the BMC credential and resume the update.
Warnings and errors reported while update is in progress
Applicable: This issue applies to all supported releases.
Cause: When Azure Stack Hub update is in status In progress, warnings and errors may be reported in the portal. Components may timeout waiting for other components during upgrade resulting in an error. Azure Stack Hub has mechanism to retry or remediate some of the tasks due to intermittent errors.
Remediation: While the Azure Stack Hub update is in status In progress, warnings and errors reported in the portal can be ignored.
Occurrence: Common
2002 update failed
Applicable: This issue applies only to the 2002 release.
Cause: When attempting the 2002 update, the update might fail and provide this message: The private network parameter is missing from cloud parameters. Please use set-azsprivatenetwork cmdlet to set private networkTrace
.
Remediation: Set up a private internal network.