Heartbeat from the Datacenter https://docs.microsoft.com/archive/blogs/ramazancan/feed.xml All Posts are provided "AS IS" with no warranties, and confers no rights Sat, 24 Feb 2018 12:41:25 GMT en-US hourly 1 SCVMM Powershell - BITSTcpPort https://docs.microsoft.com/archive/blogs/ramazancan/scvmm-powershell-bitstcpport Mon, 19 Feb 2018 17:21:06 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1466 B.I.T.S - Background Intelligent Transfer Service - is also utilized by VMM for a.e. file transfer purposes and if you come across to change the port manually, you basically have to touch the registry of each SCVMM managed computer as there is no way to do this in UI. Especially when you have large scale deployments, PowerShell more specific the VMM module become your best friend The script is really very simple and goes and checks out what key is present and modifies this based on VMM server or agent deployment, that makes it easy and we can run that script over all VMM managed computers doesn't matter if clustered or standalone deployment
first of all we collect computers from VMM and command to use here is Get-SCVMMManagedComputer 
next is the foreach loop to walkthrough of all computers and use a invoke-command with parameter -Scriptblock which has remote capabilities and execute Set-ItemProperty to update BITSTcpPort value. as I mentioned, this is depending of role server / agent therefore I created a check before and based on results I execute the right pattern
Please Note: In my example I had used TCP port 5987 - [int]$BITSTcpPort="5987" - but you can change it to any non-well known port in your environment. Verify in "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager Server\Settings" what ports are already in use by VMM and select a free available port above port 1024
You probably have seen that I do not automatically restart VMM server service so if you want to have that automated too, just remove the # before restart-service command   Once the above change was completed, I did run into following VMM deployment error
Error (2940) VMM is unable to complete the requested file transfer. The connection to the HTTP server vmmserver.contoso.com could not be established. Unknown error (0x80072ee2) Recommended Action Ensure that the HTTP service and/or the agent on the machine vmmserver.contoso.com are installed and running and that a firewall is not blocking HTTP/HTTPS traffic on the configured port.
0x80072ee2 - WININET_E_TIMEOUT - this looks like communication issues so I did some further investigation and finally root cause was the local firewall was blocking the new ports which we configured above. Which makes sense, as this is a non-well-known port and therefore blocked, so all good. if you have Windows firewall configured and actively running, you need to configure exceptions for those new ports we are using for BITS communication in my example TCP 5987. for Windows firewall you can use New-NetFirewallRule or netsh command
New-NetFirewallRule -DisplayName 'VMM Inbound Ports' -Profile @('Domain', 'Private') -Direction Inbound -Action Allow -Protocol TCP -LocalPort @('5985', '5986', '5987')
https://social.technet.microsoft.com/wiki/contents/articles/18465.vmm-2012-change-bits-port-used-for-template-provisioning-bits-over-https.aspx here are also few related articles around VMM, BITS and WMI https://social.technet.microsoft.com/wiki/contents/articles/9058.troubleshoot-vmm-bits-troubleshooting.aspx https://social.technet.microsoft.com/wiki/contents/articles/9048.wmi-troubleshooting-for-vmm.aspx Scripts for Download @ GitHub: https://github.com/ramcandev/powershell/blob/master/VMM_ChangeBITSTcpPort.ps1 https://github.com/ramcandev/powershell/blob/master/VMM_AddWFWRule_VMMBITS.ps1 as usual, any scripts I publish should only be used as examples and usually requires modification before running that against your PROD environment]]> Enable Remote Kernel Mode Debugging on Hyper-V Virtual Machine https://docs.microsoft.com/archive/blogs/ramazancan/enable-remote-kernel-debug-on-virtual-machine Mon, 18 Dec 2017 11:52:56 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1145 1) start an elevated command prompt and turn on debugging bcdedit /debug on 2) configure debugger to use COM1 bcdedit /dbgsettings SERIAL DEBUGPORT:1 BAUDRATE:115200
3) reboot so boot loader can read new bcdstore settings
Now, we need to configure Hyper-V to redirect COM1 to TCP named pipe
1) Hyper-V -> VM settings -> COM1 change from "none" to "pipe name" and provide a descriptive name which we will use later in winbdg Note: Please take a note of the named pipe path as we will use this later in windbg
  • Download and install the Debugging Tools for Windows from https://msdn.microsoft.com/en-us/windows/hardware/gg463009
  • Start 'WinDbg' and choose 'Run as administrator' (Windbg needs administrative rights to connect to the pipe)
    • In windbg open the File menu and choose 'Remote Stub'
    • Here we need 'Remote Stub' path (replace server=HYPERV1 by your Hyper-V hostname)
      • tcp:server=HYPERV1,port=10240
    • then go to 'Kernel Debug'-> COM
      • Enter a Baud Rate of 115200, to match the settings made in the VM and replace HYPERV1 by your Hyper-V hostname a.e.
        • \\HYPERV1\pipe\ramaca160902-125416-VM693
Voila - To verify your debugger connection you can break in which should give you a kd> prompt. From here it's all yours 😊
]]> Powershell - How to detect bad state VMs in a Hyper-V cluster? https://docs.microsoft.com/archive/blogs/ramazancan/powershell-how-to-detect-bad-state-vms Mon, 18 Dec 2017 11:36:52 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1055 # #Creator: ramacan #Last Modified: 06/11/14 # # - Quick & Dirty detection of VMs heartbeat state # - can run from any machine which has failovercluster powershell commandlet installed # - check only for VMs with state running (bug with saved state VMs) #V1.0 - better output view cls "" $clu=Read-Host "Cluster name which should be checked " $Clusternodes=(Get-Cluster $clu | Get-Clusternode).Name | Sort-Object $ClusternodeCount=$Clusternodes.Count [array]$NoHBVMs=@{} | Out-Null "" Write-Host "scan heartbeat status for any VM running in cluster $clu" -foregroundcolor green Write-Host "found"$ClusternodeCount" nodes in cluster $clu" -foregroundcolor green " " foreach ($node in $clusternodes) { $AllVMs=get-vm -ComputerName $node | Sort-Object $AllVMsCount=$AllVMs.Count " " Write-Host "scanning node $node...." -foregroundcolor green Write-Host "found"$AllVMsCount" VMs at node $node" -foregroundcolor green foreach ($VM in $AllVMs) { $VMN=$VM.Name $VMStat=(get-vm -computer $node "$VMN").State if ($VMStat -match "Running") { $HBStatus=(Get-VMIntegrationService -computername $node -VMName $VMN Heartbeat).PrimaryStatusDescription if ($HBStatus -match "No Contact") {Write-Host ""$VM.Name"has HB status - No Contact ! detected on host -> $node" -foregroundcolor yellow $NoHBVMs+=$VM.Name} }}} " " if ($NoHBVMs -ne $null) { Write-Host "Following VMs need to be checked :" -foregroundcolor green $NoHBVMs} else {Write-Host "Great - No VMs detected in cluster $clu which has failed heartbeat" -foregroundcolor green} " " [/notranslate]]]> Windows System State Backup from On-Prem to Azure is generally available now https://docs.microsoft.com/archive/blogs/ramazancan/windows-system-state-backup-from-on-prem-to-azure-is-generally-available-now Wed, 15 Nov 2017 22:04:40 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1286 Then we create a Backup instance   As we are going to configure system state on-prem Now lets’s install the Azure Recovery Services agent on the server on-prem we want to backup in case you are accessing internet through proxy you need to enter here creds/proxy details Once the agent installation is completed we have to register it Registering agent with previously created Azure Recovery Service vault Download and attach the vault credentials to the agent Next we need to generate a passphrase where you either can choose to generate or can provide your own passphrase Please note: If your passphrase is lost or forgotton, the data cannot be restored!! That was easy 😊 lets create now an initial backup from on-prem to Azure Regular Backup Schedule Plan Retention Policy If you are running into issues you can also check properties setting a.e. Proxy configuration. Further you can check here for any error codes around Azure Backup https://social.technet.microsoft.com/wiki/contents/articles/10183.azure-backup-errors-and-events-portal.aspx Once the backup start processing you will see the new agent in your Azure Portal under protected servers Benefits of System State Backup with Azure Backup
  • Comprehensive protection for Active Directory, File-Servers and IIS Web servers System State fully encapsulates Active Directory, which is the most important database in any organization and allows for targeted domain-controller recoveries. In addition, critical cluster information of File Servers and the IIS Web Server Metabase is fully contained in the Windows System State.
  • Centralized management in Azure Once it is backed up, all information related to System State backups across your Windows Servers is available in the Azure portal. You can also configure notifications directly from the Azure portal so you get notified of a failed backup and you can take corrective steps. You can also generate reports using Microsoft Power BI.
  • Cost-effective and secure offsite storage for Windows Server With pay-as-you-go Azure storage, Azure Backup eliminates on-premises infrastructure by directly backing up your Windows Server System State to Azure. Azure Backup also encrypts your backups at the source using a key that only you have access to. Additionally, enhanced security features built into Azure Backup ensure that your critical system state backups remain secure from ransomware, corruptions, and deletions.
  • Free restores With Azure Backup, you can restore System State files from Azure without any egress charges.
https://docs.microsoft.com/en-us/azure/backup/backup-configure-vault]]> CRAY Supercomputers are coming to Azure https://docs.microsoft.com/archive/blogs/ramazancan/cray-supercomputers-are-coming-to-azure Tue, 24 Oct 2017 10:26:40 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1285 climate modeling, precision medicine, energy, manufacturing, and other scientific research. Cray / SGI super computers are not only one of the fastest supercomputers they really also look cool :) I still have my Indy with IRIX running at home https://de.wikipedia.org/wiki/SGI_Indy From the blog article: "Microsoft and Cray are working together to bring customers the right combination of extreme performance, scalability, and elasticity. Customers can get a dedicated Cray XC or CS series supercomputers in Azure to run HPC and AI applications alongside their other cloud workloads directly on the Azure network. The Cray systems easily integrate with Azure Virtual Machines, Azure Data Lake storage, the Microsoft AI platform, and Azure Machine Learning services for rich workflows and collaboration" https://azure.microsoft.com/en-us/blog/cray-supercomputers-are-coming-to-azure/ https://www.cray.com/solutions/supercomputing-as-a-service/cray-in-azure https://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=2310378 https://www.sgi.com]]> Event ID 50 - NTFS Delayed Write Lost https://docs.microsoft.com/archive/blogs/ramazancan/event-id-50-ntfs-delayed-write-lost Tue, 17 Oct 2017 15:31:25 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1226 "an event ID 50 message is logged if a generic error occurs when Windows is trying to write information to the disk. This error occurs when Windows is trying to commit data from the file system Cache Manager (not hardware level cache) to the physical disk. This behavior is part of the memory management of Windows. For example, if a program sends a write request, the write request is cached by Cache Manager and the program is told the write is completed successfully. At a later point in time, Cache Manager tries to lazy write the data to the physical disk. When Cache Manager tries to commit the data to disk, an error occurs writing the data, and the data is flushed from the cache and discarded. Write-back caching improves system performance, but data loss and volume integrity loss can occur as a result of lost delayed-write failures" {Delayed Write Failed} Windows was unable to save all the data for the file \:$I30:$INDEX_ALLOCATION. The data has been lost. This error may be caused by a failure of your computer hardware or network connection. Please try to save this file elsewhere. Or The driver detected a controller error on \Device\Harddisk1\DR1. If you open the eventlog from source system and filter for event 50 you will get the event details In this example host SERVER1 is showing error code 0x80040032 which is IO_LOST_DELAYED_WRITE and status code is key here and this tells us 0xC000026E which equals STATUS_VOLUME_DISMOUNTED. Still not how this could happen but this is a way to better understand what is exactly causing this more details on how decode the error / status can be found here -> https://support.microsoft.com/en-us/help/816004/description-of-the-event-id-50-error-message hope this helps, Ramazan]]> Cumulative Update for October-INACCESSIBLE BOOT DEVICE https://docs.microsoft.com/archive/blogs/ramazancan/cumulative-update-for-october-inaccessible-boot-device Mon, 16 Oct 2017 11:02:32 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1175 https://support.microsoft.com/en-us/help/4041691) was installed same time. Those was released last week and if you configured your internal WSUS to automatically approve you will see this issue -> https://support.microsoft.com/en-us/help/4049094/windows-devices-may-fail-to-boot-after-installing-october-10-version-o After installing this update you probably broke your machine with stop code: INACCESSIBLE BOOT DEVICE let's calm down :) from my experience we could fix all instances with below procedure and do not attempt to run "automatic repair" in that case Let's boot into 'Troubleshooting' and open our lovely good old command prompt   Microsoft Support advises to get rid of the SessionsPending reg key. Let's load the software registry hive: reg load hklm\temp c:\windows\system32\config\software Delete the SessionsPending registry key (if exists): reg delete "HKLM\temp\Microsoft\Windows\CurrentVersion\Component Based Servicing\SessionsPending" /v Exclusive UPDATE: We saw few cases where above reg delete failed, in that case open regedit UI and delete the Exclusive key in sessionspending manually Unload the software registry hive: reg unload HKLM\temp now lets pullout with DISM.exe the list of installed packages dism /image:C:\ /get-packages Checkout the ones which are state "Install Pending" as one of those was causing the issues (highly like the last entries in your list) before we can remove them we need to create a temporary directory first to where we can move the updates to MKDIR C:\temp\packages Now let's (re)move the "pending" package(s) with the DISM command dism /image:c:\ /remove-package /packagename:PACKAGEIDENTITYNAME /scratchdir:c:\temp\packages be focused on the output of the command and if it completes successfully Reboot your VM and it will reboot again and you can run Windows update afterwards as the faulty update was already corrected Official Microsoft KB: https://support.microsoft.com/en-us/help/4049094/windows-devices-may-fail-to-boot-after-installing-october-10-version-o DISM Operating System Package Servicing Command-Line Options https://msdn.microsoft.com/en-us/library/hh825265.aspx ]]> The task XML contains a value which is incorrectly formatted or out of range https://docs.microsoft.com/archive/blogs/ramazancan/the-task-xml-contains-a-value-which-is-incorrectly-formatted-or-out-of-range Fri, 06 Oct 2017 17:11:37 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1126 You can check "C:\Windows\System32\Tasks" and get the above XML. as mine was orphaned I went ahead and removed the XML (=task) file from this folder and I could again access "Task Scheduler" MMC or via Powershell Task Scheduler Schema https://msdn.microsoft.com/en-us/library/windows/desktop/aa383609(v=vs.85).aspx]]> Powershell–How to query HP smart array for disk health check? https://docs.microsoft.com/archive/blogs/ramazancan/powershell-query-hp-smart-array-for-disk-health-check Mon, 04 Sep 2017 13:05:00 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1106 we need some global params a.e. to store in a logfile the results $localhost="SERVERNAME" Get-item ".\log.txt" -ea 0 | Remove-Item -ea 0 $logfile=".\log.txt" $programpath='C:\Program Files\HP\hpssacli\bin' we can build a function to be able to call that later in our script: function CheckSmartArray { Write-Host " " Write-Host "Checking SmartArray on system"$localhost"" -foregroundcolor green C:\Windows\System32\cmd.exe /c "C:\Program Files\HP\hpssacli\bin\hpssacli.exe" controller slot=0 physicaldrive all show } now do we want to check that daily manually? or would be an automated solution like sending a mail in case of a failure? Ok, lets build the SMTP parameters so we can grab the results from HP CLI function SentDiskDrivefailedviaMail { $Logs=Get-Content $logfile $smtpServer = "smtp.contoso.com" $smtpFrom = "datacenterteam@contoso.com" $SMTPPort = "25" $Username = "datacenterteam@contoso.com" $Password = "ContosoPassword1" $smtpTo = "datacenterteam@contoso.com" $messageSubject = "Disk Drive failed at $localhost" [string]$messagebody = "" foreach ($log in $logs ) { $messagebody = $messagebody + $log + "`r`n" } Write-Host "Failed Disk found on Server $servername - starting to sent mail to $smtpTo via $smtpServer ...." -ForegroundColor red $smtp = New-Object Net.Mail.SmtpClient($smtpServer) $smtp.Credentials = New-Object System.Net.NetworkCredential($Username, $Password); $smtp.Send($smtpFrom,$smtpTo,$messagesubject,$messagebody) Write-Host "mail sent completed " -ForegroundColor green Write-Host " " } Now we have 2 functions which can be called and executed, lets finalize the script. first we need to analyze the log for failed disk and if true, build mail object and transfer via SMTP CheckSmartArray | out-file -filepath $logfile -append [array]$logcontent=gc $logfile foreach ($line in $logcontent) { if ($line -match "Failed") { Write-Host " " write-host "failed disk found at $localhost " -foregroundcolor red write-host "detailed logs can be found $logfile " -foregroundcolor red Write-Host " " SentDiskDrivefailedviaMail } } here you can find the HP “Configuring Arrays on HP Smart Array Controllers Reference Guide“ for further details https://h10032.www1.hp.com/ctg/Manual/c02289065.pdf Hope this helps, Ramazan]]> Microsoft Security Intelligence Report Volume 22 is now available https://docs.microsoft.com/archive/blogs/ramazancan/microsoft-security-intelligence-report-volume-22-is-now-available Thu, 31 Aug 2017 15:36:03 GMT https://blogs.technet.microsoft.com/ramazancan/?p=1105 https://www.microsoft.com/en-us/security/intelligence-report at this same topic, also take a look at the new Azure Security Center -> https://blogs.microsoft.com/microsoftsecure/2016/07/21/new-microsoft-azure-security-capabilities-now-available/ [/notranslate]]]>