Agents reporting to Gateway Management servers grayed out.

Question

Hello All,

Good day :)

Here's the situation .

I have 2 domains.
Domain A has 4 MS and Domain B has 2 GW servers

We pushed around 5000 agents using a deployment tool. However, during deployment we can use only one MS and hence all servers were pushed to MS's in phases. In the end there were high number of servers reporting to one server and i ran a load balancing script to distribute the agents between the servers.

The issue is all the servers of domain B reporting to GW servers are grayed out.

Used the below article to set the Failover

https://kevinholman.com/2018/08/06/assigning-gateways-and-agents-to-management-servers-using-powershell/

PS C:\> $Gateway = Get-SCOMManagementServer | where {$.Name -eq "GW1.B.com"}
PS C:\> $Gateway.GetPrimaryManagementServer().DisplayName
MS1.A.com
PS C:\> $Gateway.GetFailoverManagementServers().DisplayName
MS3.A.com
MS2.A.com
MS4.A.com
PS C:\> $Gateway = Get-SCOMManagementServer | where {$.Name -eq "GW2.B.com"}
PS C:\> $Gateway.GetPrimaryManagementServer().DisplayName
MS4.A.com
PS C:\> $Gateway.GetFailoverManagementServers().DisplayName
MS3.A.com
MS2.A.com
MS1.A.com

Any advise on how i can get my agents reporting to GW healthy ?

Answer 1

@py , From your description, I know the agents connect to GW2 are all greyed out. And the GW2 server is still healthy. If there's any misunderstanding, please let us know.

For our issue, we can go to the affected agent side and check HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Agent Management Groups\<SCOM management group>\Parent Health Services\, Make sure Authentication name and network name are the FQDN of GW2 server.

Our issue is resolved. Here is a summary for it:

Cause:

=========

The servers in GW2 domain have been assigned a Management server rather than the GW server when the AD assignment is switched in a script

Resolution:

============

Uncheck the AD assignment and change the Management server to Gateway name.

Hope it can help.

---

If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

You should start by having a look at their Operations Manager log and check what specific issue they are facing.
And of course double check the usual suspects : certificates, dns resolution, firewall port opened...

Answer 3

>All agents report to GW2 in grey state and GW2 is in health state.

Troubleshoot steps

1. select one of the agents reporting to GW2
2. restart microsoft monitoring agent service
3. review event log for error
4. telnet GW2 with port 5723
5. using following powershell to check agent primary management server
   $agent=get-scomagent |where-object{$_.displayname -eq 'XXX.B.com'}
   get-scomparentmanagementserver -agent $agent

The result should be GW2.B.COM

Roger

Answer 4

Moreover, restart Microsoft Monitoring Agent Service such that the agent uses new setting to connect management server.

Roger

Answer 5

I seem to have gotten to the root of the issue. Thanks much for your assistance @System Center guy @Crystal-MSFT @CyrilAzoulay

When i ran the scripts to load balance the server, it interestingly has switched on the the AD assignment and the servers in the GW 2 domain have been assigned a MS rather than the GW server. Now i'm in the process of unchecking the AD assignment and change the MS name in MM proprieties in control panel