Validating Hybrid Agent error

Question

Hi,

I am setting up the Hybrid Configuration wizard with modern Hybrid and the step is failing when validating a Hybrid Agent and receving the below error.

[Client=UX, fn=SendAsync, Thread=12] START PATCH https://graph.microsoft.com/edu/7f7bc67d-797f-4224-b0f2-a0569dd15e85/applications/b2a75ade-8235-42e4-9aac-556de6deacee {"onPremisesPublishing":{"applicationServerTimeout":"Default","applicationType":"microsoftapp","externalAuthenticationType":"passthru","externalUrl":"https://b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net:443/","internalUrl":"https://internalserver.trs:443/","isOnPremPublishingEnabled":true,"isTranslateHostHeaderEnabled":false,"isTranslateLinksInBodyEnabled":false,"singleSignOnSettings":null,"verifiedCustomDomainCertificatesMetadata":null,"verifiedCustomDomainKeyCredential":null,"verifiedCustomDomainPasswordCredential":null}}
2021. 03.22 15:06:40.066 10333 [Client=UX, fn=SendAsync, Thread=12] FINISH Time=2337.4ms Results=NoContent
2021. 03.22 15:06:40.067 10332 [Client=UX, fn=SendAsync, Thread=12] START GET https://graph.microsoft.com/edu/7f7bc67d-797f-4224-b0f2-a0569dd15e85/applications/b2a75ade-8235-42e4-9aac-556de6deacee/onPremisesPublishing
2021. 03.22 15:06:40.715 10333 [Client=UX, fn=SendAsync, Thread=12] FINISH Time=648.1ms Results=OK {"@odata.context":"https://graph.microsoft.com/edu/$metadata#applications('b2a75ade-8235-42e4-9aac-556de6deacee')/onPremisesPublishing","externalUrl":"https://b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net/","internalUrl":"https://internalserver.trs/","externalAuthenticationType":"passthru","isTranslateHostHeaderEnabled":false,"isTranslateLinksInBodyEnabled":false,"isOnPremPublishingEnabled":true,"isHttpOnlyCookieEnabled":false,"isSecureCookieEnabled":false,"isPersistentCookieEnabled":false,"applicationServerTimeout":"Default","verifiedCustomDomainCertificatesMetadata":null,"verifiedCustomDomainKeyCredential":null,"verifiedCustomDomainPasswordCredential":null,"applicationType":null,"singleSignOnSettings":{"SingleSignOnMode":"none","KerberosSignOnSettings":null}}
2021. 03.22 15:06:40.730 10407 [Client=UX, Thread=12] Updating Local Client Configuration File
2021. 03.22 15:06:40.766 10341 [Client=UX, Page=HybridConnectorInstall, Step=ConfigOrgRoute, Thread=12] FINISH Time=3766.6ms Results=Succeeded
2021. 03.22 15:06:40.768 10340 [Client=UX, Page=HybridConnectorInstall, Thread=12] START
2021. 03.22 15:06:40.770 10293 [Client=UX, Thread=12] Authority=https://login.windows.net/common Resource=https://outlook.office365.com ClientId=a0c73c16-a7e3-4564-9a95-xcfre24fd12
2021. 03.22 15:06:40.790 10098 [Client=UX, Provider=Tenant, Thread=12] Opening Runspace.
2021. 03.22 15:06:42.571 10374 [Client=UX, Page=HybridConnectorInstall, Thread=12] Snoozing for 30 seconds...
2021. 03.22 15:07:12.580 10276 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12] START Test-MigrationServerAvailability -ExchangeRemoteMove: $true -RemoteServer 'b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net' -Credentials (Get-Credential -UserName Domain\ServiceAccount)
2021. 03.22 15:07:56.739 10277 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12]
FINISH Time=44.2s Results=1
TestMigrationServerAvailabilityOutcome {ErrorDetail='Microsoft.Exchange.Migration.MigrationServerConnectionFailedException: The connection to the server 'b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net' could not be completed. ---> Microsoft.Exchange.MailboxReplicationService.MRSRemoteTransientException: The call to 'https://b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net/EWS/mrsproxy.svc' timed out. Error details: The request channel timed out while waiting for a reply after 00:00:09.9996236. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. --> The remote server returned an error: (504) Gateway Timeout. --> The remote server returned an error: (504) Gateway Timeout. ---> Microsoft.Exchange
.MailboxReplicationService.MRSRemotePermanentException: The request channel timed out while waiting for a reply after 00:00:09.9996236. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException: The remote server returned an error: (504) Gateway Timeout. ---> Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException: The remote server returned an error: (504) Gateway Timeout.
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Microsoft.Exchange.MailboxReplicationService.MailboxReplicationServiceFault.ReconstructAndThrow(String serverName, VersionInformation serverVersion)
at Microsoft.Exchange.Connections.Common.WcfClientWithFaultHandling2.<>c__DisplayClass4_0.<CallService>b__0() at Microsoft.Exchange.Net.WcfClientBase1.CallService(Action serviceCall, String context)
at Microsoft.Exchange.Connections.Common.WcfClientWithFaultHandling2.CallService(Action serviceCall, String context) at Microsoft.Exchange.MailboxReplicationService.WcfClientWithVersion2.CallService(Action serviceCall, String context)
at Microsoft.Exchange.Migration.MigrationExchangeProxyRpcClient.CanConnectToMrsProxy(Fqdn serverName, Guid mbxGuid, NetworkCredential credentials, LocalizedException& error)
--- End of inner exception stack trace ---
at Microsoft.Exchange.Migration.MigrationEndpointVerifier.VerifyConnectivity(MigrationEndpointBase endpoint)
at Microsoft.Exchange.Management.Migration.MigrationService.Endpoint.TestMigrationServerAvailability.InternalProcessEndpoint(Boolean fromAutoDiscover)' IsValid=1 Message='The connection to the server 'b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net' could not be completed.' ObjectState='New' Result='Failed' SupportsCutover=0}
2021. 03.22 15:07:56.746 10408 [Client=UX, Page=HybridConnectorInstall, Action=ResetIIS, Thread=12] START
2021. 03.22 15:07:56.760 10408 [Client=UX, Page=HybridConnectorInstall, Action=ResetIIS, Thread=12] Stopping IIS Admin Service on InternalServer.Domain...
2021. 03.22 15:08:02.058 10408 [Client=UX, Page=HybridConnectorInstall, Action=ResetIIS, Thread=12] Starting IIS Admin Service on InternalServer.Domain...
2021. 03.22 15:08:02.352 10408 [Client=UX, Page=HybridConnectorInstall, Action=ResetIIS, Thread=12] FINISH Time=5606.3ms
2021. 03.22 15:08:02.354 10364 [Client=UX, Page=HybridConnectorInstall, Thread=12] Validate Hybrid Agent for Exchange usage | Attempt=2 Elapsed=00:01:21.5916301 Timeout=00:05:00
2021. 03.22 15:08:02.355 10374 [Client=UX, Page=HybridConnectorInstall, Thread=12] Snoozing for 30 seconds...
2021. 03.22 15:08:32.354 10276 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12] START Test-MigrationServerAvailability -ExchangeRemoteMove: $true -RemoteServer 'b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net' -Credentials (Get-Credential -UserName domain\serviceaccount)
2021. 03.22 15:09:13.597 ERROR 10277 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12] FINISH Time=41.2s Results=PowerShell failed to invoke 'Test-MigrationServerAvailability': Processing data from remote server outlook.office365.com failed with the following error message: WS-Management cannot process the request. The operation failed because of an HTTP error. The HTTP error (12152) is: The server returned an invalid or unrecognized response . For more information, see the about_Remote_Troubleshooting Help topic.
2021. 03.22 15:09:13.598 10364 [Client=UX, Page=HybridConnectorInstall, Thread=12] Validate Hybrid Agent for Exchange usage | Attempt=3 Elapsed=00:02:32.8431812 Timeout=00:05:00
2021. 03.22 15:09:13.599 10374 [Client=UX, Page=HybridConnectorInstall, Thread=12] Snoozing for 30 seconds...
2021. 03.22 15:09:43.599 10276 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12] START Test-MigrationServerAvailability -ExchangeRemoteMove: $true -RemoteServer 'b2a75ade-8235-42e4-9aac-556de6deacee.resource.mailboxmigration.his.msappproxy.net' -Credentials (Get-Credential -UserName domain\serviceaccount)
2021. 03.22 15:09:43.608 ERROR 10277 [Client=UX, Session=Tenant, Cmdlet=Test-MigrationServerAvailability, Thread=12] FINISH Time=9.0ms Results=The connection to the Remote PowerShell server was disconnected. Please try again.
2021. 03.22 15:09:43.609 10101 [Client=UX, Provider=Tenant, Thread=12] Disposing Runspace.
2021. 03.22 15:09:43.611 ERROR 10341 [Client=UX, Page=HybridConnectorInstall, Step=TestOrgRoute, Thread=12] FINISH Time=182.8s Results=Workflow Exception: The connection to the Remote PowerShell server was disconnected. Please try again. [Cmdlet=Test-MigrationServerAvailability, DateTime=3/22/2021 11:09:43 AM -04:00, CmdletParameters=System.String[]]
2021. 03.22 15:09:43.612 10390 [Client=UX, Page=HybridConnectorInstall, Thread=12] Test duration 00:03:02.8570323

Answer 1

Hi @TRS Admin ,
According to the error information you provided, there was an error with the remote connection.
1.Have you deployed a firewall? If so, please make sure that the firewall does not block the required traffic and connections from passing through. In addition, antivirus software may also cause this issue. If possible, please turn it off temporarily.

2.Please follow the steps provided in scenario one in the following article to troubleshoot: Troubleshooting Hybrid Migration Endpoints in Classic and Modern Hybrid

3.Please run the following command to check whether EWS virutal direcotry is confiugred correct, and make sure EWS external url could be accessed from external of your organization first. Then you can try to disable MRSProxyEnabled and enable is again, after that run IISReset in CMD with administrator rights. Then please try to run the HCW again.

Get-WebServicesVirtualDirectory -Server InternetFacingServer | fl ExternalUrl,MRSProxyEnabled  

In addition, please make sure that you meet all the conditions of Microsoft Hybrid Agent: Microsoft Hybrid Agent

---

If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.