Exchange Hybrid - HCW8078 The request channel timed out attempting to send

Matthew Ridley 171 Reputation points
2022-07-19T07:22:36.667+00:00

We have Exchange 2019 servers hosting mailboxes and want to move to Exchange Hybrid. There is currently no external access to our Exchange servers for OWA.
I have built an Exchange 2019 edge server for mail flow to and from EOP. I have configured the subscription although mail is not yet configured to flow through the edge server (only once Hybrid is configured hopefully).
I have run the HCW on one of the existing Exchange servers.
After clicking on Update to start the HCW Configure I receive a warning when it is trying to configure the MRS Proxy settings.

The full error is below:

2022.07.18 14:48:55.264 WARNING 10026 [Client=UX, Page=Configuring, fn=RunWorkflow, Thread=15]
HCW8078 Migration Endpoint could not be created.
Microsoft.Exchange.Migration.MigrationServerConnectionFailedException
The connection to the server 'mail.domain.com' could not be completed.
Microsoft.Exchange.MailboxReplicationService.MRSRemoteTransientException
The call to 'https://mail.domain.com/EWS/mrsproxy.svc' timed out. Error details: The request channel timed out attempting to send after 00:00:07.9622867. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. --> The HTTP request to 'https://mail.domain.com/EWS/mrsproxy.svc' has exceeded the allotted timeout of 00:00:07.9620000. The time allotted to this operation may have been a portion of a longer timeout. --> The operation has timed out
Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException
The request channel timed out attempting to send after 00:00:07.9622867. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout.
Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException
The HTTP request to 'https://mail.domain.com/EWS/mrsproxy.svc' has exceeded the allotted timeout of 00:00:07.9620000. The time allotted to this operation may have been a portion of a longer timeout.
Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException
The operation has timed out

From the Exchange server I am running the HCW on I can manually navigate to https://mail.domain.com/EWS/mrsproxy.svc which resolves to the Internal IP and I am prompted for a username and password.

I created a specific user to use in the HCW for the on-premises account for migration, but also tried entering my admin credentials

I created a new third party certificate which is installed on each Exchange server (and applied to IIS, SMTP, POP and IMAP) and edge server and includes the SAN of mail.domain.com

The EWS Virtual directory on the Exchange servers is configured with:
Internal URL - https://mail.domain.com/EWS/Exchange.asmx
External URL - https://mail.domain.com/EWS/Exchange.asmx
Authentication - Integrated Windows Authentication

Any help would be appreciated.

Microsoft Exchange Online Management
Microsoft Exchange Online Management
Microsoft Exchange Online: A Microsoft email and calendaring hosted service.Management: The act or process of organizing, handling, directing or controlling something.
4,386 questions
Exchange Server Management
Exchange Server Management
Exchange Server: A family of Microsoft client/server messaging and collaboration software.Management: The act or process of organizing, handling, directing or controlling something.
7,503 questions
Microsoft Exchange Hybrid Management
Microsoft Exchange Hybrid Management
Microsoft Exchange: Microsoft messaging and collaboration software.Hybrid Management: Organizing, handling, directing or controlling hybrid deployments.
1,999 questions
{count} votes

Accepted answer
  1. KyleXu-MSFT 26,246 Reputation points
    2022-07-20T01:47:56.957+00:00

    @Matthew Ridley

    There is currently no external access to our Exchange servers for OWA.

    How do you configure this for your Exchange server?

    From the Exchange server I am running the HCW on I can manually navigate to https://mail.domain.com/EWS/mrsproxy.svc which resolves to the Internal IP and I am prompted for a username and password.

    Could you provide more detailed information such as screenshots about this one?

    Mark sure your Exchange server has published to Internet with those ports: Hybrid deployment protocols, ports, and endpoints

    You also need to make sure those Exchange online port could access from your Exchange server: Exchange Online

    Then try to disable and enable MRSProxyEnabled for EWS:

    Get-WebServicesVirtualDirectory | Set-WebServicesVirtualDirectory -MRSProxyEnabled $false  
      
    Get-WebServicesVirtualDirectory | Set-WebServicesVirtualDirectory -MRSProxyEnabled $true  
    

    If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.



2 additional answers

Sort by: Most helpful
  1. Matthew Ridley 171 Reputation points
    2022-07-19T13:15:31.253+00:00

    I Have been doing as bit more research into this and manually ran the command that the HCW is running for the MRS Proxy:

    Test-MigrationServerAvailability -ExchangeRemoteMove:$true -RemoteServer "mail.domain.com" -Credentials $credentials

    The result is as follows:

    RunspaceId : be68af2e-72f5-458a-b7e1-5d7b44b21b99
    Result : Failed
    Message : The connection to the server 'mail.domain.com' could not be completed.
    ConnectionSettings :
    SupportsCutover : False
    ErrorDetail : Microsoft.Exchange.Migration.MigrationServerConnectionFailedException: The connection to the
    server 'mail.domain.com' could not be completed. --->
    Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The Mailbox Replication
    Service was unable to connect to the remote server using the credentials provided
    . Please check
    the credentials and try again. The call to 'https://mail.domain.com/EWS/mrsproxy.svc'
    failed. Error details: The HTTP request is unauthorized with client authentication scheme
    'Negotiate'. The authentication header received from the server was 'Negotiate,NTLM'. --> The
    remote server returned an error: (401) Unauthorized.. --> The HTTP request is unauthorized with
    client authentication scheme 'Negotiate'. The authentication header received from the server was
    'Negotiate,NTLM'
    . --> The remote server returned an error: (401) Unauthorized. --->
    Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The call to
    'https://mail.domain.com/EWS/mrsproxy.svc' failed. Error details: The HTTP request is
    unauthorized with client authentication scheme 'Negotiate'. The authentication header received
    from the server was 'Negotiate,NTLM'. --> The remote server returned an error: (401)
    Unauthorized.. ---> Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The
    HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication
    header received from the server was 'Negotiate,NTLM'. --->
    Microsoft.Exchange.MailboxReplicationService.RemotePermanentException: The remote server returned
    an error: (401) Unauthorized.
    --- End of inner exception stack trace ---
    --- End of inner exception stack trace ---
    --- End of inner exception stack trace ---
    at Microsoft.Exchange.MailboxReplicationService.MailboxReplicationServiceFault.<>c__DisplayClas
    s97_0.<ReconstructAndThrow>b__0()
    at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(Action operation)
    at Microsoft.Exchange.MailboxReplicationService.MailboxReplicationServiceFault.ReconstructAndTh
    row(String serverName, VersionInformation serverVersion)
    at Microsoft.Exchange.MailboxReplicationService.WcfClientWithFaultHandling2.<>c__DisplayClass7 _0.<CallService>b__0() at Microsoft.Exchange.Net.WcfClientBase1.CallService(Action serviceCall, String context)
    at
    Microsoft.Exchange.MailboxReplicationService.WcfClientWithFaultHandling`2.CallService(Action
    serviceCall, String context)
    at Microsoft.Exchange.Migration.MigrationExchangeProxyRpcClient.CanConnectToMrsProxy(Fqdn
    serverName, Guid mbxGuid, NetworkCredential credentials, LocalizedException& error)
    --- End of inner exception stack trace ---
    at Microsoft.Exchange.Migration.DataAccessLayer.ExchangeRemoteMoveEndpoint.VerifyConnectivity()
    at Microsoft.Exchange.Management.Migration.MigrationService.Endpoint.TestMigrationServerAvailab
    ility.InternalProcessEndpoint(Boolean fromAutoDiscover)
    IsValid : True
    Identity :
    ObjectState : New

    I have run the following command:

    Get-WebServicesVirtualDirectory | fl authentication, url

    CertificateAuthentication :
    InternalAuthenticationMethods : {Ntlm, WindowsIntegrated, WSSecurity, OAuth}
    ExternalAuthenticationMethods : {Ntlm, WindowsIntegrated, WSSecurity, OAuth}
    LiveIdNegotiateAuthentication :
    WSSecurityAuthentication : True
    LiveIdBasicAuthentication : False
    BasicAuthentication : False
    DigestAuthentication : False
    WindowsAuthentication : True
    OAuthAuthentication : True
    AdfsAuthentication : False
    InternalNLBBypassUrl :
    InternalUrl : https://mail.domain.com/ews/exchange.asmx
    ExternalUrl : https://mail.domain.com/ews/exchange.asmx

    In IIS, EWS authentication is set to Anonymous and Windows Authentication Enabled.
    The providers for Windows Authentication are Negotiate at the top and NTLM underneath.

    Is it something to do the the AuthenticationMethods I have configured?

    0 comments No comments

  2. Amit Singh 4,956 Reputation points
    2022-07-20T11:19:32.693+00:00

    Did you try to enable TLS 1.2? Then you can use the MS Hybrid wizard tool to connect endpoints and perform a test migration.

    Check this detailed blog for help - https://www.alitajran.com/hcw8078-migration-endpoint-could-not-be-created/