question

BrendaRojas-7259 avatar image
0 Votes"
BrendaRojas-7259 asked SadiqhAhmed-MSFT commented

Azure Site Recovery Critical Notification: vCenter health is in Critical state

Greetings,
We have a VMware to azure site recovery VMs and we are receiving this alerts today relating to vCenter:
Message: Discovery of the vCenter server X.X.X.X:443 failed with the error System.ServiceModel.CommunicationException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason) at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Vim25Api.VimPortType.ContinueRetrievePropertiesEx(ManagedObjectReference _this, String token) at VMware.VSphere.Management.Operations.VSphereUtilities.RetrieveProperties(VimPortType service, ServiceContent serviceContent, PropertyFilterSpec pfSpec) at VMware.VSphere.Management.Operations.VSphereUtilities.GetEntityViews(VimPortType service, ServiceContent serviceContent, String moRefType, String[] moRefProperies) at VMware.VSphere.Management.Operations.VirtualMachineOperations.GetVmViewsByProps() at VMware.VSphere.Management.VSphereManagementClient.GetInfrastructureViewLite(String[] options, OsType osType) at InMageDiscovery.Program.Main(String[] args) .

Possible Causes: Refer to the vSphere Power CLI documentation for possible causes.

Recommendation: Resolve the error: System.ServiceModel.CommunicationException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason) at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Vim25Api.VimPortType.ContinueRetrievePropertiesEx(ManagedObjectReference _this, String token) at VMware.VSphere.Management.Operations.VSphereUtilities.RetrieveProperties(VimPortType service, ServiceContent serviceContent, PropertyFilterSpec pfSpec) at VMware.VSphere.Management.Operations.VSphereUtilities.GetEntityViews(VimPortType service, ServiceContent serviceContent, String moRefType, String[] moRefProperies) at VMware.VSphere.Management.Operations.VirtualMachineOperations.GetVmViewsByProps() at VMware.VSphere.Management.VSphereManagementClient.GetInfrastructureViewLite(String[] options, OsType osType) at InMageDiscovery.Program.Main(String[] args) . Retry the operation once the error is resolved.

The replication is not affected, we receive successful heartbeats from process server and vCenter. We recently upgraded the process server agent to 9.48. Is this a VMware issue or a ASR issue?


azure-site-recoveryazure-vmware-solution
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SadiqhAhmed-MSFT avatar image
1 Vote"
SadiqhAhmed-MSFT answered

Hello @BrendaRojas-7259 Thank you for your patience and I apologize for the delayed response!

I see, you mentioned that Azure Site Recovery notifications are triggered for vCenter health in critical state. Usually these alerts are triggered when there is communication failure. For example: If port 443 is disabled or network connectivity is blocked.

You also mentioned that you recently upgraded the process server agent to 9.48. Ideally upgrades do not cause any failures. But at times, a network blip may cause these kind of issues.

Good thing is, the replication is not affected and you receive successful heartbeats from process server and vCenter.

I don't find this to be a problem. But if this is causing any replication failure then that need to be investigated further to understand root cause for this issue.

Please feel free to reply if you need further assistance in this matter!

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @SadiqhAhmed-MSFT, thank you for your response. I was curious as to what this alert meant since we received them random and the replication never stopped. We upgraded the Mobility agent on the VMs and have not received the alert since them.

So as to understand what this meant is that it was a network issue, correct?

0 Votes 0 ·

Could not think of any thing other than a restart of ASR services might have worked. If you are interested in investigating this issue further, you can contact Azure site recovery technical support team by raising a support request. The ticket enables you to work closely with the support engineers and get a quick resolution to your issue.

Here is the link - https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request to create support case.

0 Votes 0 ·