"Service Bus Message Broker" service takes 5 minutes to start
Symptom:
You have Service Bus 1.0/1.1 installed in a Service Bus Farm. You observe a five minutes delay when you restart Service Bus Farm Services (by executing "SB-StopFarm" and "SB-StartFarm")
In Microsoft-ServiceBus Operational Windows log several Informational messages like below are shown:
Windows Fabric Runtime
create failed (Retries left 6) Exception System.TimeoutException: Operation
timed out. ---> System.Runtime.InteropServices.COMException: Exception from
HRESULT: 0x80071BFF at
System.Fabric.Interop.NativeRuntime.FabricEndCreateRuntime(IFabricAsyncOperationContext
context) at
System.Fabric.FabricRuntime.NativeFabricRuntimeFactory.InitializeFabricRuntimeEndWrapper(FabricRuntime
runtime, AsyncCallOutAdapter adapter) at
System.Fabric.Interop.Utility.FinishNativeAsyncInvoke[TResult](String
functionTag, Func`2 endFunc, TaskCompletionSource`1 source, AsyncCallOutAdapter
adapter, Boolean expectCompletedSynchronously)
--- End
of inner exception stack trace --- at System.Fabric.FabricRuntime.Create(Action fabricExitCallback) at Microsoft.Cloud.ServiceBus.MessageContainerHost.Ring.Join()
Cause:
The delay is caused by Windows Fabric and the behavior is by design. When you take down the entire ring (Stop-SBFarm) and bring it back online (Start-SBFarm) it waits for a global lease duration of 5 minutes before it reforms the ring. This is to prevent split brain.
So, if you wait for 5+ minutes after "Stop-SBFarm" to let go off the global lease duration, and then run "Start-SBFarm", it will complete normally within a few seconds.
Whenever posible, try to use "Stop-SBHost" and "Start-SBHost" instead of stopping the whole Service Bus Farm.
Hope you find it interesting!!