Odd issue around configuring a dedicated WFE for SharePoint crawl
We had an interesting issue recently where the SharePoint services would not start up and the configuration cache would not get populated. The event logs and ULS were filled with errors about Invalid Uri.
Error details:
Exception information:
Exception type: UriFormatException
Exception message: Invalid URI: The format of the URI could not be determined.
And the call stack of the error looks something like this,
[UriFormatException: Invalid URI: The format of the URI could not be determined.]
System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind) +7204799
Microsoft.SharePoint.Administration.SPAutoSerializingObject.DeserializeBasicObject(XmlElement xmlValue, Boolean isMerge, Boolean bResolveMissingTypes) +1848
Microsoft.SharePoint.Administration.SPAutoSerializingObject.DeserializeBasicObject(XmlElement xmlValue, Boolean isMerge, Boolean bResolveMissingTypes) +3124
Microsoft.SharePoint.Administration.SPAutoSerializingObject.DeserializeBasicObject(XmlElement xmlValue, Boolean isMerge, Boolean bResolveMissingTypes) +2926
Microsoft.SharePoint.Administration.SPAutoSerializingObject.SetStateXml(XmlNodeList childNodes, Boolean isMerge, Boolean bResolveMissingTypes) +685
Microsoft.SharePoint.Administration.SPAutoSerializingObject.SetStateXml(XmlDocument value, Boolean bResolveMissingTypes) +85
Microsoft.SharePoint.Administration.SPPersistedObject.Initialize(ISPPersistedStoreProvider persistedStoreProvider, Guid id, Guid parentId, String name, SPObjectStatus status, Int64 version, XmlDocument state) +27172224
Microsoft.SharePoint.Administration.SPConfigurationDatabase.GetObject(Guid id, Guid parentId, Guid type, String name, SPObjectStatus status, Byte[] versionBuffer, String xml) +347
Microsoft.SharePoint.Administration.SPConfigurationDatabase.FetchObject(Guid id) +1396
Microsoft.SharePoint.Administration.SPConfigurationDatabase.GetObject(Guid id, Boolean checkInMemoryCache, Boolean checkFileSystemCache) +406
Microsoft.SharePoint.Administration.SPConfigurationDatabase.Microsoft.SharePoint.Administration.ISPPersistedStoreProvider.GetObject(Guid id) +59
Microsoft.SharePoint.Administration.SPPersistedObjectCollection`1.get_Item(Guid objId) +50
Microsoft.SharePoint.Administration.<GetEnumeratorImpl>d__0.MoveNext() +239
Microsoft.SharePoint.Administration.Enumerator`1.MoveNext() +23
Microsoft.SharePoint.Administration.SPWebApplication.AcquireAlternateUrlCache(SPFarm farm) +402
Microsoft.SharePoint.Administration.SPWebApplication.Lookup(SPFarm farm, Uri requestUri, Boolean fallbackToHttpContext, SPAlternateUrl& alternateUrl, SPSiteLookupInfo& hostHeaderSiteInfo, Boolean& lookupRequiredContext) +237
Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.EnsureInitialize(HttpRequest request) +646
Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.BeginRequestHandler(Object oSender, EventArgs ea) +164
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +171
We initially isolated it to a corruption in the URL within the XML for the Web Application properties. But when we looked through the XML, we did not find anything corrupt in the URLs. This was strange!
Finally capturing a dump of the process and investigating further, I found that the property that was used in the above callstack was m_SiteDataServers.
The value of this property was set to just the servername without the protocol information. i.e. it was set to "wfe1" instead of "https://wfe1" OR "https://wfe2".
NOTE: The script to configure the dedicated frontend for crawling uses the command "$webApp.SiteDataServers.Add($zoneUrl, $listOfUri);". This command will not accept just the servername, but seems like we have an entry point somewhere to send it through.