Unable to start SharePoint Search Host Controller
I recently encountered an issue with the SharePoint Search Host Controller service being unable to start in a customer's SharePoint 2013 farm, so am posting this blog to hopefully save you some troubleshooting headache. The following issue is most likely to occur in a "least privilege" configuration, especially one in which the server permissions get severely restricted by OS administrators.
In my case, the first sign of problems occurred when the SharePoint Search Administration page showed a red "X" for the health of the Content Processing and Analytics Processing components on a server. These components depend on the SharePoint Search Host Controller service, which is a Windows Service. This service generally resides in the "<drive>:\Program Files\Microsoft Office Servers\15.0\15.0\Search\Host Controller" directory. When I opened the Services MMC and tried to start the service, it failed with "access denied". I opened ProcMon (https://technet.microsoft.com/en-us/sysinternals), filtered for a result of "ACCESS DENIED", and attempted to start the service again. Same result, but ProcMon captured two access failures to the directory described above. My next step was to navigate to the "HostController" folder that hosts the service executable and compare its permissions to those of a server that was working properly. On the "good" server, the WSS_ADMIN_WPG and WSS_WPG groups had access to this folder, but on the "bad" server, those groups were missing. Now I had two questions to answer: First, why were the permissions different? Second, how should I fix the permissions?
For the "how do I fix it?" question, the answer is simple: Run "psconfig -cmd secureresources" on the "bad" server. This will reset the folder/registry/etc. permissions to the out-of-the-box configuration values.
At this point, I thought I had the issue resolved. However, a few days later, two more servers experienced the same issue. "Coincidentally", those two servers happened to be the ones that I was using as my "good" servers in the troubleshooting performed above. So what caused the issue? During troubleshooting, I had navigated down the directory structure. When I hit the "15.0" folder, I got an "access denied" message with the option to grant myself permissions (since I was logged on as an admin). I simply hit "grant permission" and continued traversing the directory structure. However, what happened when I hit "grant permission" was that the permissions got inherited by the sub-directories, including the "HostController" folder. This caused the WSS_ADMIN_WPG and WSS_WPG groups to be removed. The issue only appeared when attempting to start the service, so I did not see an immediate negative impact following the permission inheritance on the servers that were already running the service. Hope that helps!