Backup for 'Search Service 1(099045f6-a648-48f1-9a05-a9371c5d9b88)' failed. System.ServiceModel.FaultException: Management called failed with System.InvalidOperationException: 'Job failed: Have tried to perform backup/restore operation twice on all in-sync members in cluster SP569693711984.1, but none succeeded. Last failure message: Microsoft.Ceres.SearchCore.Seeding.SnapshotTransferException: Could not send chunk ms\%default\gen.000000000000024c.state: Localpath: [0-338> to target BackupDirectoryTarget[directory=\\servername\Backup$\spbr000D\I.2.1,validateTransfers=False]   

 at Microsoft.Ceres.SearchCore.Seeding.SnapshotSender.SendChunks(ISnapshot snapshot, ISeedSource source, ISeedTarget target, SeedStatus status, Func`1 checkAborted, Int32 targetFragIndex)   

 at Microsoft.Ceres.SearchCore.Seeding.SnapshotSender.FirstPhaseTransfer(ISeedSource source, ISeedTarget target, Action`1 updateProgress, Func`1 shouldAbort)   

 at Microsoft.Ceres.SearchCore.Seeding.BackupWorker.BackupWork.DoFirstPhaseWork()'

at  

 at Microsoft.Ceres.SearchCore.IndexController.BackupService.ThrowOnFailure(JobStatus status)   

 at Microsoft.Ceres.SearchCore.IndexController.BackupService.ProgressSecondPhase(String handle)   

 at Microsoft.Ceres.SearchCore.IndexController.IndexControllerManagementAgent.WrapCall[T](Func`2 original)    Server stack trace:    

 at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)   

 at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)   

 at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)   

 at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown

at [0]:    

 at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)   

 at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)   

 at Microsoft.Ceres.SearchCore.Admin.IIndexControllerManagementAgent.ProgressSecondPhase(String handle)   

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.RetryWhileNoEndPoint[T](Func`2 action, SPBackupRestoreInformation args, Guid ssaId, TimeSpan retryTimeout)   

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.RetryWhileNoEndPoint[T](Func`2 action, SPBackupRestoreInformation args, Guid ssaId, TimeSpan retryTimeout)   

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.<>c__DisplayClass13`1.<RetryWhileNoEndPoint>b__12()   

 at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass5.<RunWithElevatedPrivileges>b__3()   

 at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)   

 at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param)   

 at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode)    

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.RetryWhileNoEndPoint[T](Func`2 action, SPBackupRestoreInformation args, Guid ssaId)   

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.WaitPhaseComplete(SPBackupInformation args, Func`2 getProgress, Int32 sleepTime)   

 at Microsoft.Office.Server.Search.Administration.BRIndexComponent.WaitSecondPhaseBackupComplete(SPBackupInformation args)   

 at Microsoft.Office.Server.Search.Administration.TwoPhaseBackupHelper.WaitSecondPhaseBackupCompleteChildren(SPBackupInformation args)   

 at Microsoft.Office.Server.Search.Administration.SearchServiceApplication.OnBackup(SPBackupInformation args)

After trying dozens of things, in the end the solution to this issue is to run the script from a server that has a running Search service instance. The script will not work on servers that do not have search configured.

I'm running SharePoint 2013 RTM.

Off course every one of us has their own bag of tricks scripts to help out. And for the sorry souls out there that don't …. Well …
At least now getting Kerberos to work can get a lot easier if you use the recently released Kerberos configuration tool for SQL Server.

It is a simple tool that connects to a server containing SQL compontents (for example: DB engine, analysis services, and reporting services), and checks AD if the appropriate SPNs are set.
It also lets you generate the scripts to configure them if they are not. Pretty neat huh?

Obviously this is not the entire story when it comes to configuring Kerberos for various scenarios within SharePoint, but it does does hurt a whole lot less if you use tools like these.

For more information on the tool, go to http://blogs.msdn.com/b/analysisservices/archive/2013/05/23/released-kerberos-configuration-manager-for-sql-server.aspx

For more information on configuring Kerberos for SharePoint, go to http://technet.microsoft.com/en-us/library/ee806870.aspx

Also after doing some WCF tracing I found that there was an interesting claim in the SAML token (as part of Service Application authentication; Remember... Everything in the Service app world is claims):
<saml:Attribute AttributeName="isauthenticated" AttributeNamespace="http://sharepoint.microsoft.com/claims/2009/08" a:OriginalIssuer="SecurityTokenService" xmlns:a="http://schemas.xmlsoap.org/ws/2009/09/identity/claims">
<saml:AttributeValue>False</saml:AttributeValue>

Clearly there is something fishy going on here J.

After working on this for most part of a week, even with help of my colleagues at Premier Support, we were not able to nail this one down. Fortunately someone at the customer remembered running into similar issues with a SharePoint 2010 project they did years back.
The solution for that problem was to set the advanced Anonymous Authentication settings of the IIS root node to IUSR:

http://technet.microsoft.com/en-us/library/cc770966(v=WS.10).aspx

Using a default Windows deployment will not require you to do this, but this customer in particular followed a "customize if possible" strategy for their Windows build images, and changing this setting (among many others) to a non-default setting of "Application Pool identity".

I have tried to find out why this breaks SharePoint functionality so severely, but was unable to. (If you know, please post to the comments!).

Fortunately we were able to move forward with the project. I hope I can help at least one of you peeps out there, by posting this!

"We could not find the navigation term set for this site. Navigation integration and friendly URLs will not be configured"

I dove into ULSViewer, but there was nothing whatsoever in the logs that would indicate an issue. The term set was even reachable using the Term Store manager from the HNSC.
During the troubleshooting I noticed the root site collection did not exist, so I created one, but did not give much notice to it, until I was remembered by a fellow MCM that not having a root site collection could be an issue.

So I created a new HNSC and it just worked for that site. All other older HSNCs still did not work. The only way to get them to work was to delete the site collection and recreate it.

I have no details as to what exactly is the reason (there is an article specific to SharePoint 2010 http://support.microsoft.com/kb/2590564 ), but just remember:

Children beware! Always have a root site collection!

(this does not only apply to this issue; there are many other reasons why you would need it!)

It's like Netmon on steroids J…

There is a lot to tell about this tool, but I won't. Hey I'm a SharePoint geek, I have better things to do!

What I can tell you is that cool things have been added like payload rendering, statistics views, and proxy capturing (ala Fiddler) which makes our lives a lot easier.
Just take a look at the juice screenshot where I'm peeking in the HTTPs stream of the Connect site. Lovely!!

So get the beta from Connect and do some testing yourself!!!

Just today there was some discussion on our internal DLs about the noderunner.exe processes of the Enterprise Search service being very hungry for CPU and RAM.

It turns out that there is a memory leak and a lot of debug code involved [REMEMBER: We are still at beta here J].

Jose Vigenor from MS beta support pointed to two options to contain these processes:

1. Use Set-SPEnterpriseSearchService -PerformanceLevel Reduced to reduce the CPU impact the search service has on your test environment.
2. Modify the C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\noderunner.exe.config so that it can only consume X amount of RAM.
   Change the value at <nodeRunnerSettings memoryLimitMegabytes="0" /> to any amount of RAM you like to contain the memory leak.

Good luck!

A day goes by but, nothing happens… What is going on here?
You check your feed but it keeps saying "We're still collection the latest news. You may see more if you try again a little later."
There must be something wrong here. When you crank up the ULS logs to verbose on the Microfeeds and User Profiles categories in the ULS logs:

"Microfeeds", "User Profiles" | Set-SPLogLevel -TraceSeverity Verbose

you will find a strange message stating the User Profile Service Application Proxy is not available:

nasq    Medium     Entering monitored scope (SPMicrofeedFeedCacheService.GetPublishedFeed). Parent FeedCacheService.GetPublishedFeed    e554c59b-9365-2015-6119-380e065adfbb

d22b    High     Failure retrieving application ID for User Profile Application Proxy 'User Profile Service Application 2': Microsoft.Office.Server.UserProfiles.UserProfileApplicationNotAvailableException: UserProfileApplicationNotAvailableException_Logging :: UserProfileApplicationProxy.ApplicationProperties ProfilePropertyCache does not have 5c3e254a-4ab9-4736-a5f1-58d582d1d6d0 at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_ApplicationProperties() at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_AppID()    e554c59b-9365-2015-6119-380e065adfbb

ahv64    High     SPMicrofeedFeedCacheService.GetUserProfile() - UserProfileApplicationProxy not available    e554c59b-9365-2015-6119-380e065adfbb

b4ly    Medium     Leaving Monitored Scope (SPMicrofeedFeedCacheService.GetPublishedFeed). Execution Time=2.064    e554c59b-9365-2015-6119-380e065adfbb

At first it did not hit home for me, but then I remembered similar issues when using powershell to manage the UPA back in SharePoint 2010. This was basically solved by granted (sharing) permissions to the UPA proxy.

The problem in this case is that my MySites are hosted in a different application pool from my UPA service application. Although the MySites application pool account is added, the service app application pool account is not.

Either grant the application pool account hosting the UPA Sharing Permissions through Central Admin:

Or use the following powershell script:

$account = New-SPClaimsPrincipal <domain\user> -IdentityType WindowsSamAccountName

$UPA = Get-SPServiceApplication <ID>

$sec = Get-SPServiceApplicationSecurity $UPA

Grant-SPObjectSecurity $sec2 -Principal $account -Rights "Full Control"

Set-SPServiceApplicationSecurity -Identity $UPA -ObjectSecurity $sec

The magic happens, and voila; the feeds start showing!

So picture this… I'm building my SharePoint farm and creating my Contoso Intranet portal, and one of the first things you do when you have created your publication portal, is add Authenticated Users to the visitors group…

Hey, what's a portal without users right?

But then it hits… WTF is this screen? Where is my people picker? Well.. it's no longer there in lots of places where we add permissions.

So I start typing but No Results. I try all sorts of combinations, but no joy.
I end up searching the web, but I seem to be the only poor soul not being able to crack this nut.

Then it hits me. I'm better than that stinking dialog. I just type 'NT AUTHORITY\AUTHENTICATED USERS' and press Share.

Voila! SPUser added, and after having signed in as another user, it even works.

TAKE THAT YOU EVIL DIALOG!!!

Not sure why they have removed it. As there are some issues with this (even in 2010), I think it might be done because of that, but for most people this will be missed.
Luckily this is only a cosmetic change, as the plumbing is still there.

There are three ways of doing this:

1. Just add the following to the url: /_layouts/closeConnection.aspx?loginasanotheruser=true
2. Start your browser as another user
3. Hack welcome.ascx by adding the code below before the SharePoint:MenuItemTemplate tag with id ID_RequestAccess
   (http://nickgrattan.wordpress.com/2012/07/23/sign-in-as-different-user-and-sharepoint-2013/ )
   
   <SharePoint:MenuItemTemplate runat="server" ID="ID_LoginAsDifferentUser"
   Text="<%$Resources:wss,personalactions_loginasdifferentuser%>"
   Description="<%$Resources:wss,personalactions_loginasdifferentuserdescription%>"
   MenuGroupId="100"
   Sequence="100"
   UseShortId="true"
   />

Obviously modifying control templates is not something we should be doing once the product goes RTM. Lets hope product group decides to put everything back where it belongs J

This means its time for me as well to get acquainted with the product. Given my busy schedule I haven't had much time to do so.

The coming weeks I will spend time writing some posts about the things I see and find in this new product.

Enjoy!