Crawl Error: The item could not be indexed successfully because the item failed in the indexing subsystem. (Content group cancelled by generation controller)

Question

--- Sharepoint 2013. Vers.15.0.5285.1000 (On-premise) ----

Hi!

Recently all of my content sources, got the following error. Leaving around 80% of the items in the index, error-prone and unsearchable.

"""

The item could not be indexed successfully because the item failed in the indexing subsystem. ( The item could not be indexed successfully because the item failed in the indexing subsystem.; Content group cancelled by generation controller; Aborting insert of item in Link Database because it was not inserted to the Search Index.; ; SearchID = 6E4BF12E-19E3-4C8B-9441-F6D8E66B3786 )

"""

Searching around the internet I've found examples of many similar, but not identical errors. Specifically 'Content group cancelled by generation controller' is where the error differs from most documented instances.

I've tried the following:

1. IIS Reset
2. Index Reset
3. SP Cache Reset on each machine in the farm
4. Restarting each machine in the farm
5. Recreating the entirely new SSA, on a entirely different machine in the farm, and crawling from scratch
6. SharePoint Configuration Wizard on the crawling machine
7. Change the crawling target to another machine in the farm using the hosts file. All the machines involved have a large surplus of space on all their disks.

Have anyone experienced something similar? And if so, was it solvable

Answer 1

Hi @August V.S ,

Great to know that and thanks for sharing the update here.

By the way, since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others.". and according to the scenario introduced here: Answering your own questions on Microsoft Q&A, I would make a brief summary of this thread:

[Crawl Error: The item could not be indexed successfully because the item failed in the indexing subsystem. (Content group cancelled by generation controller)]

Issue Symptom:
Recently all of my content sources, got the following error. Leaving around 80% of the items in the index, error-prone and unsearchable.

"""

The item could not be indexed successfully because the item failed in the indexing subsystem. ( The item could not be indexed successfully because the item failed in the indexing subsystem.; Content group cancelled by generation controller; Aborting insert of item in Link Database because it was not inserted to the Search Index.; ; SearchID = 6E4BF12E-19E3-4C8B-9441-F6D8E66B3786 )

"""

Current status:
The solution was throwing a lot of CPU and Memory at the crawling machine for the time being, so i was able to smoothly create a new SSA with the index place on the D:\ drive instead. This higher specced machine, quickly got through a full crawl without any error, after which i could remove the old one.

You could click the "Accept Answer" button for this summary to close this thread, and this can make it easier for other community member's to see the useful information when reading this thread. Thanks for your understanding!

Answer 2

Hi @August V.S ,

For your question, we recommend that you first use a search diagnostic health tool to confirm that your search is configured correctly, it has some good index storage tests to confirm that you have enough storage space on your index server. Calculating search index storage is tricky because index size fluctuates during crawls. Note that just because it looks like you have enough free space doesn't mean it does during a crawl.

This applies to SharePoint 2013, 2016 and O365.

Thanks,
Yanli Jiang

===========================================

If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 3

Hi @YanliJiangMSFT-7045

After dismissing it for several weeks, i finally have to conclude that space seems to have had an impact. Even though over 60% of the disk space was available, it does seem to somewhat have been the culprit.

It seems to have been this 'lack' of disk-space, combined with a underspecced crawl server. The index was erroneously placed in the C:\ drive of the crawl machine. So when the crawl-server started doing some heavy work, it worked itself into the uncapped Swap-file, seeming grabbing the disk-space needed for the index.
The underspecced nature of the crawl machine, meant that recreating a new SSA was also quite a hassle and i got several TimeJob timeout swhen attempting to do so. So the SSA's i finally managed to create, looked fine, but didn't seem to perform very well.

In the end, the solution was throwing a lot of CPU and Memory at the crawling machine for the time being, so i was able to smoothly create a new SSA with the index place on the D:\ drive instead. This higher specced machine, quickly got through a full crawl without any error, after which i could remove the old one.

Thank you very much for the assistance.
// August