Reasons to do a full crawl https://technet.microsoft.com/en-us/library/4356bad9-de1d-4e81-b049-17248b4a86c1#Plan_full_crawl

• A Search service application administrator or site collection administrator added or changed a managed property. A full crawl of all affected content sources is required for the new or changed managed property to take effect.

• A Search service application administrator or site collection administrator added or changed a managed property. A full crawl of all affected content sources is required for the new or changed managed property to take effect.
  • If this added/changed managed property impacts a very broad or global scope (such as the Title or Author managed property), a full crawl of the entire content source(s) may be required
  • However, if the added/changed managed property only impacts a smaller subset of content (e.g. a site collection or list/library), then only a full crawl of that subset requires the full crawl
    • In SharePoint 2013/2016 and SharePoint Online, this can be achieved with the “Reindex this site” or “Reindex this list/library” functionality as described here

• Reduced: Total number of threads = number of processors, Max Threads/host = number of processors.
• Partly Reduced: Total number of threads = 16 times the number of processors , Max Threads/host = 8 plus the number of processors. Threads are assigned Below Normal priority.
• Maximum: Total number of threads = 32 times the number of processors, Max Threads/host = 8 plus the number of processors. Threads are assigned Normal priority.

*Side note: Although no longer documented, a single Crawl Component can only start a maximum of 256 threads regardless of how many CPU cores the server possesses.

• First, for a bit of clarity: "host" in this blog post refers to the domain portion of a URL and should not be confused with the underlying servers hosting a web site.
  • For example, assume I have one web application (e.g. https://foo.url.com) that is load balanced across ten SharePoint servers.
  • From the Crawler perspective, I only have one host ("foo.url.com") being crawled in this example.

• 1 web application with 10 million items
• Versus 100 web applications each with 100,000 items

• Keep in mind that this exaggerated scenario is quite feasible with Host Named Site Collections where each site collection is a unique host.

• In this case, the Crawl Component dominates the resources on the server
• And this may prevent [the Generation Controller in] the Admin Component from committing generations across the Index replicas in a timely fashion (which would show in the "Content Processing Activity" chart as Indexing time
• Which in turn, slows down crawls and commonly results in Index replicas getting out of sync.

• With the default “Maximum” setting, the threads started by the Crawl component run with Normal priority (i.e. CPU thread scheduling priority : THREAD_PRIORITY_NORMAL)
  • Have you ever had a Crawl server that was too busy to remote into during a crawl? If the Crawler was running with "Maximum" (meaning, threads with Normal priority), the remote desktop session was most likely not able to preempt the crawl threads and thus not getting sufficient CPU cycles for your remote session
  • With “PartlyReduced” and “Reduced”, the threads are started with Below Normal CPU thread prioritization

• High priority content is allowed to use the default number of threads (*which is the number of physical CPU cores + 8)
• The normal priority content is something less…
• And any know-to-be-slow content sources (think: file share in another geography), set a crawler impact rule of just 1 or 2

• Gigantic Index schema can also have a really big [negative] impact to Search performance (particularly to crawl performance)
• AV can wreck your performance:
  • See my post here: https://blogs.msdn.microsoft.com/sharepoint_strategery/2016/04/27/search-performance-a-case-of-av-slowing-down-crawls/
• Insufficient disk space for the Indexers can lead to silent failures of the replicas…
  • This definitely impacts performance …and the longer this [silently] continues to fail, the worse the problem gets
  • Eventually, the only way out is to add more disk space or reset the index
  • See my post here: https://blogs.msdn.microsoft.com/sharepoint_strategery/2015/07/08/sp2013-using-get-spindexreports-to-troubleshoot-failed-master-merge/
    • For what it’s worth, I have an updated version of the script I reference in that post in our since created “Search Health Reports (SRx)” project which has a LOT of search troubleshooting tests
    • I need to get a recent version uploaded to the TechNet gallery, but I can get you a more recent copy if you are interested
• When assessing the Crawl performance and looking for bottlenecks, I tend to start with the “Crawl Load” report
  • At Ignite 2015, I talked about this chart, too (around the 47m 30s mark)
  • And recently blogged about this point here.

Follow the money crawl flow…

#Options when initializing SRx 
 .\initSRx.ps1 -SSA "Name-of-an-SSA"    # > For handling multiple SSAs in a farm
 .\initSRx.ps1 -Verbose                 # > Enables verbose logging in the console
 .\initSRx.ps1 -RebuildSRx              # > Re-initializes the SRx shell, which
                                        #   rebuilds the $SRxEnv and $xSSA objects
                                        #   (e.g. useful after a topology change)

#Run all tests with detailed output: 
New-SRxReport -RunAllTests -Details 

#Run a specific test (in this case, "OSPingSearchServer") 
New-SRxReport -Test OSPingSearchServer  
    #Hint: After the -Test parameter, use <tab> key to iterate 
    #      through each test names or auto-complete a test name

#New-SRxReport acts as a wrapper for Test-SRx, which invokes a 
#PowerShell script with a corresponding name. For example: 
Test-SRx -Name OSPingSearchServer 
#Invokes: <SRxPath>\lib\tests\core\Test-OSPingSearchServer.ps1  

    # > Test-SRx returns a "standardized" object, which can be 
    #   pipelined to other custom functionality 
    # > New-SRxReport provides standardized formatting of the 
    #   output object from the test 

#To run all tests and generate an array of test result objects: 
Test-SRx -RunAllTests 

#Detailed indexer reports...
Get-SRxIndexReports -DiskReport

#The extended and customized SSA Object ($xSSA)
$xSSA | gm | ? {$_.Name -like "_*"}    #In SRx, custom properties and methods appended 
                                       #to any out-of-the-box objects are named with an
                                       #underscore "_" as their prefix to differentiate

#To get Search Servers details...  
 $xSSA._Servers                        # > An array of all search servers discovered
                                       #   during initialization of SRx, where each
                                       #   server is represented as a custom object
$xSSA._GetServer("-Name-of-Server-")   # > Gets a specific server by name
$xSSA._GetServerEx("-Name-of-Server-") # > Similar to above, but returns an extended
                                       #   server object by fetching the applicable
                                       #   registry keys and system properties (from
                                       #   Get-WMI for various classes)

#-------------------------------- 
#Example of tools available for a specific server... 
 $server = $xSSA._GetServer("-Name-of-Server")  
 $server.canPing()           # > Wrapper: Test-Connection <severname>
 $server.GetProcesses()      # > Under the covers, runs "Get-Process" with a list of 
                             #   applicable Search processes (e.g. noderunner, mssearch)
 
 $server = $xSSA._GetServerEx("-Name-of-Server")  
 $server | gm                #View the other extended properties and methods for this

#Topology Visualization...
 $xSSA._ShowTopologyReport()

#Component fun...
 $xSSA._GetCC()       #Get the list of Crawl Components 
 $xSSA._GetCPC()      #Get the list of Content Processing Components 
 $xSSA._GetAPC()      #Get the list of Analytics Processing Components 
 $xSSA._GetQPC()      #Get the list of Query Processing Components 
 $xSSA._GetIndexer()  #Get the list of Index Components     

#Assuming you have an component named "CrawlComponent2" or "IndexComponent4", run: 
 $xSSA._GetCC(2)      #Gets "CrawlComponent2" from the Active topology
 $xSSA._GetIndexer(4) #Gets "IndexComponent4" from the Active topology

#Each component object also has extended methods and properties... 
 $i4 = $xSSA._GetIndexer(4) 
 $i4._GetProcess()                 # > Wrapper: Get-Process noderunner -computer <name>
 $i4._GetHealthReport()            # > Wrapper: Get-SPEnterpriseSearchStatus -SSA <SSA>
                                   #              -Component IndexComponent4 -HealthReport
 $i4._BuildDiskReportData()        # > Runs underlying processing for an index disk report
 $i4._CellPath                     # > An extended property with the index cell path
 $i4 | gm | ? {$_.Name -like "_*"} # > View the list of properties and methods for this 

#To get more Content Sources details, which aggregates data from the following:
#  - The out-of-the-box content source object
#  - MSSCrawlHistory
#  - MSSCrawlComponentsState
#  - Web Application info [if in the same farm as the SSA]

 $xSSA._GetContentSource()       # > Wrapper: Get-SPEnterpriseSearchCrawlContentSource
 $xSSA._GetContentSourceEx()     # > Returns an extended content source object
                                 #   Hint: To view the extended properties, pipe the 
                                 #         output object to: | SELECT *
 $xSSA._GetContentSourceReport() #Equivalent to: $xSSA._GetContentSourceEx() | SELECT *

#Get a specific content source (either by Content Source ID or by name) 
 $cs1 = $xSSA._GetContentSource("Local SharePoint Sites") 
#And this is the equivalent (assuming "Local SharePoint Sites" has an ID of 1) 
 $cs1 = $xSSA._GetContentSource(1)

#Working with a specific content source 
 $xCS1 = $xSSA._GetContentSourceEx(1)
 $xCS1.StartAddresses.AbsoluteUri
 $xCS1.StartIncrementalCrawl()
 $xCS1._CrawlStatusDetailed

#Note: Fulfilling the extended properties can incur higher overhead for environments 
#      with many content sources. Therefore, when running GetContentSourceEx for all
#      content sources (e.g. by not specifying a parameter value () or by using "*"), 
#      this method implements an in-memory caching mechanism for 20 minutes. 
#       - However, the cache is bypassed when requesting a specific content source 

#Crawl Visualization...
 $xSSA._ShowRecentCrawlVisualization(2,1)  #Displays a visualization of crawl activity
                                           #for the last 2 hours where each character
                                           #represents a 1 minute block of time

• Announcement of SRx - Please read the blog post for install and usage directions
• Summary for each 'Test' - Provides an overview for each test
• Search Topology Report and Assessment - Extension for the SRx that helps assess the Search architecture, topology, and scale out

Semi-unrelated side note: Since the last post, our MSDN blogs got migrated over to WordPress. Although it is a better interface on the authoring front, I haven't [taken the time to] figured out how to replicate some of the formatting in our previous tool set. Being said, I will do my best to keep colors consistent from those previous posts to this and those directly related going forward... (but I admit that you'll see subtle differences if comparing old-to-new posts in this series).

Recap: Starting the crawl (Part 2)

• Seeding the Crawl Queue with the the start addresses 
• Notifying the Crawl Components to join this crawl

Recap: Enumeration/Discovery (Part 3a)

• Asks the content repository for the links to items from a container in that content repository (e.g. what items do you have within this start address? ...within this folder? ...etc)
• And then stores these emitted/discovered [child] links in the “crawl queue” (the MSSCrawlQueue table in the Crawl Store Database)

----------------------------------------------------- The [Gatherer thread in the] crawler logically works the *same regardless of the type of content it’s crawling – web, file shares, SP (there are a few slight *nuances for SP content), etc. The [Gatherer] just processes links (aka, “documents”), and it is up to the [protocol handler] to understand *how to talk to the content or know *what to do with that link (e.g. [to determine whether to] enumerate it using the finder() method or retrieve it using the specificFinder()). Thinking of everything – both containers and items – generically as a “document” and realizing the details to handle this “document” get implemented in the [protocol handler] was actually an aha! moment that helped me generalize this problem and better helped get my mind wrapped around it. -----------------------------------------------------

Enumeration ...a view from ULS

-- The root page of the web application (*notice it starts with "http") 02/27/2015 18:11:12.17    mssearch.exe (0x17F8)    0x2620 SharePoint Server Search    Crawler:Gatherer Plugin    e5e7    VerboseEx CCGatherer::LoadTransactionsFromCrawl, Application=SSA, Project=Portal_Content, crawlID=228, docID=1500, URL=https://taxes.chotchkies.lab, dwTransactionLoaded=1

-- The web application object (*notice it starts with the SharePoint protocol for content , "sts4") 02/27/2015 18:11:12.17    mssearch.exe (0x17F8)    0x2620 SharePoint Server Search    Crawler:Gatherer Plugin    e5e7    VerboseEx CCGatherer::LoadTransactionsFromCrawl, Application=SSA, Project=Portal_Content, crawlID=228, docID=1501, URL=sts4://taxes.chotchkies.lab, dwTransactionLoaded=2

...along with several other specific items marked for re-crawl.

As a side note: The "PerformanceLevel" setting at the Search Service level determines the total number of robot threads that an individual Crawl Component can run. This setting also determines the default crawler impact rule (e.g. the maximum number of robot threads that can be used for any given host domain being crawled). Use the following PowerShell cmdlet for the current PerformanceLevel setting: [Get|Set]-SPEnterpriseSearchService

• You have a correlation ID...
  • In this case f561161d-2242-4fc0-a067-2b513d692bcd
  • Filter the ULS on this correlation ID
• The filtered ULS will have events from both the mssearch.exe and mssdmn.exe processes
• But there are still several events missing - to expose those:
  • For events by the mssdmn.exe, the first effectively reports what task the Gatherer delegated to it
  • In this case:

02/27/2015 18:11:12.19 mssdmn.exe (0x1774) 0x14B0 SharePoint Server Search Connectors:SharePoint dvb0 VerboseEx New url: sts4://taxes.chotchkies.lab/, crawl type = incremental, HighDateTime = 00000000, LowDateTime = 00000001 [sts3handler.cxx:279] f561161d-2242-4fc0-a067-2b513d692bcd

• In ULSViewer, change the filter to include events that either:
  • Match the correlation ID (f561161d-2242-4fc0-a067-2b513d692bcd)
  • OR that match the thread ID of the mssdmn.exe process (e.g. 0x14B0)
• These resulting filtered ULS events will cover the Gathering, Feeding, and the completion of the Search transaction for this particular item:
  • In other words, all the events related to steps 1b, 1c, 2a, 2b, 3 ...and then 6b and 6c
  • 

Shampoo, rinse... repeat (with conditioner)

• Match the correlation ID (e64f034a-6d40-42bd-958e-0bef8e0d2dee)
• OR that match the thread ID of the mssdmn.exe process (e.g. 0x0C34)

Quick reminder: And as a brief recap around this whole scenario, I first performed a full crawl, added a new document (OfficeSpace.docx), increased logging to VerboseEx, and then captured ULS while performing an incremental crawl... so this new document is pretty much the only change to process. 

In coming posts: Continuing to peel back the onion…

1. A high-level view to orchestrating the crawl
2. A deep dive look at starting the crawl
3. Enumeration/Discovery
   1. Concepts
   2. Illustrating through ULS               /* this post */
   3. Simulating with PowerShell

• Enumeration/Discovery: The process where the Crawl Component:
  • Asks for the links to items from the content repository (e.g. “what items do you have within this start address?”)
  • And then stores these emitted/discovered links in the “crawl queue” (the MSSCrawlQueue table in the Crawl Store Database)
• Gathering: The process of the Crawl Component retrieving [think: downloading] the enumerated items (e.g. using links from the crawl queue)
  • Each Crawl Component independently earmarks a small sub-set of items from the MSSCrawlQueue to be gathered/processed; once earmarked, the item is considered part of a “search transaction”
• Feeding: The process of the Crawl Component submitting the gathered items to the Content Processing Component(s)
• Processing: The process of the Content Processing Component converting the documents into a serialized object of Managed Properties (aka: “Mars Document”)
  • The CPC will produce one Mars Document for each item gathered/fed for processing. In other words, the Mars Document is the end product from processing
• Index Submission: The process of submitting the processed document (e.g. the Mars Document) to Index Components where it will be inserted into the index
  • Just before submission, a collection of Mars Documents get batched into a “Content Group”, which gets submitted to the Index as an atomic operation
• Callback Handling: Following Index Submission:
  • A callback message gets returned back to Content Processing Component regarding the status of the Index Submission
  • Then, another callback gets returned from the Content Processing Component back to the Crawl Component regarding the status of processing (which implicitly includes the status from Index Submission as well – In other words, processing cannot report success if Index Submission failed)
  • Finally, the transaction for the item completes when its status is persisted to the MSSCrawlUrl table and the item gets cleared from the MSSCrawlQueue table
     

Which states: These folders may have to be excluded from antivirus scanning when you use file-level antivirus software in SharePoint. If these folders are not excluded, you may see unexpected behavior. For example, you may receive "access denied" error messages when files are uploaded.

The [representative] scenario...

Follow the money crawl flow...

Scenario 1: DocumentRetrieval

Scenario 2: Indexing

   *(apologies for the grainy images, but that is ~170 seconds for each item)

Into the weeds...

"Index files are never directly modified. Instead, when the Index parts on one level become >10% of the level above, they are "merged" into a new part file at that next level above. After the merging has completed, the parts on the lower levels (now merged into the higher level) can then be deleted from disk. During normal operation, the master part typically accounts for >90% of the active documents in the given partition (100% when everything is fully merged into the master part) and the levels below for the other ~10%."

NodeRunnerIndex10-bc8226eb-4f3e      Search      Search Component      ajkma      Unexpected JournalReceiverWorker[SPae590e863ac2]: Generation redo failed with exception.: System.IO.IOException: The process cannot access the file because it is being used by another process.         at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)         at System.IO.File.InternalMove(String sourceFileName, String destFileName, Boolean checkHost)         at Microsoft.Ceres.CoreServices.Storage.LocalDisk.LocalDiskStorage.MoveFile(StoragePath oldPath, StoragePath newPath)         …etc…

• The index has lots of different types of files that can be impacted (e.g. Journal files, mfs files, parts, merges, generation state files, and so on) and the exception can happen with any of them. Being said, when hunting in ULS, search for the exception (i.e. "it is being used by another process") rather than this specific event ID.
• This exception does not always occur (so "no message" does not mean "no interference from AV"). But if it does appear in your ULS, AV is most certainly at play (and Process Monitor can help verify).
• With the redundancy built into Search, a single degraded component does not result in any sort of query outage meaning the failure may "silently" occur. In this specific scenario, the customer noticed the degraded components several hours after this event occurring in ULS.
• When this specific scenario occurred, the thread in this Index component was blocked from moving forward and thus stuck on a particular Generation. On a recycle of the noderunner.exe process (by restarting the HostController service), this secondary replica could then be re-seeded and caught up.

NodeRunnerIndex10-bc8226eb-4f3e      Search      General      ainw9      Medium Remote service invocation: method=GetRecoveryAction(lastPreparedGeneration) Service = {        Implementation type=Microsoft.Ceres.SearchCore.JournalShipper.JournalShipper  Component: SPd9a8b223b429I.2.0.JournalShipper Exposer Name: JournalStreamControl } terminated with exception:       System.Threading.ThreadInterruptedException: Thread was interrupted from a waiting state.             at System.Threading.Monitor.Enter(Object obj)             at Microsoft.Ceres.SearchCore.Journal.Journal.get_LastPreparedGeneration()             at Microsoft.Ceres.SearchCore.JournalShipper.JournalShipper.GetRecoveryAction(UniqueGeneration generationId)

Followed by: NodeRunnerIndex10-bc8226eb-4f3e      Search      Search Platform Services      ajhl2      Monitorable Microsoft.Ceres.CoreServices.Remoting.InvokerService : Service invocation timed out after 179999 ms, method=GetRecoveryAction(lastPreparedGeneration) Service = {              Implementation type=Microsoft.Ceres.SearchCore.JournalShipper.JournalShipper              Component: SPd9a8b223b429I.2.0.JournalShipper              Exposer Name: JournalStreamControl }

The takeaway...

• Verify AV exclusions  /* this is by far the most typical root cause that I've seen */
• Check for failing Master Merge
• Disable TCP Chimney Offload and IP Task Offload  /* rare, but it can make an impact */
• And then dive into the other typical resources constraints (e.g. Disk, Network, CPU)

Getting Started

Generating Test Results

Looking Forward…

Software boundaries and limits for SharePoint 2013
https://technet.microsoft.com/en-us/library/cc262787.aspx#Search

For the longer answer, it makes sense to go back to see what the "Limits and Boundaries" formerly stated (until recently last week) to describe the limits for the Crawl Component...

Before going forward, let's tangent with a brief bit of background on the 64 component limit. With SharePoint 2013, the legacy SharePoint 2010 Search and FAST Search code bases were merged under a single umbrella. However, if you pull back the covers far enough, we can still see two systems at play where each resembles its predecessor and each with its own Admin Component.

The Crawl Component and the parent SSA object are largely managed by the Legacy Admin component that existed in SP2010. This is also referred to as the "Crawl Admin" because much of the object model calls for managing content sources and starting/stopping crawls gets handled by this Legacy Admin component. You can see the Legacy Admin component with $SSA.AdminComponent, but the object to which this refers is different than the Admin Component presented in the Topology web part you see in the Central Admin > Search Administration page.

The parts of Search that evolved from FAST are internally known as the Juno system (e.g. all the components in the Topology other than the Crawl Component). The Admin Component seen in the Central Admin > Search Administration page (or listed in the output of $SSA.ActiveTopology.GetComponents()) primarily manages the state of the Search Index as well as all of the Juno components (again, everything but the Crawl Component).

The 64 component limit is tied to the [Juno] Admin Component's ability to manage/synchronize all the other Juno components in a timely manner. Given that the Crawl Component is not part of the Juno system, it did not make sense to include the Crawl Component when considering this 64 component limit.

Being said, the Limits and Boundaries documentation has been updated accordingly, which now states:

In other words, you can now have a topology with, lets say, 68 total components (e.g. $SSA.ActiveTopology.GetComponents().Count) that would be considered supported as long as there were no more than 64 Juno components (e.g. the number of Admin, CPC, Index, QPC, and APC components). So. for this particular example where the topology has 68 components, at least four of them would need to be Crawl Components (e.g. 68 total components - 4 Crawl Components = 64 Juno Components).

I hope this helps...

$crawlLog = New-Object Microsoft.Office.Server.Search.Administration.CrawlLog $SSA

$dbHashtable = $crawlLog.GetCrawlDatabaseInfo()

$dbHashtable.Keys
    Guid
    ----
    2152167e-9dac-4d45-9724-4ed67f5d9643  #GUID of CrawlStoreOne
    ecd49946-2bde-4ccc-a2b2-c523554bd923  #GUID of CrawlStoreTwo

The Crawl Store is perfect for any ongoing analysis and monitoring, but there are times when troubleshooting where it is incredibly helpful to capture counts directly from the MSSCrawlUrl and MSSCrawlQueue tables in the Crawl Store DB(s). In that same rebalancing Crawl Store DBspost, I demonstrated the following SQL Query (for each crawl store):

SELECT ContentSourceID, HostID, COUNT(*) AS linkCount
FROM [V5_SSA_CrawlStore].[dbo].[MSSCrawlURL] with (nolock)
Group by ContentSourceID, HostID
Order by ContentSourceID, HostID

Which generated a report such as the following:

...but there are times where you just don't have access to SQL and don't have a DBA handy. For this, fear not for we have PowerShell to the rescue.

I recently developed/posted the Get-SPCrawlURLCounts-v2.0.psm1 module here (it's version 2.0 because this is a replacement to the one previously published by Brent Groom and Eric Dixon), which enables us to make the query from PowerShell.

In this first screen shot, we can see the comparable results from the CrawlLog versus the output from the custom module (and just to re-iterate, the CrawlLog is the preferred option for everyday use and creates the least impact) :

As I noted before (and as the script will prompt you the first time it runs), be mindful when running this module as it can have impact to the performance of your SSA. In other words, this script is intended for troubleshooting scenarios ...do not regularly run or schedule this query.

In fact, it's time to bring back my standard disclaimer: 
ALL information in this blog is provided "AS IS" with no warranties and confers no rights. This blog does not represent the thoughts, intentions, plans or strategies of my employer. All content is solely my opinion and provided with a best effort to be based in reality. All examples, code samples, demonstrations, or anything resembling a “how-to” are provided "AS IS" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. Inappropriate comments will be deleted at the authors discretion. And yes, the spelling of strategery was intentional.

OK, being said... In the screen shot below, we see further examples of how to use this new custom module:

The benefit to crafting this into a module (versus a script) is that the cmdlet can be invoked as any of the following:

Get-SPCrawlURLCounts -ByContentSource
Get-SPCrawlURLCounts -ByHostId

#"detailed" is equivalent to: Get-SPCrawlURLCounts -ByContentSource -ByHostId
Get-SPCrawlURLCounts –Detailed

We can even pipeline an $SSA object directly to it, such as:

$SSA | Get-SPCrawlURLCounts –Detailed

Also, we can easily substitute in the MSSCrawlQueue table instead of the MSSCrawlUrl on the fly and get counts from the queue instead. For example:

Get-SPCrawlURLCounts –Detailed -inQueue

But the real reason for writing this version 2.0 was to have the end product be an array of objects that could be further manipulated (and there is still the option to export the CSV). For example:

Get-SPCrawlURLCounts -Detailed |  WHERE {$_.HostName -match "chotchkies.lab"}

And exporting to CSV is easily accomplished by tacking on the flag “-toCSV” (optionally, you can specify a custom path with the “–Path c:\custom\” parameter ...otherwise the CSV file is created in the local path), such as:

Get-SPCrawlURLCounts -Detailed -toCSV

--------------------

To run this:

1. Import the module, such as:
Import-Module ".\Get-SPCrawlURLCounts-v2.0.psm1"

2. Then run it as the examples above:
$SSA | Get-SPCrawlURLCounts –Detailed -toCSV

(And, for what it’s worth: I plan to add more into this crawl diagnostics module later…)

I hope this helps...