To begin, I'll create a database named FragmentsDB.

use master
go
SET STATISTICS XML OFF
SET STATISTICS IO OFF
SET NOCOUNT ON
go
IF DB_ID('FragmentsDB') IS NOT NULL
BEGIN 
     ALTER DATABASE FragmentsDB 
     SET SINGLE_USER WITH ROLLBACK IMMEDIATE  

     DROP DATABASE FragmentsDB
END
GO
CREATE DATABASE FragmentsDB
GO
ALTER DATABASE FragmentsDB
SET RECOVERY SIMPLE
GO

Next, create a table named NumbersTable and load some data into it.  The manner in which I am loading data will ensure that once I'm done the non-clustered index that I've created on the table will be noticeably fragmented.  The way I am doing this is to load numbers into the table, skipping every 3rd number.  Then, I'll load back into the table some of the data that I skipped before.  In doing this, the data that I load in the second insert will need to be placed in the appropriate pages (remember, an index is ordered and we have to put the rows into the page where the sorted values fit logically), causing pages to split and logical fragmentation to occur. Note that I am not making the pages completely full, as is the default fillfactor. This will ensure that for the purposes of this test that the index isn't 100% fragmented.

USE FragmentsDB
GO
CREATE TABLE NumbersTable
( 
     NumberValue BIGINT NOT NULL, 
     BiggerNumber BIGINT NOT NULL, 
     CharacterColumn CHAR(50)
)
GO

ALTER TABLE NumbersTable
ADD CONSTRAINT PK_NumbersTable PRIMARY KEY CLUSTERED (NumberValue)
GO  

CREATE INDEX idx_NumbersTable ON NumbersTable(BiggerNumber)
WITH(FILLFACTOR=75)
GO

INSERT INTO NumbersTable
( 
     NumberValue, BiggerNumber, CharacterColumn
)
SELECT 
     NumberValue, 
     NumberValue + 5000000, 
     LEFT(REPLICATE((CAST(NumberValue as VARCHAR(50))),50),50)
FROM
( 
     SELECT 
           NumberValue = row_number() over(order by newid() asc) 
     FROM master..spt_values a 
     CROSS APPLY master..spt_values b 
     WHERE a.type = 'P' AND a.number <= 500 AND a.number > 0 AND 
     b.type = 'P' AND b.number <= 500 AND b.number > 0
) a
WHERE NumberValue % 3 != 0
GO  

INSERT INTO NumbersTable
( 
     NumberValue, BiggerNumber, CharacterColumn
)
SELECT TOP(30000) 
     NumberValue,  
     NumberValue + 5000000, 
     LEFT(REPLICATE((CAST(NumberValue as VARCHAR(50))),50),50)
FROM
( 
     SELECT 
           NumberValue = row_number() over(order by newid() asc) 
     FROM master..spt_values a 
     CROSS APPLY master..spt_values b 
     WHERE a.type = 'P' AND a.number <= 500 AND a.number > 0 AND 
     b.type = 'P' AND b.number <= 500 AND b.number > 0
) a
WHERE NumberValue % 3 = 0
GO

To look at the amount of logical fragmentation in the leaf layer of the index, use the DMV sys.dm_db_index_physical_stats.  We can see that the fragmentation is right at 53.32%.

SELECT avg_fragmentation_in_percent, index_level, record_count, page_count
FROM sys.dm_db_index_physical_stats(DB_ID(), OBJECT_ID('NumbersTable'), 2, NULL, 'Detailed')

So, what does a logical fragmentation of 53.32% really mean?  In the simplest of terms, it means that just over half of the pages in the leaf level of the index are not physically located in the order in which the keys in the index indicate that they should be.  In an index that had 0 fragmentation, the leaf level pages would be listed right next to each other logically and physically.  It would look something similar to this:

Note: for simplicity purposes I am leaving out the fact that the rows on a given page aren't stored in physical order. An offset array is used at the page level to track the ordering of the rows on a given page.

However, when logical fragmentation comes into play, the tree can look like something totally different.  The physical order (the allocation order) doesn't match up to the logical order of the keys (the sequential numbers in this case).  So, unless SQL Server is running an allocation order scan then it is very difficult to perform a scan operation in an efficient manner. 

SQL Server 2012 exposes a new DMV, sys.dm_db_database_page_allocations, that we can use to see the page linkages for a particular entity.  (The output from this DMV is very similar to the output to the DBCC IND command.  It is just more easily consumable).

In this case, I will try to calculate the fragmentation in the index based upon checking the allocation order linkages in the DMV vs. logical page linkages that I calculate using the new LEAD windowing function in 2012.  This new function allows you to look at a value that came just before the current record in a resultset.

SELECT 
     Fragmentation = ((COUNT(*)*1.00 - SUM(Case When NextPhysicalPageID = NextLogicalPageID then 1 else 0 end))/COUNT(*)*1.00)*100, 
     PageCount = COUNT(*) 
FROM

( 
     SELECT allocated_page_page_id, NextLogicalPageID = next_page_page_id, 
     NextPhysicalPageID = LEAD(allocated_page_page_id) over(partition by object_id, index_id order by allocated_page_page_id) 
     FROM sys.dm_db_database_page_allocations(DB_ID(), OBJECT_ID('NumbersTable'), 2, NULL, 'DETAILED') 
     WHERE page_type_desc = 'INDEX_PAGE' and page_level = 0
) a

Looking at the results from the query above, the fragmentation output that is the same as produced from the sys.dm_db_index_physical_stats DMV.  There are a few outlying cases I've seen so far where the output between these 2 didn't match up exactly (usually off < 1 %).  I need to dig into the source code to determine why. 

At this point, you may be thinking…"Big Deal.  I can already find the fragmentation of an index.  Why would I need a new way to do it?"  The answer is – you don't.  You should still use the regular method of identifying logical fragmentation.  The aggregated query above is a sanity check to make sure the two methods match up.  The real reason for this blog post is below.  In this query I show the detail around my fragmented index.  The logic is the same as the aggregated query above, but is this time showing the detail associated with the page linkages.

SELECT 
     *, PageLinkMatch = Case When NextPhysicalPageID = NextLogicalPageID then 1 else 0 end
FROM
( 
     SELECT allocated_page_page_id, NextLogicalPageID = next_page_page_id, 
     NextPhysicalPageID = LEAD(allocated_page_page_id) over(partition by object_id, index_id order by allocated_page_page_id) 
     FROM sys.dm_db_database_page_allocations(DB_ID(), OBJECT_ID('NumbersTable'), 2, NULL, 'DETAILED') 
     WHERE page_type_desc = 'INDEX_PAGE' and page_level = 0
) a

I am checking to see if the NextLogicalPage (as is returned by the DMV) matches up to the NextPhysicalPage that I am calculating.  If the pages match, then that specific linkage is not fragmented.  If they do not match, then the link IS fragmented.  Let's take a look at a specific example from my output (your mileage will vary with the page numbers, etc).  In the case of Page 997 the next physical page matches to the next logical page.  The page before 997 pointed logically to page 3019.  This was due to the page splits caused by the insert statements created above.  However, the next physical page (the order of the page allocation) is page 997.  This "logical fragmentation" is the cause of SQL Server having to break up IOs into multiple operations for scanning purposes. 

In fact, we can further investigate the page using DBCC PAGE.  By supplying the page number, we can see that the previous page and next page in the page header matches up to the output that we are seeing in the DMV.

DBCC TRACEON(3604, -1)
DBCC PAGE(FragmentsDB, 1, 997, 2)

I hope this helps in visualizing what logical fragmentation looks at from the SQL Server point of view. From a performance point of view, fragmentation isn't really that big of a deal for operations that are doing singleton style lookups – a few pages returned isn't going to be hindered by fragemented index. Scan operations will suffer because the engine will need to break up the IO for the scan into multiple IOs when fragmentation is present.

Before I get to the demo, let's compare the two methods.  Here is a snippet taken from this link:  https://technet.microsoft.com/en-us/library/ms189858.aspx

"You can remedy index fragmentation by reorganizing or rebuilding an index. For partitioned indexes built on a partition scheme, you can use either of these methods on a complete index or a single partition of an index. Rebuilding an index drops and re-creates the index. This removes fragmentation, reclaims disk space by compacting the pages based on the specified or existing fill factor setting, and reorders the index rows in contiguous pages. When ALL is specified, all indexes on the table are dropped and rebuilt in a single transaction. Reorganizing an index uses minimal system resources. It defragments the leaf level of clustered and nonclustered indexes on tables and views by physically reordering the leaf-level pages to match the logical, left to right, order of the leaf nodes. Reorganizing also compacts the index pages. Compaction is based on the existing fill factor value."

OK, now that we are all on the same page I'll start the demo.  First, I'll create a database to use for the example.  If it exists, drop it and then create it.

USE [master]
SET NOCOUNT ON
GO
IF DB_ID('RebuildReorg') IS NOT NULL
BEGIN
ALTER DATABASE RebuildReorg
SET SINGLE_USER WITH ROLLBACK IMMEDIATE  

DROP DATABASE RebuildReorg
END
GO

CREATE DATABASE RebuildReorg ON PRIMARY
( NAME = N'RebuildReorg_Data', FILENAME = N'C:\SQL\RebuildReorg_Data.mdf' ,
SIZE = 2048MB , MAXSIZE = UNLIMITED, FILEGROWTH = 50MB )
LOG ON
( NAME = N'RebuildReorg_Log', FILENAME = N'C:\SQL\RebuildReorg_log.LDF' ,
SIZE = 2048MB , MAXSIZE = 5GB, FILEGROWTH = 200MB)
GO

Next I want to make sure I am in the full recovery model.  If I happened to have the DB in the Simple recovery mode then the log would be truncated when a checkpoint occurs.  I want the log records that I generate to stay in the log so I can see how much log is generated for a rebuild vs a reorganization.

ALTER DATABASE RebuildReorg
SET RECOVERY FULL
GO

Next I will create a table named Sales and pump some data into it.  I've added a clustered index on a uniqueidentifier column on the table.  This should leave the index in a very fragmented state once the data has been entered.  This is useful because I want to simulate a "worst case" scenario so that the rebuild and reorganize routines have a lot of work to do.

USE RebuildReorg
GO

IF OBJECT_ID('dbo.Sales') IS NOT NULL
DROP TABLE dbo.Sales
GO

 CREATE TABLE dbo.Sales
(
[SalesOrderID] [int] NOT NULL,
[SalesOrderDetailID] [int] NOT NULL,
[CarrierTrackingNumber] [nvarchar](25) NULL,
[OrderQty] [smallint] NOT NULL,
[ProductID] [int] NOT NULL,
[SpecialOfferID] [int] NOT NULL,
[UnitPrice] [money] NOT NULL,
[UnitPriceDiscount] [money] NOT NULL,
[LineTotal] AS (isnull(([UnitPrice]*((1.0)-[UnitPriceDiscount]))*[OrderQty],(0.0))),
[ModifiedDate] [datetime] NOT NULL,
GUIDCol UNIQUEIDENTIFIER NOT NULL DEFAULT(NEWID())
) ON [PRIMARY]
GO

CREATE CLUSTERED INDEX cixSales ON dbo.Sales (GUIDCol)
GO

INSERT INTO dbo.Sales
(
SalesOrderID,
SalesOrderDetailID,
CarrierTrackingNumber,
OrderQty,
ProductID,
SpecialOfferID,
UnitPrice,
UnitPriceDiscount,
ModifiedDate
)
SELECT
SalesOrderID,
SalesOrderDetailID,
CarrierTrackingNumber,
OrderQty,
ProductID,
SpecialOfferID,
UnitPrice,
UnitPriceDiscount,
ModifiedDate
FROM AdventureWorks2012.Sales.SalesOrderDetail
GO 20

I need to take a backup to start the recovery chain.  I just created this database and while it is in the full recovery mode, the database will be treated as though it is in the simple recovery mode since I have not yet taken a full backup.  In other words, the log will be truncated when a checkpoint occurs right now.  As soon as I take a full backup, the log will only be truncated when a log backup occurs.  In the full backup and log backups I am taking below, I am using the "NUL" backup device.  I'm just using this because I don't really want to write the backup out to disk.  I don't plan on using the backup in later operations.  You really never want to use this in a production environment unless you are testing how long it takes to read the data from the data files for the backup or another special case which I won't go into here.

BACKUP DATABASE RebuildReorg
TO DISK = 'NUL'
GO

BACKUP LOG RebuildReorg
TO DISK = 'NUL'
GO

Now rebuild the clustered index on the table.

ALTER INDEX cixSales ON dbo.Sales
REBUILD

I can use the system function sys.fn_dblog to find out how many log records were generated by the rebuild statement.  Remember, I took a log backup before I performed the rebuild, so I was working with an "empty" transaction log when the rebuild occurred.

SELECT LogRecordsGenerated = COUNT(*)
FROM sys.fn_dblog(NULL, NULL)
WHERE AllocUnitName = 'dbo.Sales.cixSales'

So, it looks like rebuilding the index resulted in 35516 transaction log records.  Cool.  What kind of log generation will I be looking at if I would have performed a reorganize instead?  To find out, I'll need to recreate the above scenario again. 

USE RebuildReorg
GO

IF OBJECT_ID('dbo.Sales') IS NOT NULL
DROP TABLE dbo.Sales
GO

 CREATE TABLE dbo.Sales
(
[SalesOrderID] [int] NOT NULL,
[SalesOrderDetailID] [int] NOT NULL,
[CarrierTrackingNumber] [nvarchar](25) NULL,
[OrderQty] [smallint] NOT NULL,
[ProductID] [int] NOT NULL,
[SpecialOfferID] [int] NOT NULL,
[UnitPrice] [money] NOT NULL,
[UnitPriceDiscount] [money] NOT NULL,
[LineTotal] AS (isnull(([UnitPrice]*((1.0)-[UnitPriceDiscount]))*[OrderQty],(0.0))),
[ModifiedDate] [datetime] NOT NULL,
GUIDCol UNIQUEIDENTIFIER NOT NULL DEFAULT(NEWID())
) ON [PRIMARY]
GO

CREATE CLUSTERED INDEX cixSales ON dbo.Sales (GUIDCol)
GO

INSERT INTO dbo.Sales
(
SalesOrderID,
SalesOrderDetailID,
CarrierTrackingNumber,
OrderQty,
ProductID,
SpecialOfferID,
UnitPrice,
UnitPriceDiscount,
ModifiedDate
)
SELECT
SalesOrderID,
SalesOrderDetailID,
CarrierTrackingNumber,
OrderQty,
ProductID,
SpecialOfferID,
UnitPrice,
UnitPriceDiscount,
ModifiedDate
FROM AdventureWorks2012.Sales.SalesOrderDetail
GO 20

Make sure to backup the log again so that we can compare the two operations.

BACKUP LOG RebuildReorg
TO DISK = 'NUL'
GO

Now perform the reorganize.

ALTER INDEX cixSales ON dbo.Sales
REORGANIZE

And run the query against the transaction log to see how much log was generated by the reorganize operation.

SELECT LogRecordsGenerated = COUNT(*)
FROM sys.fn_dblog(NULL, NULL)
WHERE AllocUnitName = 'dbo.Sales.cixSales'

As you can see, from a log generation perspective the reorganize statement created a SIGNIFICANTLY (35516 vs 2.7M) higher number of transaction log records as compared to the rebuild.  This is by design.  For large, heavily fragmented indexes the reorganization operation is much less efficient than the index rebuild operation.  This is important to know because if you're using a technology that relies on reading from the transaction log, such as mirroring, transactional replication/CDC then performing a reorg is going to generate a lot more activity that needs to be sorted through.  Of course, a reorg does have its benefits, such as being a progressive operation.  A reorg can be stopped while it is occurring and you won't lose the work it has already completed, as compared to a rebuild which is an atomic operation.  Like everything in technology, there are benefits and drawbacks to both operations. 

Hope this helps!

Tim

Let's take a look at how significant the memory consumption savings can be when using this option.  First, let's clear out the plan cache (remember to never do this on a production system).

DBCC FREEPROCCACHE

Now, let's look at what is returned for our cached plans that reference the table SalesOrderHeader (which is in the AdventureWorks database).  This should return NULL since we just cleared out the plan cache.

SELECT PlanSizeInMB = SUM(size_in_bytes)/1024.0/1024.0
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
WHERE text LIKE '%SalesOrderHeader%' AND
text NOT LIKE '%dm_exec_cached_plans%'

I also want to make sure the "optimize for ad hoc workloads" option is turned off.

EXECUTE sp_configure 'optimize for ad hoc workloads', 0
RECONFIGURE

Now, let's run a load that is similar to an application that builds SQL strings on the fly and executes the statements in a non-parameterized fashion.  This is a contrived example, but illustrates what can happen.  I'm simply looping through the Sales.SalesOrderHeader table a record at a time and issuing a SELECT statement for each record encountered with an explicit value for the predicate.  This will create an entry in the plan cache for every single SELECT statement.

DECLARE @SalesOrderID VARCHAR(20)
DECLARE @SQL NVARCHAR(MAX)

DECLARE @TableVar TABLE (SalesOrderID INT)  

DECLARE CursorExample CURSOR FOR
SELECT SalesOrderID
FROM Sales.SalesOrderHeader

OPEN CursorExample
FETCH NEXT FROM CursorExample INTO @SalesOrderID

WHILE @@FETCH_STATUS = 0
BEGIN

SET @SQL = 'SELECT SalesOrderID
FROM Sales.SalesOrderHeader h
JOIN Sales.SalesPerson p on p.BusinessEntityID = h.SalesPersonID
WHERE SalesOrderID = ' + @SalesOrderID

INSERT INTO @TableVar(SalesOrderID)
EXECUTE (@SQL)

FETCH NEXT FROM CursorExample INTO @SalesOrderID
END

CLOSE CursorExample
DEALLOCATE CursorExample

When the "optimize for ad hoc workloads" option is not on, a full plan is created for every single SELECT statement.  In this case, around 1259 MB of plan cache was used to store these simple plans.  That is just over 1.2GB of plan cache used! 

SELECT PlanSizeInMB = SUM(size_in_bytes)/1024.0/1024.0
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
WHERE text LIKE '%SalesOrderHeader%' AND
text NOT LIKE '%dm_exec_cached_plans%'

So, what happens if I enable the 'optimize for ad hoc workloads' setting and rerun the test?  First, I'll want to enable the setting and then clear the procedure cache again.

EXECUTE sp_configure 'optimize for ad hoc workloads', 1
RECONFIGURE

DBCC FREEPROCCACHE

Rerun the same cursor code from earlier in this email.  After you do, run the following statement to see the size of the plan cache:

SELECT PlanSizeInMB = SUM(size_in_bytes)/1024.0/1024.0
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
WHERE text LIKE '%SalesOrderHeader%' AND
text NOT LIKE '%dm_exec_cached_plans%'

Using the 'optimize for ad hoc workloads' option results in a plan cache that is right at 10MB (compared to a bloated cache of 1.2GB).  If these plans are never used again, that is an incredible amount of memory you're saving for the buffer pool (and for other plans).  The good thing is, there is really no reason to not have this setting on – the downside is negligible. 

HTH!

Tim

Filtered indexes are well-suited for scenarios where you frequently query a well-defined subset of data.  This often includes querying ranges of specific values such as dates or monetary ranges, along with scenarios where you frequently have to filter out NULL values – such as Sparse columns or minimally updated columns.  Another scenario where filtered indexes work really nicely is when you're dealing with heterogeneous data, such as frequent queries on highly correlated columns in a table for certain subsets of data.   In this case, because of the frequency that you deal with the relationship between columns for a given data set, it may make sense to create a specific filtered index to meet your query needs rather than using a non-clustered index that contains the same number of rows as the base table.

Because the filtered index only needs to be maintained when there are rows inserted/updated/deleted that meet the index and filter definition, the overhead of maintaining the filtered index can be significantly less.  There are also many cases where the Query Optimizer is more likely to choose a lower-cost plan based on the presence of the filtered index.

Let's take a look at a filtered index in action.  First, I'll create a new table named Sales.NewSales, which is a copy of the AdventureWorks table Sales.SalesOrderDetail.

USE AdventureWorks

GO

IF OBJECT_ID('Sales.NewSales') IS NOT NULL
DROP TABLE Sales.NewSales
GO

SELECT *
INTO Sales.NewSales
FROM Sales.SalesOrderDetail
GO

Create a new column in the Sales.NewsSales table named NewUnitPrice.

ALTER TABLE Sales.NewSales
ADD NewUnitPrice MONEY NULL
GO

Update every 10th record in Sales.NewSales to have a non-NULL value.

UPDATE Sales.NewSales
SET NewUnitPrice =
CASE
    WHEN SalesOrderDetailID % 10 = 1
    THEN UnitPrice
    ELSE NULL
END

Create a filtered index on the NewUnitPrice column to include any value that is NOT NULL.

CREATE NONCLUSTERED INDEX NewSalesIDX
ON Sales.NewSales(NewUnitPrice)
WHERE(NewUnitPrice IS NOT NULL)

Create an index on the UnitPrice field for comparison purposes.

CREATE NONCLUSTERED INDEX UnitPriceIDX
ON Sales.NewSales(UnitPrice)

Now, let's compare the number of pages between the two indexes.

SELECT
    i.name, d.index_type_desc,
    d.index_depth, d.avg_fragmentation_in_percent,
    d.page_count,
    IndexSizeMB = (page_count * 8192)/1024.0/1024.0
FROM sys.dm_db_index_physical_stats
    ( DB_ID(), 
      OBJECT_ID('Sales.NewSales'), 
      NULL,NULL, NULL
    ) d
JOIN sys.indexes i ON
d.index_id = i.index_id AND
d.object_id = i.object_id
WHERE
i.index_id > 1

From the screenshot below, you can see that the filtered index has many fewer pages – which also means that it takes less disk space.

So, how does performance compare between the two indexes?  Running the following query you can see that the filtered index out performs the regular non-clustered index by a significant margin.

SELECT NewUnitPrice
FROM Sales.NewSales
WHERE NewUnitPrice = 49.99

SELECT UnitPrice
FROM Sales.NewSales
WHERE UnitPrice = 49.99

For more information on the downside of filtered indexes, check out my post here:  https://blogs.msdn.com/b/timchapman/archive/2012/08/27/the-drawback-of-using-filtered-indexes.aspx

HTH,

Tim

Let's look at an example of what I am talking about.  First, I'll create a filtered index to play around with.

CREATE INDEX FIDX_SalesOrderDetail_ProductID
ON Sales.SalesOrderDetail (ProductID)
WHERE ProductID = 870

Run a quick query against the SalesOrderDetail table for ProductID 870. As you may expect, this should result in a SEEK of our newly created filtered index.

Great – that is exactly what we want. The new filtered index was used in the previous query. One thing to notice in the previous query is that I used an explicit value 870 in the predicate. SQL Server didn't have to make any guesses as to what value was being sought. However, what happens if I use a local variable?

DECLARE @ProductID INT
SET @ProductID = 870  

SELECT ProductID
FROM Sales.SalesOrderDetail
WHERE ProductID = @ProductID

 
In this case, SQL Server did NOT use the filtered index – even though the query was searching for the value 870. The reason for this is the same reason you want to try to avoid using local variables in predicates. The query optimizer did not what value was held in the local variable at compile time, so it had to make some assumptions based on the statistics it contained for the column being searched. Because SQL Server didn't know what value was in the variable, a filtered index cannot be considered.

In many cases, a way to get around the local variable problem is to recompile the statement:

DECLARE @ProductID INT
SET @ProductID = 870  

SELECT ProductID
FROM Sales.SalesOrderDetail
WHERE ProductID = @ProductID
OPTION(RECOMPILE)

Unfortunately, in the case of the filtered index it does not work:
Note:  The fix for this behavior was fixed in SQL Server 2008 R2 SP1.  Now an OPTION(RECOMPILE) will the actual local variable (or parameter) value.

OK, at this point we know that using local variables presents a challenge. So, what about using parameterized SQL statements? Here I'll create a parameterized SQL statement using sp_executesql.

DECLARE @SQL NVARCHAR(MAX)
SET @SQL = N'SELECT ProductID
FROM Sales.SalesOrderDetail
WHERE ProductID = @ProductID'
EXECUTE sp_executesql @SQL, N'@ProductID INT', @ProductID = 870

Still no luck. Even those this statement was compiled using the ProductID of 870, the filtered index still isn't used. But, why not? Well, let's think back to what a filtered index actually is. A filtered index is a NC structure that contains values based on an explicit predicate. This means that the filtered index is only useful if the query is searching explicitly for the query defined in the filter definition for the index. If SQL Server were to create an execution plan that used that filtered index, it would ONLY be useful for reuse if subsequent statements passed in that specific criteria. Since SQL Server has no idea if that is going to happen, the filtered index isn't considered for reuse.

But what if you REALLY want SQL Server to use that specific filtered index? It's easy enough to try to force it using an index hint.

DECLARE @SQL NVARCHAR(MAX)
SET @SQL = N'SELECT ProductID
FROM Sales.SalesOrderDetail WITH(INDEX=FIDX_SAlesOrderDetail_ProductID)
WHERE ProductID = @ProductID '
EXECUTE sp_executesql @SQL, N'@ProductID INT', @ProductID = 870

In this case, I actually get an error message telling me that my usage of the hint is not valid. This is due to the fact that the index I am forcing is filtered.

While incredibly useful, you can see that filtered indexes do have some drawbacks when it comes to how they can be applied. In most cases, the best way to take advantage of the filtered indexes you have is through the use of dynamic string execution. In this case I was lazy and just modified how I was using sp_executesql. The EXECUTE statement would have worked just as well.

DECLARE @SQL NVARCHAR(MAX), @ProductID INT
SET @ProductID = 870
SET @SQL = N'SELECT ProductID
FROM Sales.SalesOrderDetail 
WHERE ProductID = ' + CAST(@ProductID AS VARCHAR(10))
EXECUTE sp_executesql @SQL

And, sure enough, we are able to take advantage of the filtered index because SQL Server is given an explicit value at compile time.

Now for some cleanup:

DROP INDEX FIDX_SalesOrderDetail_ProductID
ON Sales.SalesOrderDetail

HTH,
Tim     

Note:  While you CAN dump the contents from DBCC MEMORYSTATUS into a temp table on the server using INSERT...EXEC, an important aspect of the data isn't inserted into the temp table. 

The script accepts 3 different parameters:

$FileLocation - if you have the DBCC MEMORYSTATUS output in a file, specify the location here.  Otherwise the script will grab the output from DBCC MEMORYSTATUS from the instance you connect to.
$SQLServer - this is the SQL instance to connect and load the output from DBCC MEMORYSTATUS.
$DBName - this is the name of the database you want the DBCC MEMORYSTATUS output stored.

I've attached the file instead of pasting the output here as I couldn't get the formatting how I wanted it to be.  If I get it corrected I'll make the change a later.

Once you've executed the script, you'll have output very similar to the following, which will allow you to run any reports as needed.

I hope you enjoy, and I'd love to hear comments as to how to improve the script!

Thanks,

Tim

DBCCMemoryStatus.ps1