​The fix is very important becuase the bug is causing data loss when rebuilding indexes with some options enabled and deadlock happened during the rebuilding. The full documentation in the link below. 

Another script that I wanted to share for so much time but didn't have the commitment to be persisted with my postsJ. Using the built-in SQL Server maintenance plans for index maintenance works well for small databases without HA solutions that are affected by the amount of transaction logs generated. When it comes to mid-size or large databases, the process of reorganizing or rebuilding the indexes has to be optimized. In this case, there has to be a script used to reorganize/rebuild only the indexes needed to be maintained. There are many factors that influence the decision like

• Number of pages in the index: Small indexes will not cause problem because SQL will read them one time and they stay in memory
• Fragmentation percentage: small fragmentation makes more sense to have them reorganized. Big fragmentation will be rebuilt much faster than reorganization
• Whether or not will use online index rebuilding
• Whether or not sorting in the tempdb (recommended if the tempdb on a faster disk)
• Number of range scans on the index. If we have an index with mostly singleton searches then there's no need to defrag it.

The script is taken care of all these. At the beginning of the script, there's a part where I set the default values for these configurable variables.

--set values

--------Configuration-----------

SET @useOnlineIndexing = 1;

SET @SORT_IN_TEMPDB = 'OFF';

--change it to ON to sort in tempdb

SET @minPagestoConsider = 1000;

--8MB indexes

SET @MDOP = 8;

--Max Degree of Parallelism. Change it to 1 to reduce the number of CPUs involved in rebuilding the index

SET @MinNumberOfScansToConsider = 1000;

--how many range scans to consider

SET @WhenPageLockOFFandReBuilding = 'EnablePageLock';

--either 'EnablePageLock' or 'SetMDOP1'

/* When building index online with MDOP>1 & Page locks disabled, would cause more fragmentation. deciding the best action either enable Page locks then disable it

or set the MDOP=1 */

--------------------------------

One important option is when rebuilding index online while MDOP>1 and Enable Page Lock disabled, you will notice that fragmentation is increased not decreased. That's a known behavior documented on this KB article. The script give you two options, either to Enable Page locking before the rebuilding and disable it after rebuilding or make MDOP=1.

The script works against one database. If you would like to run it against more than one database, use the SQLCMD variable named $(DatabaseName) to target multiple databases using SQLCMD calls in a batch file or using a SQL Job.

The script can be downloaded from here

This blog post is cross posted on

http://blogs.msdn.com/b/mosharaf/

http://www.msharaf.com/blog

When troubleshooting customer's performance issue that's happening at the moment, we have many tools out of the box. The good old SQL traces that are not needed for the majority of the cases and if it's needed then my first option would ne XE if we are on 2012. Also there are many DMV that we can use and these are the building blocks that SQL Server Management Studio Activity Monitor is built on. However to get the best response and the result sets that you are only interested in you need to use your own script.

I used to write an ad-hoc script every time I'm troubleshooting something based on the situation. There are some great scripts online that covers what I need but I wanted to have my own implementation as simple and function-based so one can use it with filters easily. If you are looking for more comprehensive script, I'd refer to Adam Machanic's whoisactive stored procedure. I don't have much experience using it but he's the first one who used the idea of printing the TSQL code as XML –as far as I know- so will be easier to view in SSMS by clicking on it. I'm using the same idea as well.

Eventually I wrote the function dbo.GetCurrentActivity. I'm using it for few months now and if you are a customer whom I promised that it will be uploaded to my blog soon, please accept my apology and find the details below J

The function will show only the running/runnable or suspended requests. If we have a session that does nothing then it will not be seen by this function.

The function make it easy for these scenarios

    -Waiting on any waiting type as it shows the waiting summary column

    -Blocking chain as it shows the complete chain so you can get the root blocker

    -It shows the original TSQL when having FETCH API_CURSOR operation

The output of the function is detailed in the table below.

To setup the function, execute the script file from the location below. The function will be installed by default on the master database. Then from any other database you can call

SELECT * FROM master.dbo.GetCurrentActivity(1); --with lightweight parameter =1, quick execution

SELECT * FROM master.dbo.GetCurrentActivity(0); --with lightweight parameter =0, slower execution but more details.

The complete script can be found here

This blog post is cross posted on

http://blogs.msdn.com/b/mosharaf/

http://www.msharaf.com/blog

Enjoy your performance troubleshooting.

~Mohamed

This is not correct and if you have the patience to go through the rest of the post you will get the proof. That goes for both profiler trace and extended events as well. However network sniffing is possible which can be prevented by SSL encryption to the data in the wire but that's outside the scope of this post.

USE master;

GO

Create Database TestEncryption01

GO

USE TestEncryption01;

GO

--Create the seniorEmployees table

IF EXISTS (SELECT name from sys.tables where name='SeniorEmployees')

DROP TABLE dbo.SeniorEmployees

GO

CREATE TABLE dbo.SeniorEmployees

    (

    ID int NOT NULL,

    EmployeeID varbinary(MAX) NOT NULL,

    EmployeeSalary decimal(8, 2) NOT NULL,

    EmployeePosition varbinary(MAX) NOT NULL

    ) ON [PRIMARY]

     TEXTIMAGE_ON [PRIMARY]

GO

ALTER TABLE dbo.SeniorEmployees ADD CONSTRAINT

    PK_SeniorEmployees PRIMARY KEY CLUSTERED

    (

    ID

    ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]

--CREATE database master key

CREATE MASTER KEY ENCRYPTION BY PASSWORD ='Str0ngPassw0rd!'

--Create certificate first

CREATE CERTIFICATE EmployeesCert WITH SUBJECT='Certificate for encrypting symmetric keys';

--check the certificate

SELECT * FROM sys.certificates

--now create the symmetric key

CREATE SYMMETRIC KEY EmployeesKey WITH ALGORITHM = AES_256

ENCRYPTION BY CERTIFICATE EmployeesCert;

--Key meta data

SELECT * FROM sys.symmetric_keys WHERE name='EmployeesKey'

Now let's stop here a moment to create a quick trace. I used a small trace definition to track the SQL:batchCompleted event, below is the output for the definition, the trace filters by the database name that we created above.

-- Create a Queue

declare @rc int

declare @TraceID int

declare @maxfilesize bigint

set @maxfilesize = 5

-- Please replace the text InsertFileNameHere, with an appropriate

-- filename prefixed by a path, e.g., c:\MyFolder\MyTrace. The .trc extension

-- will be appended to the filename automatically. If you are writing from

-- remote server to local drive, please use UNC path and make sure server has

-- write access to your network share

exec @rc = sp_trace_create @TraceID output, 0, N'InsertFileNameHere', @maxfilesize, NULL

if (@rc != 0) goto error

-- Client side File and Table cannot be scripted

-- Set the events

declare @on bit

set @on = 1

exec sp_trace_setevent @TraceID, 12, 1, @on

exec sp_trace_setevent @TraceID, 12, 11, @on

exec sp_trace_setevent @TraceID, 12, 12, @on

exec sp_trace_setevent @TraceID, 12, 13, @on

exec sp_trace_setevent @TraceID, 12, 16, @on

exec sp_trace_setevent @TraceID, 12, 35, @on

-- Set the Filters

declare @intfilter int

declare @bigintfilter bigint

exec sp_trace_setfilter @TraceID, 35, 0, 6, N'TestEncryption01'

-- Set the trace status to start

exec sp_trace_setstatus @TraceID, 1

-- display trace id for future references

select TraceID=@TraceID

goto finish

error:

select ErrorCode=@rc

finish:

go

Now after setup the trace and running it, let's go back to management studio to try inserting and selecting from the encrypted column using the keys

--now use the key to insert some records

OPEN SYMMETRIC KEY EmployeesKey DECRYPTION BY CERTIFICATE EmployeesCert;

DECLARE @X INT=1

WHILE @X<=1000    

BEGIN

    INSERT INTO SeniorEmployees(ID,EmployeeID,EmployeeSalary,EmployeePosition)

    VALUES(@X,ENCRYPTBYKEY(KEY_GUID('EmployeesKey'),CONVERT(NVARCHAR(20),CEILING(RAND()*999999)),1,CONVERT(varbinary,@X)),

    CEILING(RAND()*999999),

    ENCRYPTBYKEY(KEY_GUID('EmployeesKey'),N'BOSS#' + CONVERT(NCHAR(4),@X),1,CONVERT(varbinary,@X)))

    SET @X=@X+1;

END

CLOSE SYMMETRIC KEY EmployeesKey;

--Selecting WITH THE KEYS

OPEN SYMMETRIC KEY EmployeesKey DECRYPTION BY CERTIFICATE EmployeesCert;

SELECT ID,CONVERT(NVARCHAR(20),DECRYPTBYKEY(EmployeeID,1,CONVERT(VARBINARY,ID))) as 'EmployeeID',EmployeeSalary,

CONVERT(NVARCHAR(20),DECRYPTBYKEY(EmployeePosition,1,CONVERT(VARBINARY,ID))) as 'EmployeePosition'

FROM SeniorEmployees

CLOSE SYMMETRIC KEY EmployeesKey;

In the trace when we open the file in the profiler here's what we will get. The statement removed for both the insert and select statement and replaced by comments for protecting the encrypted data.

Same goes for the EncryptByPassPhrase function as well, continue with the remaining sample while running the trace

/*

Encrypt by pass phrase.

*/

--Create a new table

IF EXISTS (SELECT name from sys.tables where name='SeniorEmployees2')

DROP TABLE dbo.SeniorEmployees2

GO

CREATE TABLE dbo.SeniorEmployees2

    (

    ID int NOT NULL,

    EmployeeID varbinary(MAX) NOT NULL,

    EmployeeSalary decimal(8, 2) NOT NULL,

    EmployeePosition varbinary(MAX) NOT NULL

    ) ON [PRIMARY]

     TEXTIMAGE_ON [PRIMARY]

GO

ALTER TABLE dbo.SeniorEmployees2 ADD CONSTRAINT

    PK_SeniorEmployees2 PRIMARY KEY CLUSTERED

    (

    ID

    ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]

/*Insert by encryptByPassPhrase*/

DECLARE @X INT=1

WHILE @X<=1000    

BEGIN

    INSERT INTO SeniorEmployees2(ID,EmployeeID,EmployeeSalary,EmployeePosition)

    VALUES(@X,EncryptByPassPhrase('This is My Pass Phrase',CONVERT(NVARCHAR(20),CEILING(RAND()*999999)),1,CONVERT(varbinary,@X)),

    CEILING(RAND()*999999),

    EncryptByPassPhrase('This is My Pass Phrase',N'BOSS#' + CONVERT(NCHAR(4),@X),1,CONVERT(varbinary,@X)))

    SET @X=@X+1;

END

/*Select by DecryptByPassPhrase */

SELECT ID,CONVERT(NVARCHAR(20),DecryptByPassPhrase('This is My Pass Phrase',EmployeeID,1,CONVERT(VARBINARY,ID))) as 'EmployeeID',EmployeeSalary,

CONVERT(NVARCHAR(20),DecryptByPassPhrase('This is My Pass Phrase',EmployeePosition,1,CONVERT(VARBINARY,ID))) as 'EmployeePosition'

FROM SeniorEmployees2

And here's the view from the trace.

And for Extended Events as well, here's a quick screenshot of the XE outoupt

We all know the undocumented (then it's not supported) procedure that executes a code in every database which is sp_msforeachdb

And its twin that executes against all tables in one database which is sp_msforeachtable but sometimes we want to mix them together and it's challenging because the replacement character in both is "? " But there's a way to change it and I thought it's nice to share

The sp_msforeachtable takes another parameter for the replacement character so you can change it

So this statement will work for one database

exec sp_msforeachtable 'select * from $', '$'

And this one will get you everything from ALL databases

exec sp_msforeachdb 'USE ?;exec sp_msforeachTable ''select * from !'',''!''';

For testing this, I created a sample database in a SQL 2012 instance; I used the DMF sys.dm_db_database_page_allocations just to build some data which is only available in 2012 only. You can however copy the objectsPages table to SQL 2008/R2 and you will get the same results. The code below

IF DB_ID('inequality') IS NOT NULL

DROP DATABASE inequality;

GO

CREATE DATABASE inequality;

GO

USE inequality;

GO

IF EXISTS(SELECT name FROM sys.tables WHERE name='objectsPages')

DROP TABLE objectsPages;

CREATE TABLE objectsPages

(object_id bigint,index_id int,allocation_unit_id bigint,allocation_unit_type_desc nvarchar(50),allocated_page_iam_page_id bigint,

allocated_page_page_id bigint PRIMARY KEY, is_allocated bit,is_iam_page bit,is_mixed_page_allocation bit)

GO

INSERT objectsPages

SELECT object_id,index_id,allocation_unit_id,allocation_unit_type_desc,

allocated_page_iam_page_id,allocated_page_page_id,is_allocated,is_iam_page,is_mixed_page_allocation

FROM sys.dm_db_database_page_allocations(DB_ID('AdventureWorks2012'),null,null,null,'Limited');

GO

--Create indexes

CREATE INDEX is_iam_page ON objectsPages(is_iam_page);

CREATE INDEX is_mixed_page_allocation ON objectsPages(is_mixed_page_allocation);

To discuss the SARGability or whether or not SQL Server will consider a predicate as searchable or not, we have to differentiate between two operations

    -SQL Server is doing scan (table or index doesn't matter) because the Query Optimizer doesn't know how many rows will be returned

    -SQL Server is doing scan because SQL knows how many rows to be returned BUT the cost of scanning is calculated to be less than the cost of seeking

The first case, SQL Server consider the predicate non-SARGable and the second case, SQL Server is capable of seeking but decided to do a scan. Although the output is the same (scan operation) but the reason is totally different.

Now if we tried two queries like

SELECT allocated_page_page_id FROM objectsPages WHERE is_iam_page = 1;

SELECT allocated_page_page_id FROM objectsPages WHERE is_iam_page <> 1;

The execution plan will be as below. Note that in both cases SQL Server used seeks not scanning, and SQL Server can tell the expected number of Rows because we have statistics created with the index and SQL can get the estimates. However SQL Server has to convert the query to be equality by using seek predicates of < and > instead of <> (not equal). You can check that in the seek predicate in the index seek. The part of the execution plan marked with the blue marker is the part where SQL Server prepare the values for > and <

But if we changed the query a little bit and instead of only selecting the clustered index key; selecting the whole table

SELECT * FROM objectsPages WHERE is_iam_page = 1;

SELECT * FROM objectsPages WHERE is_iam_page <> 1;

The execution plan shows in both cases that SQL Server didn't use the predicate and decided to scan the table because the cost of scanning is estimated to be lower than the cost of index seek + key lockups operations. Whether or not we are using inequality is irrelevant in that case.

I found this blog post that describe the same problem and it has good solution for it. I used the solution 1. The only difference is in my NVidia control panel, it says clone instead of duplicate. I'm adding that blog post here for reminding me and to share it in case there are others facing the same problem.

http://bretstateham.com/lenovo-w520-and-an-external-projector-with-nvidia-quadro-2000m-and-nvidia-optimus/

Open_transaction_count: which shows the number of open transactions for the session whether or not these transactions are reading or writing. So an open transaction that you cannot see in DBCC OpenTran will be counted here. Similar to open_tran in sys.sysprocesses.

Database_id: The long awaited column in exec_sessions is finally here. No need for sp_who2 any more to know which session connected to which database. Of course to show to database name you need to use db_name() function.

Authenticating_database_id: This columns was added to support the contained databases. It contains the database_id of the database that did the authentication for the session. If it's a login then it should show the database_id for the master database (1) otherwise it will show the database_id of the contained database that has the user with password. The sys.dm_exec_sessions documentation documented this column as it will have 0 when it's a login but I tested it and it shows 1. I'll try to file a documentation bug for this one.

~M.S.

However when checking sysprocesses I see there's a session with open_tran =1. And this behavior is reproducible in SQL 2012 as well so it's not related to SQL 2000. So why DBCC OpenTran doesn't show it?

Well, it's not clear in in the DBCC OpenTran documentation but DBCC OpenTran shows the open transactions by reading the Transaction log and gets the open transactions from it. If the transaction is not listed in the Transaction Log it will not be listed by DBCC OpenTran. In this case the open transaction was a select transaction that it doesn't show up in the transaction log so the DBCC OpenTran didn't list it however you can get it using sysprocesses. Also after troubleshooting this issue I found that the select transaction is using Serializable transaction isolation level which caused lots of range locks which blocked the index maintenance job and in return blocked many other processes.

To reproduce the problem you can do this by opening two sessions to AdventureWorks sample database

P1

Set Transaction Isolation Level Serializable;

go

Begin Tran

Select * from HumanResources.Department;

P2

DBCC OpenTran('AdventureWorks2008R2'); --reads from the T-Log

Select * from sys.sysprocesses where open_tran>0;

select * from sys.dm_tran_locks;

Also if you are using Simple recovery model you can confirm that P1 didn't leave a trace in the Transaction Log using the undocumented function fn_dblog().

For more information and download, go to product site