Determining the appropriate size for tempdb in a production environment depends on many factors. All those factors have been mentioned in the article “Capacity Planning for tempdb”.

Capacity Planning for tempdb
https://msdn.microsoft.com/en-us/library/ms345368(v=sql.105).aspx

You can also use the basic calculation like below

declare @tempdb_size1 decimal(15,2) = 0
declare @tempdb_size2 decimal(15,2) = 0
declare @tempdb_size3 decimal(15,2) = 0
declare @tempdb_size decimal(15,2) = 0
declare @database_size_info table (
    database_name sysname, database_size decimal(15,2)
)
insert into @database_size_info
select db_name(database_id) database_name, 
SUM (CONVERT (numeric (15,2) , (convert(numeric, size) * 8192)/1048576)) 
from sys.master_files
group by database_id
--10% of the total size of databases greater than 1TB
select @tempdb_size1 = isnull(sum(database_size),0)
from @database_size_info
where database_size >= 1000000
--15% of the total size of databases greater than 100GB and less than 1TB
select @tempdb_size2 = isnull(sum(database_size),0)
from @database_size_info
where database_size >= 100000 and database_size < 1000000
--25% of the total size of databases greater less than 100GB
select @tempdb_size3 = isnull(sum(database_size),0)
from @database_size_info
where database_size < 100000

set @tempdb_size = (@tempdb_size3*0.25)+(@tempdb_size2*0.15)+(@tempdb_size1*0.10)

select @tempdb_size tempdb_estimated_database_size, 
        (@tempdb_size*0.75 ) tempdb_estimated_data_initial_size, 
        (@tempdb_size*0.25 ) tempdb_estimated_log_initial_size

There are some other best practices for tempdb such as one tempdb file for each core processors. If you plan to add addition tempdb data files, tempdb_estimated_data_initial_size can be divided the core processor count.

Setup Windows Server 2012 Cluster in Hyper-V

• iSCSI Target Configuration
• iSCSI Initiator Configuration
• Setup Windows Server 2012 Cluster

There are different types of Quorum models. Nodes and Disk Majority type will be used fir this example.

Pre-Requisites

• 1 Windows Server 2012 server (It will be called as DC01 in this example)
• 2 Windows Server 2012 servers (They will be called as Node01 and Node02 in this example)
• 1 Windows Server 2012  server (It will be called Storage01 in this example – I also used DC01 as a Storage01 server)

Storage Configuration

Step 1: Storage01 iSCSI Target Server Installation

• Open “Server Manager” and click  “Dashboard”
• Click “Add roles and features”
• Before You Begin: Click Next
• Installation Type: Select “Role-based or feature-based installation” and click Next
• Server Selection: Select the server on which to install roles and features. If there are more than one servers within the “Server Pool”, select the appropriate one to enable iSCSI Target. Select Storage01 server and click Next
• Server Roles: Select “File and Storage Services” and then “File and iSCSI Services” and then “File Server” and then select “iSCSI Target Server” to enable iSCSI target server role and click Next
• Features: Click Next
• Confirmation: Click Installation
• Restart in necessary

Step2: Node01 Start “Microsoft iSCSI Initiator Service”

• Open Services console
• Right click on “Microsoft iSCSI Initiator Service” and then select properties
• Change “Startup type” to Automatic and then press Start button to start the service and then press OK button

Step3: Node02 Start “Microsoft iSCSI Initiator Service”

• Open Services console
• Right click on “Microsoft iSCSI Initiator Service” and then select properties
• Change “Startup type” to Automatic and then press Start button to start the service and then press OK button

Step 4: Storage01 iSCSI Target Server Configuration

• Open “Server Manager” and click  “File and Storage Services” and then click “iSCSI”
• Select “New iSCSI Virtual Disk…” within TASKS menu
• iSCSI Virtual Disk Location: Select the Volume Drive to create the virtual disk. Click Next
  Virtual Hard Disks are created within the “C:\iSCSIVirtualDisks” folder by default. If necessary, the patch can be changed by selecting “Type a custom path” option.
  If there is not enough space in the selected volumes (Check the Free Space column), create a new “Hard Disk…” using the “Hyper-V” console (Actions – New – Hard Disk).
• iSCSI Virtual Disk Name: Type the name of the virtual disk (LUN1.vhd) and click Next
• iSCSI Virtual Disk Size: Type the size of the virtual disk (512MB for Quorum Disk) and click Next
• iSCSI Target: Select “New iSCSI target” and click Next
  iSCSI target can have more than one virtual disks and any server can connect the “iSCSI Target” and access the Hard Disks within the iSCSI Target. If you plan to create windows cluster, you can create one iSCSI Target with a few Hard Disks and then you can set the same iSCSI Target for both nodes using the iSCSI Initiator on the nodes. It is called as shared storage. If you plan another set of Hard Disks for another server, you can create a new iSCSI Target with a few Hard Disks and that server can access the Hard Disk by connecting the new Target Server.
• Target Name and Access: Type the name of the new Target Server (iscsitarget1) and click Next
• Access Servers: Click Add to add an initiator(s) (Initiator is the server name which will have access to the Target Server which has the Virtual Hard Disks) and click Next
  If it is planned for the windows cluster, every single node name within the cluster should be added
  Our example is also a windows cluster so Add “Node01” and “Node02” as Initiators.
  When you add the initiator, you can select “Enter a value for the selected type” and then select “IQN” and then type Value. To write the correct “Value”, connect to Node01 and start the “iSCSI Initiator” and then select the “Configuration” tab and copy the “Initiator Name” and paste it within the Value field.
• Enable authentication: Click Next
• Confirmation: Click Create
• Results: Click Close

Step 5: Node01 iSCSI Initiator Configuration

• Open iSCSI Initiator and Click “Discovery” tab
• Click “Discovery Portal…” and type the IP address of the iSCSI Target server (IP Address of Storage01) and then click OK
• Click “Targets” tab and Select the target within the “Discovered targets” field (iscsitarget1) and then click Connect
• Keep the all default values and Click OK
• Status should be “Connected”
• Click OK to close the “iSCSI Initiator Properties”
• Open “Server Manager” and click  “File and Storage Services” and then click “Disks”
• The new disk (512MB for Quorum) should be presented as Offline
• Right click on the disk and select “”Bring Online” and then Click Yes to accept the message
• Right click on the new disk and select “New Volume”
• Before You Begin: Click Next
• Server and Disk: Select the appropriate Server and Disk and then click Next and click OK to accept the message
• Size: Select the appropriate size for the new disk and then click Next
• Drive Letter or Folder: Select the appropriate drive letter and ten Click Next
• File System Settings: Type the label of the volume (Quorum Disk)
• Confirmation: Click Create
• Results: Click Close

Step 6: Node02 iSCSI Initiator Configuration

• Open iSCSI Initiator and Click “Discovery” tab
• Click “Discovery Portal…” and type the IP address of the iSCSI Target server (IP Address of Storage01) and then click OK
• Click “Targets” tab and Select the target within the “Discovered targets” field (iscsitarget1) and then click Connect
• Keep the all default values and Click OK
• Status should be “Connected”
• Click OK to close the “iSCSI Initiator Properties”

• Open “Server Manager” and click  “File and Storage Services” and then click “Disks”
• The new disk should be presented as Offline
• Right click on the disk and select “”Bring Online” and then Click Yes to accept the message

512MB Disk (Quorum) can be seen on both nodes.

Setup Windows Server 2012 Failover Cluster

Step 7: Node01 Install Failover Clustering

• Open “Server Manager” and click   “Dashboard”
• Click “Add roles and features”
• Before You Begin: Click Next
• Installation Type: Select “Role-based or feature-based installation” and click Next
• Server Selection: Select the server on which to install roles and features. If there are more than one servers within the “Server Pool”, select the appropriate one to enable iSCSI Target. Select Storage01 server and click Next
• Server Roles: Click Next
• Features: Select “Failover Clustering” Click Next
• Confirmation: Click Installation
• Restart in necessary

Step 8: Node02 Install Failover Clustering

• Open “Server Manager” and click  “Dashboard”
• Click “Add roles and features”
• Before You Begin: Click Next
• Installation Type: Select “Role-based or feature-based installation” and click Next
• Server Selection: Select the server on which to install roles and features. If there are more than one servers within the “Server Pool”, select the appropriate one to enable iSCSI Target. Select Storage01 server and click Next
• Server Roles: Click Next
• Features: Select “Failover Clustering” Click Next
• Confirmation: Click Installation
• Restart in necessary

Step 9: Node01 Validate and Create Windows Cluster

• Open “Failover Cluster Manager”
• Click “Validate Configuration”
• Before You Begin: Click Next
• Select Servers or a Cluster: Add the node names (Node01 and Node02) and click Next
• Testing Options: Select “Run all tests (recommended)” and click Next
• Confirmation: Click Next
• Validating: “Create the cluster now using the validated nodes..” is checked and click Finish
• Before You Begin: Click Next
• Access Point for Administering the Cluster: Type the cluster name (SQLNodeClu). Cluster Name object will be created in Active Directory. The user who creates the cluster should have permission in active directory such as create computer object. Type the IP address from the public network.
• Confirmation: “Add all eligible storage to the cluster” is checked. Click Next
• Creating New Cluster: Finish
• Check the Quorum Setting: On the left-hand side select “Failover Cluster Manager” and windows cluster name (SQLNodeClu) and right click on the windows cluster name (SQLNodeClu) and select “More Actions” and then “Configure Cluster Quorum Settings”
• Before You Begin: Click Next
• Select Quorum Configuration Option: Select “Add or change the quorum witness” and then click Next
• Select Quorum Witness: Select “Configure a disk witness” and then click Next
• Configure Storage Witness: Check the Quorum disk. If it is not correct, select the correct disk and then click Next
• Confirmation: Click Next
• Summary: Finish
• Check the “Current Host Server” and test the failover by selecting “Move Core Cluster Resources” option.

The Result

It is ready to have one of the options below

• New SQL Server failover cluster installation
  Failover cluster installation requires shared storage for system databases, user databases, tempdb, backup. Check the steps between “Step 1” and “Step 6”
• SQL Server 2012 AlwaysOn

If there is an index on the object, the data is ordered in the page using the slot array. When a new insert comes, SQL engine will search an available page to keep the data by using the index. The data should be saved in the specific page because it should be ordered. The data should be in the specific page but if there is no enough space in the page, a new page will be allocated and the original page will be split and previous, next page numbers will be updated. This process will cause fragmentation. It can be monitored using the “page split/sec” counter.

Another reason is because of the data type. Let’s think about that we have one table with a column which is varchar(100) data type. If 10 characters are inserted, it will allocate 10 bytes and if the page is full and if the data is updated to 20 characters it will look for 10 more bytes in the same page. But the page is full so it will place a pointer to forward the date to a new page. This process will also cause fragmentation. It can be monitored using the “forwarded records/sec” counter.

If the pages are fragmented, it will increase the disk search time and it will cause the performance issue.

How can we monitor the fragmentation and How can we fix the fragmentation issue?

Maintenance Plans can be used like below.

Create a maintenance plan	Maintenance Tasks

If the fragmentation level is greater than 25 or 30 percent, Rebuild Index Task is recommended otherwise Reorganize Index Task can be used. But there is no any option to give the threshold in those tasks.

Rebuild Index Task also updates the statistics but Reorganize Index Task does not.

Rebuild Index Task can take longer time to be completed. There may be some big indexes on read-only tables so it is not useful to rebuild them. It will only cause longer time. How can we eliminate those indexes from the plan? The answer is custom script.

You can check the script below and implement it on your environment. You can make some customization on that. Please let us know if you have any improvement on that

--part 1: create the objects 
SET NOCOUNT ON;

if not exists (select name from sys.databases where name = 'SQLAdmin')
    create database SQLAdmin
go
use SQLAdmin
go
if not exists (select name from sys.objects where name = 'fragmentation_history')
   create table fragmentation_history(
        history_id int identity(1,1), 
        database_id smallint NULL,
        database_name sysname NULL,
        schema_id int null,
        schema_name sysname null,
        object_id int NULL,
        object_name sysname NULL,
        index_id int NULL,
        index_name sysname NULL,
        partition_number int NULL,
        avg_fragmentation_in_percent_before float NULL,
        avg_fragmentation_in_percent_after float NULL,
        alter_start datetime NULL,
        alter_end datetime NULL,
        progress datetime NULL
    ) ON [PRIMARY]
if not exists (select name from sys.objects where name = 'sql_errors')
    create table sql_errors(
        error_id int identity(1,1),
        command varchar(4000) null,
        error_number int null,
        error_severity smallint null,
        error_state smallint null,
        error_line int null,
        error_message varchar(4000) null,
        error_procedure varchar(200) null,
        time_stamp datetime null,
        primary key clustered 
        (
            error_id asc
        ) ON [PRIMARY]
    ) ON [PRIMARY]
GO
if exists (select name from sys.objects where name = 'p_error_handling')
    drop procedure p_error_handling
go
create procedure p_error_handling 
    @command varchar(4000)
as
DECLARE @error_number int
DECLARE @error_severity int
DECLARE @error_state int
DECLARE @error_line int
DECLARE @error_message varchar(4000)
DECLARE @error_procedure varchar(200)
DECLARE @time_stamp datetime

SELECT @error_number = isnull(error_number(),0),
        @error_severity = isnull(error_severity(),0),
        @error_state = isnull(error_state(),1),
        @error_line = isnull(error_line(), 0),
        @error_message = isnull(error_message(),'NULL Message'),
        @error_procedure = isnull(error_procedure(),''),
        @time_stamp = GETDATE();

INSERT INTO dbo.sql_errors (command, error_number, error_severity, error_state, error_line, error_message, error_procedure, time_stamp)
SELECT @command, @error_number, @error_severity, @error_state, @error_line, @error_message, @error_procedure, @time_stamp

GO

--part 2: Collect the fragmentation data
SET NOCOUNT ON;
DECLARE @command varchar(8000);
DECLARE @databaseid int;
DECLARE @databasename sysname;

DECLARE database_list CURSOR FOR 
    SELECT database_id, name 
    FROM sys.databases 
    where database_id > 4 and state not in (1)
    order by name

-- Open the cursor.
OPEN database_list

-- Loop through the partitions.
FETCH NEXT FROM database_list 
   INTO @databaseid, @databasename

WHILE @@FETCH_STATUS = 0
    BEGIN
        --set @databasename = 'AdventureWorks2008'
        set @command = 'use [' + @databasename + '];'
        set @command = @command + '
                                    insert into SQLAdmin.dbo.fragmentation_history (database_id, database_name, schema_id, schema_name, object_id, object_name, index_id, index_name, partition_number, avg_fragmentation_in_percent_before)
                                    SELECT     D.database_id, D.name, O.schema_id, s.name, IPS.object_id, O.name, IPS.index_id, I.name, partition_number, avg_fragmentation_in_percent
                                    FROM sys.dm_db_index_physical_stats (' + convert(varchar(3), @databaseid) + ', NULL, NULL , NULL, ''LIMITED'') IPS
                                    join sys.databases D on IPS.database_id=D.database_id
                                    join sys.objects O on IPS.object_id = O.object_id
                                    join sys.schemas as s ON s.schema_id = O.schema_id
                                    join sys.indexes I on IPS.object_id = I.object_id and IPS.index_id = I.index_id
                                    WHERE D.state not in (1) and avg_fragmentation_in_percent > 5.0 AND IPS.index_id > 0
                                    and page_count>1000                                    
                                    '
        exec (@command)
        FETCH NEXT FROM database_list 
           INTO @databaseid, @databasename
    END;
-- Close and deallocate the cursor.
CLOSE database_list;
DEALLOCATE database_list;

--part 3: fix the fragmentation with rebuild or reorganize according to the threshold
SET NOCOUNT ON;
DECLARE @historyid int;
DECLARE @command varchar(8000);
DECLARE @command1 varchar(7950);
DECLARE @command2 varchar(50);
DECLARE @databaseid int;
DECLARE @databasename sysname;
DECLARE @schemaid int;
DECLARE @schemaname sysname;
DECLARE @objectid int;
DECLARE @objectname sysname;
DECLARE @indexid int;
DECLARE @indexname sysname;
DECLARE @partitionnumber bigint;
DECLARE @frag_before float;
DECLARE @frag_after float;
DECLARE @alterstart datetime;
DECLARE @alterend datetime;
DECLARE @progress datetime;

set @progress = getdate()

DECLARE fragmentation_list CURSOR FOR 
    SELECT history_id, database_id, database_name, schema_id, schema_name, object_id, object_name, index_id, index_name, partition_number, avg_fragmentation_in_percent_before 
    FROM SQLAdmin.dbo.fragmentation_history 
    where progress is null;

-- Open the cursor.
OPEN fragmentation_list;

-- Loop through the partitions.
FETCH NEXT
   FROM fragmentation_list
   INTO @historyid, @databaseid, @databasename, @schemaid, @schemaname, @objectid, @objectname, @indexid, @indexname, @partitionnumber, @frag_before;

WHILE @@FETCH_STATUS = 0

    BEGIN;

        -- 25 is an arbitrary decision point at which to switch between reorganizing and rebuilding
        IF @frag_before < 25.0
            BEGIN;
                set @command1 = 'use ' + @databasename + ';'
                set @command1 = @command1 + '
                                            DECLARE @partitioncount bigint;
                                            SELECT @partitioncount = count (*)
                                            FROM sys.partitions
                                            WHERE object_id = ' + convert(varchar(10), @objectid) + ' AND index_id = ' + convert(varchar(10), @indexid) + ';
                                            IF @partitioncount > 1
                                                ALTER INDEX [' + @indexname + '] ON ' + @schemaname + '.[' + @objectname + '] REORGANIZE' + ' PARTITION=' + CONVERT (CHAR, @partitionnumber) + ';
                                            else
                                                ALTER INDEX [' + @indexname + '] ON ' + @schemaname + '.[' + @objectname + '] REORGANIZE' + ';
                                            UPDATE STATISTICS ' + @schemaname + '.[' + @objectname + '] [' + @indexname + '];
                                            '
                set @command = @command1
            END;

        IF @frag_before >= 25.0
            BEGIN;
                set @command1 = 'use ' + @databasename + ';'
                set @command1 = @command1 + '
                                            DECLARE @partitioncount bigint;
                                            SELECT @partitioncount = count (*)
                                            FROM sys.partitions
                                            WHERE object_id = ' + convert(varchar(10), @objectid) + ' AND index_id = ' + convert(varchar(10), @indexid) + ';
                                            IF @partitioncount > 1
                                                ALTER INDEX [' + @indexname +'] ON ' + @schemaname + '.[' + @objectname + '] REBUILD' + ' PARTITION=' + CONVERT (CHAR, @partitionnumber) + '
                                            else
                                                ALTER INDEX [' + @indexname +'] ON ' + @schemaname + '.[' + @objectname + '] REBUILD' + '
                                            '
                set @command2 = 'with (online=ON)'
                set @command = @command1 + ' ' + @command2
            END;

            begin try
                    set @alterstart = getdate() 
                    EXEC (@command);
                    set @alterend = getdate()
    
                    select @frag_after = avg_fragmentation_in_percent  
                    from sys.dm_db_index_physical_stats(@databaseid, @objectid, @indexid, @partitionnumber,  'LIMITED')

                    update SQLAdmin.dbo.fragmentation_history
                    set alter_start = @alterstart,
                        alter_end = @alterend,
                        avg_fragmentation_in_percent_after = @frag_after,
                        progress = @progress
                    where history_id = @historyid
            end try    
            begin catch
                    if error_number() = 2275 or error_number() = 153
                        begin
                            set @alterstart = getdate() 
                            EXEC (@command1);
                            set @alterend = getdate()
    
                            select @frag_after = avg_fragmentation_in_percent  
                            from sys.dm_db_index_physical_stats(@databaseid, @objectid, @indexid, @partitionnumber,  'LIMITED')

                            update SQLAdmin.dbo.fragmentation_history
                            set alter_start = @alterstart,
                                alter_end = @alterend,
                                avg_fragmentation_in_percent_after = @frag_after,
                                progress = @progress
                            where history_id = @historyid
                        end
                    --select @command command, @databasename database_name, @schemaname schema_name, @objectname object_name, @indexname index_name, @partitionnumber partition_number, 
                        --    error_number() error_number , ERROR_SEVERITY() error_severity, ERROR_STATE() error_state, ERROR_LINE() error_line, ERROR_MESSAGE() error_message
                    exec p_error_handling  @command
            end catch

            FETCH NEXT
               FROM fragmentation_list
               INTO @historyid, @databaseid, @databasename,  @schemaid, @schemaname, @objectid, @objectname, @indexid, @indexname, @partitionnumber, @frag_before;
    END;

-- Close and deallocate the cursor.
CLOSE fragmentation_list;
DEALLOCATE fragmentation_list;

--part 4: get the most expensive index maintenance by duration
select datediff(ms, alter_start, alter_end) duration_ms, alter_start, alter_end,
database_name, schema_name, object_name, index_name, avg_fragmentation_in_percent_before, 
avg_fragmentation_in_percent_after from SQLAdmin.dbo.fragmentation_history
order by duration_ms desc

[RECID] [varchar](128) NOT NULL,
[XMLRECORD] [xml] NULL,

I used a couple of queries to tune the performance issue

select wait_type, 
coalesce (db_name(st.dbid), db_name(convert (int, pa.value)), 'Empty') as DBName, 
SUBSTRING(st.text, (qs.statement_start_offset/2)+1, 
        ((CASE qs.statement_end_offset
          WHEN -1 THEN DATALENGTH(st.text)
         ELSE qs.statement_end_offset
         END - qs.statement_start_offset)/2) + 1) AS StatementText,
st.text as ProcedureTextOrBatchText,         *
from sys.dm_exec_requests qs
cross apply sys.dm_exec_sql_text(sql_handle) st
cross apply sys.dm_exec_query_plan(plan_handle) qp
cross apply sys.dm_exec_plan_attributes(qs.plan_handle) pa
where attribute = 'dbid' and session_id <> @@SPID

Most of the queries were waiting on the RESOURCE_SEMAPHORE wait type.

RESOURCE_SEMAPHORE: Occurs when a query memory request cannot be granted immediately because of other concurrent queries. High waits and wait times can indicate excessive number of concurrent queries or excessive memory request amount.
COMMON for DSS like workload and large queries such as hash joins; must wait for memory quota (grant) before it is executed.

SQL Server 2005 Waits and Queues

https://download.microsoft.com/download/4/7/a/47a548b9-249e-484c-abd7-29f31282b04d/Performance_Tuning_Waits_Queues.doc

The queries below can be used to troubleshoot memory related issues

SELECT grantee_count, waiter_count
FROM sys.dm_exec_query_resource_semaphores
where pool_id > 1

grantee_count    4
waiter_count      5

It looks that there are 5 queries waiting for the memory

Used another query like below

SELECT text, session_id, requested_memory_kb, granted_memory_kb, required_memory_kb 
FROM sys.dm_exec_query_memory_grants
cross apply sys.dm_exec_sql_text(sql_handle)

It looks that 3 queries (b, d, f) are waiting for the memory (granted_memory_kb is null) and those queries require 12.8GB memory. It is too much. Each of the other queries (a, c, e, g, h) has already allocated 12.8GB memory. Those queries are really expensive.
The server has 98GB memory and max server memory has been set to 86GB memory.

Check the queries below to double check the memory pending issue

select  cntr_type as [Memory Grants Outstanding]
from sys.dm_os_performance_counters
where object_name = 'SQLServer:Memory Manager'
and counter_name = 'Memory Grants Outstanding'

select cntr_value as [Memory Grants Pending]
from sys.dm_os_performance_counters
where object_name = 'SQLServer:Memory Manager'
and counter_name = 'Memory Grants Pending'

It look that “Memory Grants Pending” is 4.

So it is understood that the queries which are killing the memory should be improved.

Let’s work on the performance improvement of one of the queries. As you remember there is an XML data so which one is better? To create XML indexes or PERSISTED column?

Table design

CREATE TABLE [dbo].[XMLDEMO_XMLINDEX](
    [RECID] [varchar](128) NOT NULL,
    [XMLRECORD] [xml] NULL,
 CONSTRAINT [PK_XMLDEMO] PRIMARY KEY CLUSTERED 
(
    [RECID] ASC
)
)

Fill the table with XML data. The below script is just an example. You generate XML data for each row and insert it to test table called XMLDEMO_XMLINDEX

SELECT TOP 1 * FROM [AdventureWorksDW2012].[dbo].[FactResellerSalesPtnd]
FOR XML RAW, ELEMENTS

Create a PRIMARY XML Index and SECONDARY XML PATH Index on the table and test the query and see the results

CREATE PRIMARY XML INDEX [IX_PRIMARY_XML] ON [dbo].[XMLDEMO_XMLINDEX]
(
    [XMLRECORD]
)
GO
CREATE XML INDEX [IX_SECONDARY_XML_PATH] ON [dbo].[XMLDEMO_XMLINDEX]
(
    [XMLRECORD]
)
USING XML INDEX [IX_PRIMARY_XML] FOR PATH 
GO

Check the size of the table and index

sp_spaceused 'XMLDEMO_XMLINDEX'

rows = 616669
reserved = 5842864 KB
data = 1332880 KB
index_size = 4509240 KB
unused = 744 KB

Check the query performance below (Table has Clustered Index, PRIMARY XML Index and SECONDARY XML Indexes)

SET STATISTICS IO ON
GO
SET STATISTICS TIME ON
GO
SELECT RECID FROM [dbo].[XMLDEMO_XMLINDEX] 
WHERE XMLRECORD.exist(N'/row[SalesOrderNumber/text()="SO43676"]') = 1

/*
SQL Server parse and compile time: 
   CPU time = 0 ms, elapsed time = 0 ms.

(572 row(s) affected)
Table 'XMLDEMO_XMLINDEX'. Scan count 0, logical reads 2288, physical reads 0, read-ahead reads 0, 
Table 'xml_index_nodes_1298103665_256000'. Scan count 1, logical reads 2298, physical reads 0, read-ahead reads 0, 

 SQL Server Execution Times:
   CPU time = 15 ms,  elapsed time = 18 ms.
*/

Create the similar table with the same data like below

CREATE TABLE [dbo].[XMLDEMO_PERSISTEDCOLUMN](
    [RECID] [varchar](128) NOT NULL,
    [XMLRECORD] [xml] NULL,
 CONSTRAINT [PK_XMLDEMO_PERSISTEDCOLUMN] PRIMARY KEY CLUSTERED 
(
    [RECID] ASC
)
)
GO
INSERT INTO [XMLDEMO_PERSISTEDCOLUMN] (RECID, XMLRECORD)
SELECT RECID, XMLRECORD FROM [XMLDEMO_XMLINDEX]
GO

sp_spaceused 'XMLDEMO_PERSISTEDCOLUMN'

rows = 616669
reserved = 991944 KB
data = 986672 KB
index_size = 5128 KB
unused = 144 KB

Check the query performance below (Table has Clustered Index)

SET STATISTICS IO ON
GO
SET STATISTICS TIME ON
GO
SELECT RECID FROM [dbo].XMLDEMO_PERSISTEDCOLUMN 
WHERE XMLRECORD.exist(N'/row[SalesOrderNumber/text()="SO43676"]') = 1

/*
SQL Server parse and compile time: 
   CPU time = 0 ms, elapsed time = 0 ms.

(572 row(s) affected)
Table 'XMLDEMO_PERSISTEDCOLUMN'. Scan count 9, logical reads 140107, physical reads 0, read-ahead reads 0, 

 SQL Server Execution Times:
   CPU time = 55677 ms,  elapsed time = 7168 ms.
*/

Create a PERSISTED Column on the table and Proper index on the persisted column like below

CREATE FUNCTION [dbo].[udf_XMLDEMO_PERSISTEDCOLUMN_SalesOrderNumber](@xmlrecord XML) 
RETURNS nvarchar(14) 
WITH SCHEMABINDING 
BEGIN
RETURN @xmlrecord.value('(/row/SalesOrderNumber/text())[1]','nvarchar(20)')
END
GO
ALTER TABLE XMLDEMO_PERSISTEDCOLUMN
ADD SalesOrderNumber AS dbo.udf_XMLDEMO_PERSISTEDCOLUMN_SalesOrderNumber(XMLRECORD) PERSISTED
GO
CREATE INDEX [ix_XMLDEMO_PERSISTEDCOLUMN_SalesOrderNumber] ON XMLDEMO_PERSISTEDCOLUMN(SalesOrderNumber)

sp_spaceused 'XMLDEMO_PERSISTEDCOLUMN'

rows = 616669
reserved = 1011536 KB
data = 986672 KB
index_size = 24640 KB
unused = 224 KB

Check the query performance below (Table has Clustered Index, Persisted Column, Non-Clustered Index)

SET STATISTICS IO ON
GO
SET STATISTICS TIME ON
GO
SELECT RECID FROM [dbo].XMLDEMO_PERSISTEDCOLUMN  
WHERE SalesOrderNumber = 'SO43676'

/*
SQL Server parse and compile time: 
   CPU time = 0 ms, elapsed time = 0 ms.
SQL Server parse and compile time: 
   CPU time = 0 ms, elapsed time = 0 ms.

(572 row(s) affected)
Table 'XMLDEMO_PERSISTEDCOLUMN'. Scan count 1, logical reads 6, physical reads 0, read-ahead reads 0, 

 SQL Server Execution Times:
   CPU time = 0 ms,  elapsed time = 0 ms.
*/

Summary of this comparison

Table with a PRIMARY KEY CLUSTERED

Table 'XMLDEMO_PERSISTEDCOLUMN'. Scan count 9, logical reads 140107, physical reads 0, read-ahead reads 0,
SQL Server Execution Times: Elapsed time = 7168 ms

Table with a PRIMARY KEY CLUSTERED and PRIMARY XML INDEX and SECONDARY XML PATH INDEX

Table with a PRIMARY KEY CLUSTERED and PERSISTED Column and NON-CLUSTERED Index

Table 'XMLDEMO_PERSISTEDCOLUMN'. Scan count 1, logical reads 6, physical reads 0, read-ahead reads 0,
SQL Server Execution Times: Elapsed time = 0 ms

The question is “If you have a few busy tables which are in the same file group, The IO performance will be improved after moving those files to another drive?”

The answer is No. It is recommended to keep those busy tables in separate files. How can you do that? You can create different file groups and each file group has its own files. Then you can create tables or create clustered indexes or rebuild clustered indexes on those file groups to move those busy tables to separate files.

Another question is “How can you find the busy tables in SQL Server?”

A DMV (dynamic management view) called sys.dm_db_index_operational_stats can be used for this purpose.

Detailed information about this DMV can be taken from the link below

https://msdn.microsoft.com/en-us/library/ms174281.aspx

You can use the columns leaf_insert_count, leaf_update_count, leaf_delete_count, range_scan_count to get the access hit for the objects. This DMV keeps the cumulative data and returns the access method activity for each partition of a table or index in the database. So if a table has 2 indexes, the query returns 3 rows, 2 rows for indexes and 1 row for HEAP table or CLUSTERED index. You should consider the below items:

DMVs are kept in the memory. If SQL SERVER is restarted, the history of the DMV will be lost.

You should have a good history to talk about the access hits. You can create a job which runs everyday and get the data from DMV to static table. You can overwrite the static data. But if SQL SERVER is restarted, it shouldn’t be overwritten otherwise you can lose your static history data. If it is restarted, the data from DMV should be added to the static data.

A good script has been created for you. You can create a job with the name “SQLAdmin_TableAccessHistory” and schedule it for every day.

USE master;
GO
if not exists (select name from sys.databases where name = 'SQLAdmin')
    create database SQLAdmin
go
use SQLAdmin
go
if not exists (select name from sys.objects where name = 'TableAccessHistory')
    create table TableAccessHistory (
        TableAccessHistoryID int identity (1,1),
        database_id int,
        database_name sysname,
        object_id int,
        object_name sysname,
        insert_count bigint,
        update_count bigint,
        delete_count bigint,
        select_count bigint
        constraint [PK_TableAccessHistory] primary key clustered
        (TableAccessHistoryID ASC)
    )
go
--use AdventureWorks2008
--go
declare @service_start_time datetime
declare @job_last_success_time datetime --it is the job scheduled to collect historic data from sys.dm_db_index_operational_stats
declare @database_id int
declare @database_name sysname
declare @sql as varchar(4000)

--test
--set @service_start_time = '2013-03-06 16:00:00'
--prod
select @service_start_time = sqlserver_start_time
from sys.dm_os_sys_info
--print @service_start_time

--test
--set @job_last_success_time = '2013-03-06 02:00:00' --last successfull time
--prod
select top 1 
@job_last_success_time = convert(datetime, rtrim(run_date)) 
+ ((run_time/10000 * 3600) 
+ ((run_time%10000)/100*60) 
+ (run_time%10000)%100) / (86399.9964 )
from msdb.dbo.sysjobhistory jh
join msdb.dbo.sysjobs j on jh.job_id = j.job_id
where j.name = 'SQLAdmin_TableAccessHistory' 
and jh.step_id = 0 and jh.run_status = 1
order by instance_id desc
--print @job_last_success_time 

set @database_id = db_id('SQLAdmin')

declare dbname_cursor CURSOR FOR
SELECT name, database_id from sys.databases 
where database_id not in (1,2,3,4, @database_id) and state=0

OPEN dbname_cursor

FETCH NEXT FROM dbname_cursor
INTO @database_name, @database_id

WHILE @@FETCH_STATUS = 0
BEGIN
    
    set @sql = '
        set nocount on
        declare @object_id int
        declare @object_name sysname
        declare @insert_count bigint
        declare @update_count bigint
        declare @delete_count bigint
        declare @select_count bigint

        declare TableAccessHistory_cursor CURSOR FOR
        SELECT ios.object_id, object_name(ios.object_id, ios.database_id) object_name, 
        leaf_insert_count insert_count, 
        leaf_update_count update_count, 
        leaf_delete_count delete_count, 
        range_scan_count select_count
        FROM  sys.dm_db_index_operational_stats (' + convert(varchar(5),@database_id) + ', NULL, NULL, NULL) ios
        join  [' + @database_name  + '].sys.objects o on ios.object_id = o.object_id
        where ios.index_id in (0, 1)
        and o.type = ''U'' and o.is_ms_shipped = 0
        
        OPEN TableAccessHistory_cursor

        FETCH NEXT FROM TableAccessHistory_cursor
        INTO @object_id, @object_name, @insert_count, @update_count, @delete_count, @select_count

        WHILE @@FETCH_STATUS = 0
        BEGIN
            if exists (select TableAccessHistoryID from TableAccessHistory
                        where database_id = ' + convert(varchar(5),@database_id) + ' and object_id = @object_id) 
                begin
                    if convert(datetime,''' + convert(varchar(30), @service_start_time) + ''') < convert(datetime, ''' + convert(varchar(30), @job_last_success_time) + ''')
                        begin
                            --print ''1''
                            update TableAccessHistory
                            set insert_count = @insert_count,
                            update_count = @update_count,
                            delete_count = @delete_count,
                            select_count = @select_count
                            where database_id = ' + convert(varchar(5),@database_id) + ' and object_id = @object_id
                        end
                    else
                        begin
                            --print ''2''
                            update TableAccessHistory
                            set insert_count = insert_count + @insert_count,
                            update_count = update_count + @update_count,
                            delete_count = delete_count + @delete_count,
                            select_count = select_count + @select_count
                            where database_id = ' + convert(varchar(5),@database_id) + ' and object_id = @object_id
                        end
                end
            else
                    insert into TableAccessHistory values
                    (' + convert(varchar(5),@database_id) + ', ''' + @database_name + ''', @object_id, @object_name, 
                     @insert_count, @update_count, @delete_count, @select_count)


           FETCH NEXT FROM TableAccessHistory_cursor
           INTO @object_id, @object_name, @insert_count, @update_count, @delete_count, @select_count
        END

        CLOSE TableAccessHistory_cursor
        DEALLOCATE TableAccessHistory_cursor
        '
    --print @sql
    exec (@sql)

    FETCH NEXT FROM dbname_cursor
    INTO @database_name, @database_id
END

CLOSE dbname_cursor
DEALLOCATE dbname_cursor

Result

You can identify the busy tables using the script below

select database_name, object_name, 
sum(insert_count+ update_count+delete_count+select_count) as access_hit
from SQLAdmin.dbo.TableAccessHistory
group by database_name, object_name
order by database_name asc, access_hit desc

After identifying those busy tables

If there is no clustered index on the table, create a clustered index on different File Group which has different files

If there is a clustered index on the table, REBUILD index on different File Group which has different files

And monitor the performance

How to revoke CONNECT permission from the Guest user in a database

use [AdventureWorks2008]
go
REVOKE CONNECT FROM GUEST

How to grant CONNECT permission to the Guest user in a database

use [AdventureWorks2008]
go
GRANT CONNECT TO GUEST

Replication Agent Security Model

https://msdn.microsoft.com/en-us/library/ms151868.aspx

Merge Agent for a pull subscription

The Windows account under which the agent runs is used when it makes connections to the Subscriber. This account must at minimum be a member of the db_owner fixed database role in the subscription database.

The account that is used to connect to the Publisher and Distributor must:

• Be a member of the PAL.
• Be a login associated with a user in the publication database.
• Be a login associated with a user in the distribution database. The user can be the Guest user.
• Have read permissions on the snapshot share.

The script below can be used to identify the Guest Users in all databases

--guest users
USE master;
GO
if not exists (select name from sys.databases where name = 'SQLAdmin')
    create database SQLAdmin
go
declare @name sysname 
declare @cmd varchar(4000)
DECLARE databases_cursor CURSOR FOR
SELECT name FROM sys.databases where state in (0)
order by name

create table #guest_users (
    database_name sysname, principal_name sysname, permission_name nvarchar(128), state_desc nvarchar(6)
)

OPEN databases_cursor;

FETCH NEXT FROM databases_cursor into @name;

WHILE @@FETCH_STATUS = 0
BEGIN
set @cmd = 'use ' + @name + '; 
insert into #guest_users
SELECT ''' + @name + ''' as database_name, name, permission_name, state_desc
 FROM sys.database_principals dpr
 INNER JOIN sys.database_permissions dpe
 ON dpr.principal_id = dpe.grantee_principal_id
 WHERE name = ''guest'' AND permission_name = ''CONNECT''
' 
exec (@cmd)
   FETCH NEXT FROM databases_cursor into @name;
END

select * from #guest_users order by database_name asc

drop table #guest_users

CLOSE databases_cursor;
DEALLOCATE databases_cursor;
GO

It is recommended to keep 1024MB for data files and 256MB for log files. Log files are really important because when the transaction occurs, it is written to memory and log files first. Log files are very important because of point-in-time recovery and there shouldn’t be any latency on the log file.

The query below can be used to check the next auto growth size for each data and log files. And if is_percent_growth column equals to “Yes”, File Growth option can be selected as “In Megabytes”

--auto growth percentage for data and log files
select DB_NAME(files.database_id) database_name, files.name logical_name, 
CONVERT (numeric (15,2) , (convert(numeric, size) * 8192)/1048576) [file_size (MB)],
[next_auto_growth_size (MB)] = case is_percent_growth
    when 1 then CONVERT(numeric(18,2), (((convert(numeric, size)*growth)/100)*8)/1024)
    when 0 then CONVERT(numeric(18,2), (convert(numeric, growth)*8)/1024)
end,
is_read_only = case is_read_only 
    when 1 then 'Yes'
    when 0 then 'No'
end,    
is_percent_growth = case is_percent_growth 
    when 1 then 'Yes'
    when 0 then 'No'
end, 
physical_name
from sys.master_files files
where files.type in (0,1)
and files.growth != 0

Open SQL Server Management Studio and run of the following queries.

First Query:

SELECT SERVERPROPERTY('productversion'),
       SERVERPROPERTY('productlevel'),
       SERVERPROPERTY('edition')

Second Query:

SELECT @@VERSION

To view all versions click the link [more]

Builds SQL Server 2012

Builds SQL Server 2008 R2

Builds SQL Server 2008

Builds SQL Server 2005

After we have configured the Distributed Replay as explained in the previous post, we can use the Distributed Replay environment to create the load on the SQL Server.

I posted this article to Microsoft SQL PFE site and it can be found from there

https://blogs.msdn.com/b/mspfe/archive/2012/11/14/using-distributed-replay-to-load-test-your-sql-server-part-2.aspx

Have you decided to migrate SQL Server to a newer version? Would you like to test the load on the new SQL Server? How can you create a load on the new SQL Server? Do you need to develop a multi threaded application to simulate load on SQL Server? Using a new feature of SQL Server 2012, called Distributed Replay, it is so simple to simulate load on a SQL Server for testing purposes.

I posted this article to Microsoft SQL PFE site and it can be found from there

https://blogs.msdn.com/b/mspfe/archive/2012/11/08/using-distributed-replay-to-load-test-your-sql-server-part-1.aspx