The internet is swimming with examples of really inefficient TSQL code. Just because everyone does it that way doesn’t make it optimal. I’ve already blogged about the issues you will find with NOLOCK hints, this post show that using system functions correctly can result in your Data Definition Language (DDL) running 3 times faster.

Example 1: Faster way to check existence of a table (object)

Common but Slowest

If you Right Click on a Table in SQL”s Management Studio you may enjoy the benefits of “Script Table As…”. Among other options it lets you create a script to “DROP AND CREATE”. It will generate a script similar to

-- Common Style of DML Check, then Drop & Create a table.
IF NOT EXISTS ( SELECT * 
                FROM sys.objects
                WHERE object_id = OBJECT_ID(N'dbo.MyTable')
                AND type IN (N'U')
               ) 
CREATE TABLE dbo.MyTable(Rep_ID INT Null) 

For this discussion I’d draw your attention to the TSQL that searches for existence of the user table ie:

-- Popular Approach
    SELECT * 
    FROM sys.objects 
    WHERE object_id = OBJECT_ID(N'dbo.MyTable') AND type IN (N'U')

Better use of System function & faster

A quick read of SQL’s Books Online will make you aware that the SQL Function OBJECT_ID() has a 2nd parameter that lets you specify the type of the object you are searching for. If you use it, you will find that your query runs twice as fast. Why? There was no need to test the condition on the Type column separately.

-- Better: Use the 2nd Parameter instead of an AND   - Same results but neater
    SELECT * 
    FROM sys.objects 
    WHERE object_id = OBJECT_ID(N'dbo.MyTable', N'U')

You may have read that “SELECT *” was inefficient & you should always specify the columns you need. That is very true, in most circumstances. The exception to the rule is when the select is part of the “EXISTS” clause as it is here. In this case, the SQL Optimiser is smart enough to know that it only needs to know if a row is returned & doesn’t care about any of the columns in the select list. So performance in the following alternative is exactly the same. Use whatever you prefer, either alternative is slower than the last alternative.  

-- Better: Use the 2nd Parameter instead of an AND   - Same speed as SELECT *
    SELECT 1 
    FROM sys.objects 
    WHERE object_id = OBJECT_ID(N'dbo.MyTable', N'U')

Fastest

As the OBJECT_ID() function can be used in all TSQL, there is no need to wrap it up in a SELECT statement. It returns NULL if the object is not found so just compare its output to NULL. e.g:

-- Faster: just use the function & 2nd parameter - 3 Times faster
    IF OBJECT_ID(N'dbo.MyTable', N'U') IS NOT NULL
        CREATE TABLE dbo.MyTable(Rep_ID INT Null)
--Only save millisecs here but same applies with any IF EXISTS which could be way more expensive. (Use an index instead of table scan etc.)

Not only is this is 3 times faster. It means less code, so less chance of making errors. Most importantly it avoids the bad (& unsupported) practice if querying the system tables directly (in this case sys.objects).

In case you haven’t heard, you should be using the Informational Schema Views & System Functions instead of the system tables because Microsoft may change the system tables between releases which will break all your code. Yes I know “everyone” does it. Yes I am a hypocrite as many of my tuning tools are very vulnerable to System Table changes. But Microsoft was saying this from Version 1.1 (in 1990) they never changed & DBA’s got blasé about it. But 15 years later with the release of SQL 2005 we found sysobjects became sys.objects. While they did a great job of backward compatibility it does highlight the risk you take using System Tables in the heart of your application.

Post thought

You might wonder “why didn’t the SQL Server Management Studio doesn’t produce a better example of code. Surely they know how”. The tools team were likely to have a different goal to you. They were writing a context menu function that worked anywhere in the Object Explorer tree; Tables, Views, Functions Indexes etc. While many of these objects can be handled by OBJECT_ID(), some can’t. So it is more expedient to write a generic routine what worked everywhere than to produce optimal code samples with heaps of special case code. Especially when in this case the % improvement my be huge, but the actual time saved is still measured in millisecs, so who cares.

This blog is about the concept of system functions. And for some queries it makes a huge difference.

Example 2: Faster way find indexes on User Tables.

Perhaps you want to see all the Indexes in your system but don’t want the results cluttered with indexes on the System Catalog. Unfortunately when you query the Sys.Indexes table it doesn’t tell you if the Index is on a User Table, a System Table or an Indexed View. So you find yourself frequently joining on the Sys.Objects table & filtering with Type = N’U’ eg:

-- 2 table join to filter on User Tables.
SELECT  OBJECT_SCHEMA_NAME(I.object_id) AS [Schema],
        OBJECT_NAME(I.object_id) AS [Table],
        I.* 
FROM sys.indexes AS I
JOIN sys.objects AS O ON O.object_id = I.object_id
WHERE O.type = 'U'

Often I’ve found that I needed to think about indexes on Indexes views too. So it was better just to filter out all system objects than to limit my query to User Tables. So the query changes to :-

-- 2 table join to filter out System Objects.
SELECT  OBJECT_SCHEMA_NAME(I.object_id) AS [Schema],
        OBJECT_NAME(I.object_id) AS [Table],
        I.* 
FROM sys.indexes AS I
JOIN sys.objects AS O ON O.object_id = I.object_id
WHERE O.schema_id <> 4

But as I only wanted to filter the System Objects out of the Sys.Indexes Table I didn’t need the join with sys.objects at all. As I have the Object_ID I could use OBJECT_ID() or OBJECT_SCHEMA_NAME() to remove rows I didn’t like. eg:

-- Single table scan, Filtered by System Function, nearly 4 times faster.
SELECT  OBJECT_SCHEMA_NAME(I.object_id) AS [Schema],
        OBJECT_NAME(I.object_id) AS [Table],
        I.* 
FROM sys.indexes AS I
WHERE OBJECT_SCHEMA_NAME(I.object_id) <> 'sys'
go

Its nearly 4 times faster, & less code to write. Of course if you still want to filter for User Tables the OBJECT_ID() function will work as I showed in Example 1.

This approach is especially handy when working with Dynamic Management Views which I’ll show in a future post.

Hope this encourages you to read up on some of the other SQL System functions.

Dave

Thought for the Post

OMG (Oh! My God) is an expression that seems to be everywhere these days. But I wonder if I could read Arabic, do they type OMA (Oh! My Allah) or is the term God sufficiently generic? Will we start seeing OMB for those who follow Buddha. OMJ (Oh! My Jehovah), OMI (Oh! My Infinity) for the Scientologists. etc

If someone says “Oh! My” are they an Atheist or just too lazy to mention the omnipotent super-being of their choice.

I assume that an Antagonist would type OYG (Oh! your God!) or perhaps OME (Oh! My Ex-god).

It was so much easier when people used four letter words to express themselves.

So if you ever find yourself texting someone who you can see from where you are sitting. Or you say “LOL” instead of actually laughing. Be aware that most of the people around you including those your own age, probably think you are a <insert expletive here>.

Perhaps one of the less useful of the pre-scans it is still handy. It highlights the use of depreciated datatypes, new feature usage & uncommon design. So it really doesn't matter how many INT's or VARCHAR's there are. You are looking to see if the less common datatypes are in use.

It is background info & will only hold your attention for 1-3 mins. But may alert you to something out of the ordinary. Remember to run it against the Model database first. This it the template for all User databases & holds the System Schema. So if model holds 2 sql_variant cols then sql_variant is only interesting if there is more than 2 in your database.

Some of the things I look for are :-

NTEXT, TEXT & IMAGE fields Eventually these will need to converted to NVARCHAR(MAX), VARCHAR(MAX) & VARBINARY(MAX) or they will block a future upgrade. Also they lets you use TSQL String functions & can remove some messy special code. Might be an opportunity to speed up searches by using Full Text, could also consider FileStream to improve performance & streamline operations.

User Defined Types I like these as they reduce the potential for error by ensuring data type consistency throughout the schema. Especially useful for key columns & commonly used attributes like Surname, Phone & Address fields. But you need to get them right as changing them is a nightmare. 

SQL_VARIANT This sets off an alarm bell as usually it means some attempt at Object Relational Design or a central Application Parameter table. Both can cause huge locking issues & overhead, so it is often worthwhile to examine closely. Note: The system schema in SQL2008 has 2, ignore these.

UNIQUEIDENTIFIER  I’ve never been a huge fan of this 16 byte datatype. But it is handy for storing GUIDS. I have seen people use these for primary keys, yet have no code to deal with the very small but real possibility of a duplicate primary key. Many use them to resolve the issues of replicated systems, I prefer a composite key with a site ID, but can see merit in the GUID approach. Others use it to hibernate Objects in their code into the database. Whatever the reason, examine it closely as often there are more efficient alternatives.

TIMESTAMP Often this is used to implement Optimistic Concurrency Control which is basically a way to free up record locks by rolling your own transaction. I feel this approach is inferior to Optimistic Concurrency Control by Value. Where instead of looking to see if the timestamp has changed since you read the record, you look to see if any of the values you want to change are different. This increased concurrency as it permits multiple people to simultaneously update different fields in the same row without collision. For that reason I dislike seeing timestamp cols. Note: TIMESTAMP is an 8 byte binary counter which increments each time the row is modified, it has nothing to do with the actual time.

FLOAT is rarely useful in business systems due to its lack of precision. Check they aren’t storing financial data in float types. For most applications Financial Transactions should be stored in Money (or SmallMoney) it holds up to 4 decimal digits ie 1/100 cent). However Financial Institutions, Interest Rate calculations or Overseas currency amounts should consider Decimal. FOREX rates are quoted in fractional PIPs (5 decimal digits) so don’t use the default storage for Decimal. Reduce the bytes to consume the precision you actually need.

XML Extensive use of XML is neither good nor bad. It is bad if you are trying to put your entire database into 1 XML field. But generally it is underused by DBA’s & can be a very powerful, efficient solution when used correctly.

GEOGRAPHY, GEOMETRY There are spatial aspects to this database. 

HIERARCHYID Could be good. Many schemas have recursive structures which often cause pain. If they have converted it to HierarchyID may be 1 less problem to solve. That said, HierarchyID’s aren’t the magical solution I’d hoped they would be. You do need to manage their updates carefully which can be a source of data corruption. 

BINARY & BIT Bitmasks & Flags suggest a more sophisticated algorithm most often it is a good thing. It is worth finding out what they are doing. Rarely will this be a bottleneck, but large binary fields can be a workaround for IMAGE, so FileStream might be relevant.

SMALLINT, TINYINT, SMALLMONEY, SMALLDATETIME tell me the DBA was concerned with efficient storage & actually looked at range of values contained in each field. Boosts your confidence in this design.

DATE, DATETIME2 or some of the new Date types introduced in SQL 2008. Suggest the schema was enhanced or designed recently. It is less likely to have the common bottlenecks caused by trimming Time from Dates. But also raises a flag if the data is being consumed by SSIS or SSAS. These apps will process the higher precision of DATETIME2 but most of their TIME functions do not recognise DATETIME2 as a valid input format. This can be a pain. Similarly all the ParallelPeriod style functions in SSAS & PowerPivot think Date is the lowest granularity. So you need to do your own thing if you want to analyse in Hourly, 5 min, Min or Second.

NVARCHAR / NCHAR vs VARCHAR / CHAR Neither is correct, but useful to see if they are used consistently. Joins, Filters & Comparisons between ASCII & Unicode types will cause implicit conversions. This may result in an an Index Scan rather than an Index Seek. It can be a pain dealing with Sorting & Collation sequences. It slows development as UI interface may require different datatypes to hold the double byte characters & often the programmers don’t even think about it.  It also may cause problems if your company expands into overseas markets & parts of your application can deal with different characters & other parts break. If your entire database is Varchar / Char then you gain from reduced storage, reduced IO & more data remaining in cache. But you lose language flexibility. Either way make a decision & be consistent. The exception here is tables hold state internal to your application. They could be char while the rest of the system is NChar as they will never hold data inputed by the end user.

Sample of TSQL for this type of Pre-Scan

/* =========================================

Pre-Scan: DataTypes of Columns

Notes: What to look for :-

Depreicated types: Image, Text

New types: XML, Spatial (Geometry/Geography), CLR, HeirarchyID, DateTime+

Challenging types: uniqueidentifier (GUID), sql_variant, binary (what for)

User-Defined Types: Are they extensive?

-- =======================================*/

;WITH cols AS (

SELECT T.name AS [DataType], C.*

FROM sys.types AS T

JOIN sys.columns AS C ON C.system_type_id = T.system_type_id

)

SELECT COUNT(*) AS Count, DataType

--, name,

--MIN(datatype)

FROM cols

GROUP BY DataType

-- HAVING COUNT(*) > 10

ORDER BY 1 DESC

Sample of Output

In the above example I’m comparing the distribution of datatypes in a few production databases.

The Model database is a set of tables included in every database you create. I’ve highlighted in Purple some of the datatypes of interest. In the other databases, if they are completely purple then they aren’t using that type in any of the user tables. Where I’ve highlighted they numbers then you need to subtract from model to calculate the real number.

Legend

Red: Depreciated features Text & Image.
The Orange bands draw your attention to the Float vs Money discussion.
The Dark Green is New feature usage.
Light Green: they considered the range of values the column will hold & didn’t blindly map everything to INT.
Blues: ASCII vs Unicode chars.
Yellow: Explore how these are being used.

Quick Impressions:

Debtrak: Mainly ASCII based app. Need to check the use of Float & financial info. What are they using Text & Image for? (Probably to hold Scanned Documents)

 Oh: No Money Types, how is float being used. Mostly NChar but not all, why are the Char types different. What are they using GUID’s for? (possibly as keys for Scanned Images). Lots of Image fields. No Money, Is this CRM or are they using Decimal instead?

Spinn: UID’s are huge, bet it is used for all PK & FK’s, bummer. What are the Bit fields for?  No Date / Time fields probably written pre-2008. More Char than INT fields, probably CRM or text based system, clearly UID’s could be considered INT’s for this comparison.

ReportServer: Database used by SSRS. I leave it to you to check out why they are using GUID’s, Binary & Image fields. You can bet there will be some redesign when they ship the next SQL release.

AssetTrax: Uses Spatial Fields, Unicode & not much else. Probably a central store for a Mobile GPS system, perhaps it is an optional module for a bigger system.

SRDDBA: No Unicode, mainly Numbers, Dates, Characters. The 17 Numeric types could hold financials but I’d suspect this to be non-financial app, perhaps reference data.

I hope you find this useful, Let me know if you want me to publish more scans.

Dave

Recently I tore my hair out trying to create a great end-user experience for my Excel 2010 users when they access SharePoint.

Eventually I found a way to get it to work. This is not a perfect post. I’m far from an expert in SharePoint & I’m hoping a sharepoint expert will comment & show me a slicker way. But given I found nothing on the net, I hope this may assist you.

The Goal.

A Sharepoint person should publish a set of connections to data sources & provide permissions to all who need them. All end users automatically see them. Some time later as IT moves the databases & data to new servers, they update the sharepoint data connection libraries & the hundreds of business users don’t notice & don’t care.

What I wanted the Business user to experience

Step 1: Open Excel & click on Existing Connections

Step 2: See something in the “Connection files on the Network”. Double click on one of the Sharepoint Data Connection Libraries listed.

Step 3: Select the Data Source they want.

Step 4: Use it in Excel.

How to do it – Sharepoint Site Admin.

Step 1: Open Sharepoint 2010 Central Administration app & click Manage Service Applications

Step 2: Check the Excel Services Application has been installed & started.
(As this process is reasonably well documented elsewhere I’ll not repeat it here)

Step 3: Click User Profile Service Application

Step 4: Click “Publish links to Office Client Applications”
This bit took me ages to find, as it was in a different location in Sharepoint 2007 & all the 2010 doc’s I found still had Sharepoint 2007 pictures in it.

Step 5: Click New Link

Step 6: Paste in the URL to the Sharepoint Data Connection Library that you’ve created earlier.
NB: If you don’t have one, now would be a very good time to
a) Create a Data Connection Library (DCL),
b) Create a Data Connection
c) Upload it into Sharepoint &
d) Give permissions & Approve both the Library & the Data Connection.
(As these steps are also well doc’ed I’ll not repeat them here. )

How to do it – Excel Client Admin.

Note: This is the part I think it insane. It works, it is in the official documentation for older versions of Sharepoint. But I suspect there is much better way to do this. Some have told me “Yes we do that, but we push it out via group policy”. I can’t imaging anyone in the Sharepoint team architecting this & deciding it was the recommended way to do this. So there is probably a script or button somewhere but I’ve not found it.

Step 1. Open up RegEdit.Exe.
(Note: You are about to edit the Windows Registry. Good idea to back it up first. If you stuff it up you may create serious issues that may prevent your computer from starting & cause you to reinstall windows.)

Step 2:  Move to Key HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Common\Server Links\Published

Step 3: Right Click on Published & add a new String Key.
Call it anything you like, this will be the name that appears in the Excel “Existing Connections” Dialog. As you can see above I called mine “Secure Data Connections” as that was the name of the Sharepoint DCL I created & management liked the idea of it being secure.

Step 4: Set the value to the key to URL to point to the Sharepoint Data Connection Library you created.

Step 5: Enjoy.

Note: The environment I set this up was had a crazy design for the Windows Authentication in their Test domain. So I’m unsure if some of the Trust & Security warnings I was receiving from Excel was something others would see. If you also get “Click here to trust this data” try looking in the Excel Trust Centre & Trusted Locations / Documents areas. 

Like I said at the start. I usually prefer to present well researched material & have a “this is the way to do it” blog. But in this case the goal is to help you break thru a few of the issues I had & maybe move you closer to a solution.

Feel free to add comments if you have more insight on this process.

Thanks Dave.

At the very start of a performance tuning exercise I do a Quick Appraisal to assess what I'm dealing with.  This is one of a number of “information collection” scripts I run.

This post shows (a) Code & (b) some ideas on insight you can get from it.

What are PreScans?

When someone has a performance problem they often expect you to "know” everything about their system & pinpoint the problems almost before you’ve even seen their system. To bring me up to speed fast, I run a series of trivial light weight queries that give me some statistics & a rough idea of what I’m dealing with. I run these Database scans prior to starting a job, hence the term “Pre-Scan” or “Quick Appraisal” phase.

As I mentioned my “Quick Appraisal” set of queries in a recent presentation to the Sydney SQL Server User Group. I thought I’d post a few on my blog.  This is one of them. Note, this is not an in-depth anything, often a 5 sec glance at the results is sufficient. But after looking thru 10-15 of these simple queries, it is amazing how accurate your “first impressions” will be for speculating on the potential problem areas of a system.

Quick Appraisal

There are many different styles of database design.  Looking at the number of Objects, the types of Objects & ratios between them, often permits you to make a quick assessment on both the complexity of the database as well as the mindset of those who created it. Knowing this "mindset" permits you assess the probability of where their poor coding practices might lie. 

Is it unfair to assume there are suboptimal coding practices in the database? No, no-one  will every pay you to tune a system that they think is working well. There is bound to be a bottleneck somewhere. Most organisations will have already tried “throw more H/W at it”  ;-)

How does it work?

The above table shows output from 7 different databases. Debtrak, O4, Spinn are production systems used by very large companies. SSRS, MSDB, Master are SQL System Databases. AW08 is the SQL Sample Adventureworks database.

You will note, I’ve created a few extra rows to look at the Primary keys, Foreign Keys & Stored Procs as a percentage of overall tables.

Debtrak: High Use of Stored Procedures (Could be a good thing, unless they are full of cursors). Fairly high number of tables. 1/3 of the tables have no primary key. Almost no foreign keys. But heaps of Stored Procedures, perhaps they do data integrity checks there. If not, could be heaps of duplicate rows & dirty data. Using functions so schema has been developed for SQL2005+ or modified to use newer features introduced in SQL 2005. No CLR, Extended SP’s, or Service Broker, generally DBA’s who implement these in their design are highly skilled in both DBA & Dev. So less likely to find any easy wins.

O4: Lots of Tables & views. Need to check on the PK use, only 8% don’t have them. Very few FK’s so Declarative Referential Integrity (DRI) is not used much, probably done with rules. Lots of Rules, database likely to created on pre-SQL2000 & upgraded. Watch for depreciated items, especially Text & image fields & functions that trim the time off dates. Quite a few triggers, need to check how complex they are.

Spinn: Lots of FK’s (so extremely heavy use of DRI = good), No views or Stored procs. Probably created by a programmer, where all data integrity would be handled by their application. Their app will generate TSQL. This would be a nightmare to tune unless they have Unit & Load tests created for their code. At best you will be creating indexes or playing with H/W. I’d expect to find a lot of dirty data as very few applications do a good job at checking anything other than the datatypes & a drop down list of values. The relations between the fields in a row are often not checked at all eg: This results in Pregnant males, Customers born in the future, Orders delivered before they were placed & 32 year old Primary School students. At least the relationships between rows should be good.

SSRS (SQL Reporting Services): Good use of Stored Procs & FK’s. Very few tables. Not a lot to look at, easy to understand.

AdventureWorks 2008: Modest number of tables. DRI is brilliant, data likely to be of good quality, especially when you look at the number of Rules & Check constraints. Few stored procs, so app likely to be passing TSQL. Rules also suggest an older design, so look for depreciated constructs.

Master: The numbers suggest there are many things wrong with this database. The only good news is that a tiny number of tables suggest it is not used by more than 1 app. Of course in this case we know that since SQL2005 the Master DB was split in two. The other part is the resource database, which is not shown here. So it makes little sense drawing any conclusion about it other than “Hey it is a small number of tables, should be easy to figure out what they do.”

Sample of TSQL for this type of Pre-Scan

Version 1.  - Faster, Sort in Order you want.

Only shows objects that exist. It will also display new Objects Types if they are introduced in future versions of SQL

USE O4Jul09

go

;WITH ObjectTypes AS (

SELECT [type], type_desc, COUNT(*) AS [Count]

FROM sys.objects

GROUP BY [type], type_desc

)

SELECT *

FROM ObjectTypes

ORDER BY CASE [type]

WHEN 'SN' THEN 10 -- Synonym

WHEN 'V' THEN 11 -- View

WHEN 'U' THEN 20 -- User Table

WHEN 'PK' THEN 21 -- PRIMARY KEY constraint

WHEN 'F' THEN 22 -- FOREIGN KEY constraint

WHEN 'UQ' THEN 23 -- UNIQUE constraint

WHEN 'C' THEN 24 -- CHECK constraint

WHEN 'D' THEN 25 -- DEFAULT (constraint or stand-alone)

WHEN 'R' THEN 26 -- Rule (old-style, stand-alone)

WHEN 'TR' THEN 30 -- SQL Trigger

WHEN 'SQ' THEN 31 -- Service queue

WHEN 'P' THEN 40 -- SQL Stored Procedure

WHEN 'TF' THEN 41 -- SQL Table-valued-function

WHEN 'IF' THEN 42 -- SQL Inline Table-valued function

WHEN 'FN' THEN 43 -- SQL Scalar function

WHEN 'X' THEN 50 -- Extended stored procedure

WHEN 'RF' THEN 51 -- Replication-filter-procedure

WHEN 'PC' THEN 60 -- Assembly (CLR) stored-procedure

WHEN 'FT' THEN 61 -- Assembly (CLR) table-valued function

WHEN 'FS' THEN 62 -- Assembly (CLR) scalar-function

WHEN 'AF' THEN 63 -- Aggregate function (CLR)

WHEN 'TA' THEN 64 -- Assembly (CLR) DML trigger

WHEN 'PG' THEN 80 -- Plan guide

WHEN 'IT' THEN 90 -- Internal table

WHEN 'S' THEN 91 -- System table

WHEN 'TT' THEN 92 -- TYPE_TABLE

ELSE 100

END

go

Version 2.  - Slightly Slower.  Highlights which object types the schema doesn't use. 

Only shows objects that exist

USE O4Jul09

go

;WITH ObjectTypes AS (

SELECT [type], type_desc, COUNT(*) AS [Count]

FROM sys.objects

GROUP BY [type], type_desc

)

, ObjectGroups AS (

SELECT 'SN' AS [type] , 10 AS SortOrder, 'Synonym' AS type_desc, 'Any Redirection to be aware of?' AS Action UNION ALL

SELECT 'V' , 11 , 'View', 'Does it make extensive use of Views?' UNION ALL

SELECT NULL, 19, '.', '.' UNION ALL

--< User Tables >--

SELECT 'U' , 20, 'User TABLE', 'Amount of tables' UNION ALL

SELECT 'PK', 21, 'PRIMARY KEY constraint', '% of Tables with PK''s defined' UNION ALL

SELECT 'F' , 22, 'FOREIGN KEY constraint', 'Are they using DRI?' UNION ALL

SELECT 'UQ', 23, 'UNIQUE constraint', 'Check M:M join tables are correctly protected' UNION ALL

SELECT 'C' , 24, 'CHECK constraint', 'Are Data values protected by schema?' UNION ALL

SELECT 'D' , 25, 'DEFAULT (constraint or stand-alone)', 'Do they rely on Defaults or SPs or nothing' UNION ALL

SELECT 'R' , 25, 'RULE (old-style, stand-alone)', 'Are they using Depreciated Checks' UNION ALL

SELECT NULL, 29, '.', '.' UNION ALL

--

SELECT 'TR', 30, 'TRIGGER', 'Look at ratio vs User Tables. Heavy / Light use of Triggers?' UNION ALL

--< SQL Broker >--

SELECT 'SQ', 31, 'Service QUEUE', 'Are they using Async "Triggers" or SQL Broker?' UNION ALL

SELECT NULL, 39, '.', '.' UNION ALL

--< Procedural Code >--

SELECT 'P' , 40, 'Stored Procedure', 'Number & Ratio to User Tables' UNION ALL

SELECT 'TF', 41, 'Table-Valued-function', 'Are they being used?' UNION ALL

SELECT 'IF', 42, 'Inline Table-Valued function', 'Are they being used?' UNION ALL

SELECT 'FN', 43, 'Scalar function', 'Are they being used?' UNION ALL

SELECT NULL, 49, '.', '.' UNION ALL

--

SELECT 'X', 50, 'Extended stored procedure', 'Need to think about security' UNION ALL

SELECT 'RF', 51, 'Replication-filter-procedure', 'Will need to think about external connections' UNION ALL

SELECT NULL, 59, '.', '.' UNION ALL

--< CLR Use >--

SELECT 'PC' , 60, 'Assembly (CLR) stored-procedure', 'Are they useing CLR?' UNION ALL

SELECT 'FT', 61, 'Assembly (CLR) table-valued function', 'Are they useing CLR?' UNION ALL

SELECT 'FS', 62, 'Assembly (CLR) scalar-function', 'Are they useing CLR?' UNION ALL

SELECT 'AF', 63, 'Aggregate function (CLR)', 'Are they useing CLR?' UNION ALL

SELECT 'TA', 64, 'Assembly (CLR) DML trigger', 'Are they useing CLR?' UNION ALL

SELECT NULL, 69, '.', '.' UNION ALL

SELECT NULL, 79, '.', '.' UNION ALL

    --< Other >--

SELECT 'PG', 80, 'Plan guide', 'Some serious compatibility work has been done on this DB' UNION ALL

SELECT NULL, 89, '.', '.' UNION ALL

--

SELECT 'IT', 90, 'INTERNAL_TABLE', '-' UNION ALL

SELECT 'S' , 91, 'SYSTEM_TABLE', '-' UNION ALL

SELECT 'TT', 92, 'TYPE_TABLE', '-'

)

SELECT COALESCE(G.[type], T.[type], '.') AS Type, COALESCE(G.type_desc, T.type_desc) AS [Type Description], ISNULL(CAST(T.[Count] AS VARCHAR(8)), '.') AS Quantity, G.Action

FROM ObjectGroups AS G

FULL JOIN ObjectTypes AS T ON T.[type] = G.[type]

ORDER BY SortOrder

Examples of insight you can gain from this information.

User Tables

 The more they have the more complex the schema. Obvious huh?

Primary Keys

 Generally every table should have one. If they don't there should be a really good reason. Find out what it is. More often I hear "Oh! Really", "Yer well I was in a hurry that day & was going to get back to it." or "That was when I was new" or “What’s a Primary key?”.
Sometimes are good reasons eg: these are “Staging Tables” where data is bulk loaded into them, manipulated with Table scans & moved on. This supports an Extract, Load & Transform (ELT) style of data movement. (NB: Most people still refer to this as Extract, Transform & Load (ETL) processing). You may think that this activity may be better handled in TempDB, but sometimes a complex series of batch load processes need; i)a way to pass data from one to the other or ii) die & be restarted without doing everything from scratch.

Foreign Keys

Are they using Declarative Referential Integrity (DRI)? If not, Why not?

Perhaps this is a Data warehouse & the relationships were validated in the staging database or the ETL process.

Perhaps they are relying on Stored Procs & Triggers - fragile & possibly slow.

You should ensure that FK Constraints have a useful index to enforce them. Without it, queries can deadlock on themselves &/or cause table scans even on single row modifications.  Note: That does not mean you should automatically create a Non-Clustered index on just that specific column. Perhaps adding another column as a composite key or as an Included field will  permit the index to cover the needs of a commonly run query as well.

Unique Constraints

• These may be unnecessary if the Primary Key fields were selected correctly but if there are none, it raises an alarm bell.
  • Look for tables designed to support Many  to Many Joins. There is a high probability they've not been indexed / constrained correctly. Resulting in duplicate rows & highly inefficient queries.
  • Most databases I've seen suffer from duplicate rows. Especially ones with Web front ends that don't prevent the users from hitting OK multiple times when the interface fails to respond quickly. The only difference in the rows is the Auto-generated Primary key.  A Unique constraint can prevent that but effectively you are creating the entire table again in the index. So use with care. It is probably better to make the two foreign Key columns a composite Primary key & remove the Identity column you are currently using as a primary key.

Check Constraints

• The use of Check Constraints tends to suggest the database was designed by a DBA rather than a programmer. Programmers tend to validate the input in Stored Procedures &/or their application.  (NB: I did not say one approach was always better than the other. )
• Check Constraints are vital to get all the performance benefits from a Partitioned table.

Default Constraints

• Very few or zero Defaults, suggests they are setting these values in Stored Proc's or in the App code.
• Be alert, DB's with high numbers of Default, often use the poor practise of Insert a row & then immediately Select it to determine values of the Defaults & Identity fields.

This often results in an index search to Update/Insert a row & another to select it. Also resulting in 2 network round trips, so more network traffic.

If you need to return the values generated by your Insert ie: set by Identity, Default,  & triggers.  Use an OUTPUT Clause on your insert, instead of re-Selecting the rows you've just inserted. 

-- < Inefficient 2 Separate queries, 2 Network round trips >--

INSERT INTO InsertOutput(name) VALUES('Dave11')   

go

SELECT * FROM InsertOutput WHERE name = 'Dave11'

go

-- < More efficient 1 query, 1 Network trip, ~33+% faster >--

INSERT INTO InsertOutput(name)

OUTPUT inserted.ID, inserted.Name, inserted.DateIn, inserted.UserName

VALUES ('Dave21')

go

Rules

• If you find any, make time to rewrite them as CHECK Constraints.
• I really liked the old style Rules, Defaults & User Defined Types as they could be created once & use in many tables. Thus ensuring consistency throughout the design. Unfortunately they are not ANSI compliant & are depreciated. Also you needed to be sure of your decisions when you created them as changing them caused a maintenance nightmare. 

Triggers

• Debugging Triggers &/or assessing their impact on performance can be very tricky. Knowing that Triggers exist in the design helps you factor that into your plans. Be aware that neither sp_depends nor  the dependency DMV's do a perfect job at looking at objects called by triggers.
• Look for  INSTEAD OF triggers they may completely change the impact of what you thought DML statements would do to that table.
• Where Triggers are used to maintain other entities see if you can rewrite it using Queues. Eg: If placing a Customer order causes a trigger to fire that initiates a workflow to restock your warehouse if the stock level is low. This activity is often better handled asynchronously using SQL broker. One benefit is that your end user will get a more predictable response time when their Order placement transaction is not suddenly burdened with the overhead of restocking the warehouse.

Service Queues

• If they are not being used, it is likely that they could be beneficial .

Stored Procedures

• Look at the number compared to User Tables.
• If there are 3 SP's for each table it is likely a poor design. An Insert, an Update & a Delete  stored procedure, is unlikely to encapsulate logic very well.
• If there are very few SP's it is likely to become a maintenance nightmare. Without the SP's to abstract the Database schema from the application, all the queries will be created & generated in code. Making changes to the schema will be almost impossible to test. (Unless they have 100% code coverage in their Unit Tests & are not using Mocking to prevent the tests actually calling the DB. This is very rare)

SQL Functions

When used carefully these can improve code reuse & improve performance. Most often these create big perf issues. Watch for :-

• Calling a trivial function from within a WHERE clause, if may prevent you using an index & result in a table scan.
• Calling a Function from a Select list. It gets called once for each row in the resultset, which could be a huge perf hit. Often it is better to rewrite the Function as part of the Join clause.

Extended Stored Procedures

• Check security is done correctly.
• Check if they are necessary. Perhaps it is only there to support 1 batch process which might be better done via SQLCMD, Powershell or SSIS Workflow.

Replication Filter Procedures

This gives you a hint that you need to think about the replication system & interop with other SQL databases.

CLR Objects

• Lets you know if they are being used.
• They can be extremely powerful especially to enhance string manipulation.
• These are not used by many people. Typically they are only suggested by DBA's with an extensive knowledge of the entire feature set of SQL Server & fairly good programming skills. So generally the DB design will be sound, there will be no easy wins & you will need to work hard to prove your value.

Plan Guides

• Again these are an advanced feature. Anyone who created them would need to be extremely  familiar with query plan optimisation. Thus you will need to think hard about any changes you make.

As always I’m interested in any thoughts & suggestions you may have

Dave

Random Thought for the post:

“Look before you leap” might be good advice. But it is unlikely to be a great legal defence if police catch you hanging off the roof, staring into the change room window at the local public pool.

In this post I’ll start with the really obvious. Often there you can think of multiple coding alternatives that yield the same answer. How do you know which code construct is the most efficient?  This post offers one technique to help you determine that.

It is useful when comparing  different approaches to those High Frequency – Low Impact queries. By that I mean, comparing queries that by themselves really don’t take long, but they are run so often that they begin to represent a significant portion of the load on your system. Another area of use is, testing a code construct in a Select List that might be executed once for each row, in a billion row update they can really influence the time to complete the query.

For Example

One of the columns in your select statement must display a cheque number with 10 digits. Eg: 1,234,567 displays as 0001234567. So you need to append leading zeros if your integer contains fewer than 10 digits.  Which approach should you use?

1.  Convert your int to a string, Find the length of the string. Subtract the length from 10 & pre-pend that many zeros.

SUBSTRING('0000000000', 1, 10 - LEN(CAST(@ChequeID AS varchar(10)))) + CAST(@ChequeID AS varchar(10))

2.  Convert your int to a string, Prepend 10 zeros. Then take the Rightmost 10 chars.

RIGHT( '0000000000' + CAST(@ChequeID AS varchar(10)), 10)

3.  Same as Approach 1 but test to see if the Integer converts to exactly 10 digits & only do the Substring manipulation when you have less than 10 digits.

CASE WHEN LEN(CAST(@ChequeID AS varchar(10))) < 10

THEN SUBSTRING('0000000000', 1, 10 - LEN(CAST(@ChequeID AS varchar(10)))) + CAST(@ChequeID AS varchar(10))

ELSE CAST(@ChequeID AS varchar(10))

END

Note:

From a practical point of view you may think "OK, this is trivial stuff, after all there is only a few milliseconds difference anyway".

Answer 1

You may be correct, a good index is likely to be far more important. But remember, multi-table joins can generate millions of rows before they are filtered by the where clause. Saving a few seconds CPU time on each query can have a cumulative effect on overall system performance.

It may not be a huge win, but if it takes just as long to write inefficient code as it does to write more efficient code. Why not learn which code constructs are more efficient?

Answer 2

Try looking at the difference when you call a User Function from your select list. It can make your query 50 times slower. Sometimes performance differences can be massive, these code alternatives can occasionally be much more important than an index.

The basics

For queries that are executed very frequently (High frequency queries), the normal approaches like “SET STATISTICS IO ON” & “SET STATISTICS TIME ON” are inadequate. You get thousands of rows of output all saying “0 ms”. So I’ve reverted back to creating a loop, executing the test many times then comparing the time at the start & end of the loop.

Simple code example

-- ---< Template Testing Loop code >---
DECLARE @cntLoop INT;        SET @cntLoop = 1
DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

WHILE @cntLoop < 1000 BEGIN

    -- < Insert your test code here >

    SET @cntLoop = @cntLoop + 1
END
SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Test Description Here (ms)';
Go

(NB: If you think the layout of the DECLARE - SET looks odd, I converted the 2008 code back to 2005. In 2005 it is milliseconds faster to Declare all variables in one statement separated by commas. Then set them all in a single SELECT Statement. In 2008 you can Declare & Initialise on the same line. So for readability I've used the SQL 2008 layout but in SQL2005 compatible syntax.)

Taking a Baseline

Unlike Heisenberg we can & should adjust for the overhead of our measuring equipment. So at a minimum we should run 3 tests :-

1. The test loop with nothing inside.

2. The test loop with our orignial TSQL code.

3. The test loop with our hopefully faster TSQL code.

That way we can subtract time it takes to run the test “doing nothing” from the other 2 runs in order to really understand the percentage improvement between our alternatives.

Example - Measure the overhead of your looping code

NB: To make it easier to manage multiple tests I use a SQLCMD Variable to set the maximum Iterations.
-- =========================================================================================
-- Test Harness to compare impact of small High Frequency Operations &/or Things called once for each row in a query.
-- Method: Remember to subtract the overhead of the Test Harness from each result.
--           Remember to Click "Menu: Query->SQLCMD mode" or the SETVAR MAXLOOPS line will give you an error.
-- =========================================================================================
-- Change this value to impact the duration of ALL tests.
:SETVAR MAXLOOPS 2000000

-- =========================================================================================
-- ---< Pre-Test: Calculate Overhead of Looping code >---
DECLARE @cntLoop INT;         SET @cntLoop = 1
DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()
DECLARE @Result VARCHAR(8); -- or whatever data type your test returns.

WHILE @cntLoop < $(MAXLOOPS) BEGIN

    SELECT @Result = 1 -- Empty assignment

    SET @cntLoop = @cntLoop + 1
END
SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Empty MAXLOOPS Overhead';
go

-- =========================================================================================
-- ---< Test 1: Calculate Overhead of Calling existing User Function >---
DECLARE @cntLoop INT;         SET @cntLoop = 1
DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()
DECLARE @Result VARCHAR(8); -- or whatever data type your test returns.

WHILE @cntLoop < $(MAXLOOPS) BEGIN

    -- Orignial version
    SELECT @Result = [dbo].[uf_MinutestoStringTime] (@cntLoop)

    SET @cntLoop = @cntLoop + 1
END
SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Original';
go

-- =========================================================================================
-- ---< Test 2: Calculate Overhead of Calling "New Improved" Function >---
-- My results show this takes ~4% of Original ie: 25 Times faster. (or whatever you got)
DECLARE @cntLoop INT;         SET @cntLoop = 1
DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()
DECLARE @Result VARCHAR(8); -- or whatever data type your test returns.

WHILE @cntLoop < $(MAXLOOPS) BEGIN

    -- Refactored Improved version
    SELECT @Result = [dbo].[uf_MinutestoStringTime2] (@cntLoop)

    SET @cntLoop = @cntLoop + 1
END
SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Refactored to use Set functions';
go
-- =========================================================================================

Factor in the random crap

Of course the universe will gang up on you to ensure your results are meaningless.

• Clearly it is best if you can run it in a way to avoid load from others. Perhaps on a machine where you are the only user. eg: your workstation or a test server or after hours. But even then you are never alone, all sorts of background threads & tasks are going to partially impact a test run. eg: A SQL Checkpoint process could completely overlap just one of the tests & not the other. The SQL log may run out of space & need to be expanded in the middle of your test. etc

• Another bugger factor is that Windows is interrupt driven, at least every 15ms. (well on x86 PC architectures anyway). Your test could be paused while something else runs. So the smaller the timeframe the greater potential for statistical error. But as mentioned in the prior paragraph, if you loop more times, the longer the timeframe of the tests, the greater potential for impact via other threads.

• Another pain is that the test you try first could warm up the cache or environment & somehow assist the test that runs second. Making it look faster than it really is. Sure you could Free buffers &/or stop & restart the SQL Instance each time, but that slows you down & is probably not typical of the hot environment your queries will most likely be running in.

So some of the ways you can minimise errors are:-

1. In the same test run, run each test more than once. I usually run 6 tests:-
             a) Test Harness  (for the baseline)
             b) Test A
             c) Test B
             d) Test B (again)
             e) Test A (again)
             f) Test Harness (again)
   That way you can check that each of the 3 tests deliver similar results in both executions. If they don’t, the entire test run is invalid. (Or at least can’t be reliably extrapolated)
2. Like all good statisticians. Run the test group a few times. 30 would be a statistically good number, but 5 is probably adequate especially if each run is giving similar results.
3. If you can & it is important enough for you. Remember to try it on multiple machines.
   CPU & Memory Access speeds often vary between machine & I have seen coding alternatives which are dramatically different on one machine be not as dramatic on another. (but usually there is a clear winner everywhere or there was only a negligible 1-3% difference anywhere.)
   Especially important is different disk capacities. While this type of testing is less useful for I/O intensive queries, I don't know what you will be doing so maybe you need to think about it.
   

Example Results

These were obtained from running the 10 digit leading zero tests mentioned above.

In the above you can see the raw results in Columns B-I.

Column J shows the difference between the two Right Append Tests (cols D & G)

Cols N - Q shows the % change between each of the similar tests in that run. Where 100% is No difference. Anything greater or lower is bad.

While I'm not really happy with the huge variation within each test run, it is sufficient to make a decision. It is easy to see the difference between each type of test is even more dramatic than the margin of error in each test.  So it is reasonable to conclude that the "Right()" approach is more efficient than the "Substring()" technique. And any benefit you thought you'd get by using a case to avoid any unnecessary substring commands is a waste of time.

Bonus TIP:

These results are consistent with every TSQL test I've done. In TSQL there is no point testing to see if an operation needs to occur & then doing it. Just do it to all rows.

For example this query is slower

SELECT CASE WHEN LastName IS NOT NULL THEN UPPER(LastName) END As Lastname

FROM Custs

than this query. The UPPPER() function is perfectly capable of dealing with NULL's.

SELECT UPPER(LastName) As Lastname

FROM Custs

On my machine the later took 100 units of time & the former (with the CASE statement) took 187 Units. Ie 87% slower.

Template Loop Code:

This code is extended slightly from the basic example above. Showing setting any other variables you may need in your test. Together with an improved way to view the results of the query. If you don't assign them to a variable they will be returned to the screen (or calling app), this will slow your test.

You will note the :SETVAR this lets me refer to a variable with a life beyond each batch. So remember to enable SQLCMD in SSMS. Eg: Menu: Query => SQLCMD Mode.

Changing this value will increase / decrease the length of each test. As Windows timer is in 15 ms chunks, it is best to adjust this so that each test runs for at least several seconds.

-- ---< Set Test Loop Counter >---

:SETVAR LOOPCNT 100000 -- < How many times each test will be repeated.

-- ===< Empty Test: Measure Test Harness overhead >==============

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

-- < do nothing here - empty test >

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Empty Harness 1' -- << Meaningful label for test

GO

I also like to display the value returned by my query (ie:@Results) this is a sanity check just to ensure that the codes snippets do actually return identical code. I realise this is only displaying the last value assigned & every value assigned in prior iterations of the loop were overwritten. Normally this would be bad, in this case I don't care, they should all be identical.

Sample Code to test the 3 alternatives discussed in the introduction

Use this as your template.

/* ===========================================================

** TEST HARNESS for small high frequency tests.

** eg: TSQL run for every row in a large query.

** Notes: Before reaching any conclusion.

** 1. You should repeat these tests at least 5 times

**    & ensure the variance between each run is small.

** 2. You need to mirror the tests components

**    ie: Harness, Test A, Test B, Test B, Test A, Harness.

**    Then check if Harness = Harness, Test A = Test A, etc

**    If not then the Test Run would likely to be adversely impacted by something.

** 3. It is safer to push the Test Output into a variable then to the screen.

**    This removes delay caused but network & client.

** 4. This uses Old-Style Variable initialisation to ensure it works on pre-SQL2008

** 5. Ensure SQLCMD mode = ON

** ===========================================================

*/

SET NOCOUNT ON

-- ---< Set Test Loop Counter >---

:SETVAR LOOPCNT 100000 -- < How many times each test will be repeated.

-- ===< Empty Test: Measure Test Harness overhead >==============

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

-- < do nothing here - empty test >

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Empty Harness 1' -- << Meaningful label for test

GO

-- ===< TEST A: 1st Run >==============

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = SUBSTRING('0000000000', 1, 10 - LEN(CAST(@CommSourceID AS varchar(10)))) + CAST(@CommSourceID AS varchar(10))

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Substring Manipulation 1'

GO

-- ===< TEST B: 1st Run >==============   

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = RIGHT( '0000000000' + CAST(@CommSourceID AS varchar(10)), 10)

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Right Append 1'

GO

-- ===< TEST C: 1st Run >==============   

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = CASE WHEN LEN(CAST(@CommSourceID AS varchar(10))) < 10

THEN SUBSTRING('0000000000', 1, 10 - LEN(CAST(@CommSourceID AS varchar(10)))) + CAST(@CommSourceID AS varchar(10))

ELSE CAST(@CommSourceID AS varchar(10)) END --AS BarcodeNumber,

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'CASE & Substring 1'

GO

-- ==============================================< Repeat Tests in reverse order >==============

-- ===< TEST C: 2nd Run >==============   

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = CASE WHEN LEN(CAST(@CommSourceID AS varchar(10))) < 10

THEN SUBSTRING('0000000000', 1, 10 - LEN(CAST(@CommSourceID AS varchar(10)))) + CAST(@CommSourceID AS varchar(10))

ELSE CAST(@CommSourceID AS varchar(10)) END --AS BarcodeNumber,

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'CASE & Substring 2'

GO

-- ===< TEST B: 2nd Run >==============   

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = RIGHT( '0000000000' + CAST(@CommSourceID AS varchar(10)), 10)

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Right Append 2'

GO

-- ===< TEST A: 2nd Run >==============

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

SELECT @Results = SUBSTRING('0000000000', 1, 10 - LEN(CAST(@CommSourceID AS varchar(10)))) + CAST(@CommSourceID AS varchar(10))

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Substring Manipulation 2'

GO

-- ===< Empty Test: Measure Test Harness overhead >==============

-- ---< Set Test Specific Variables >---

DECLARE @CommSourceID BIGINT; SELECT @CommSourceID = '1234567'

-- ---< Set Harness Variables >---

DECLARE @Results VARCHAR(20)

DECLARE @cnt INT;             SET @cnt = 0

DECLARE @StartTime DATETIME; SET @StartTime = GETDATE()

-- ---< Execute Test many times >---

WHILE @cnt < $(LOOPCNT)

BEGIN

-- ---< Insert your Test Query Below here >---

-- < do nothing here - empty test >

-- ---< Insert your Test Query Above here >---

SET @cnt = @cnt + 1;

END

-- ---< Display Test Results >---

SELECT DATEDIFF(ms, @StartTime, GETDATE()) AS Duration_ms

,@Results AS 'Empty Harness 2' -- << Meaningful label for test

GO

-- ===< END >==============

Example 2:  Can trivial User Defined Functions slow you down.

This code sample has been truncated to save space it only shows tests running once.

It compares a User Defined Function,vs the UDF rewritten, vs the rewrite just executed in the query without calling a UDF vs the inline code trimmed to its essentials. While the rewritten UDF code was much faster, calling it as a UDF made the difference negligible. By removing the calling overhead & incorporating it into the TSQL Query that was calling the UDF you start to see real perf gains. Eg ~50 times faster.

Note 1:  It is wrong to draw a conclusion from these results that all User Functions are bad & hurt performance. I have seen many occasions where they result in a perf gain. However, they are nearly always slower than putting the same code in-line or taking a huge join out of a UDF & extending the joins in your base table to include these extra table.  Ie: It is more efficient to join an extra table to your base query.

NB: An exception to this, is if the UDF permits you to return 1 row based on a parameter, this becomes like a bookmark lookup & can be much more efficient than joining with a huge table.

Note 2: I agree it would be easier to use the SQL 2008 Time data type, this code was written long ago & needed to run on SQL 2000 & SQL 2005 systems.

Note 3: As we iterate thru the loop it is also passing all possible values as parameter to the User Function. Just in case one code path is slower than another.

Original User Function

-- =============================================

-- Author:        Some Noddy

-- Create date: 5 June 2005

-- Description:     This converts a number of Minutes into a Time string without a date.

-- =============================================

USE TempDB

go

CREATE FUNCTION [dbo].[uf_MinutestoStringTime] (@minutes int)

RETURNS varchar(8)

AS BEGIN

DECLARE @RETURN varchar(8),

@computedminutes int

IF @minutes IS NULL

RETURN ''

IF @minutes > 1440

RETURN ''

IF @minutes < 1

RETURN ''

IF @minutes = 720

SELECT @RETURN = '12:00 PM'

IF @minutes = 1440

SELECT @RETURN = '12:00 AM'

ELSE IF @minutes < 780

SELECT @RETURN = CAST(ABS(@minutes / 60) AS VARCHAR(3))+':'+ right('0'

+ CAST(@minutes - ( ABS(@minutes / 60) * 60) AS VARCHAR(4)),2) + ' AM'

ELSE IF @minutes > 780

BEGIN   

SET @computedminutes = @minutes - 720   

SELECT @RETURN = cast(abs(@computedminutes / 60) AS VARCHAR(3))+':'+ right('0'

+ CAST(@computedminutes - ( ABS(@computedminutes / 60) * 60) AS VARCHAR(4)),2)

+ CASE WHEN @minutes > 719 THEN ' PM'

WHEN @minutes < 719 THEN ' AM'

ELSE ''

END

END

RETURN @RETURN

END

Refactored User Function

-- =============================================

-- Author:        David LEAN

-- Create date: 25 Sept 2009

-- Description:    This converts a number of Minutes into a Time string without a date.

-- Note:    It is an error to pass more minutes than contained in a single day. ie: 1400

--            It is an error to pass a negative value.

-- 0 = 12:00 AM, 59 = 12:59 AM

-- 719 = 11:59 AM, 720 = 12:00 PM

-- 1440 = 12:00 AM

-- =============================================

USE TempDB

GO

CREATE FUNCTION [dbo].[uf_MinutestoStringTime2] ( @minutes int )

RETURNS varchar(8)

AS BEGIN

IF     (@minutes >= 1) AND (@minutes <= 1440)

RETURN ISNULL(LTRIM(STUFF(RIGHT( CONVERT( VARCHAR(20), DATEADD(mi, @minutes, '12:00 AM')) ,7 ), 6,0, ' ')), '')

RETURN ''

END

Sample Test Harness to compare both User Functions and Putting the code inline.

-- ==============================================

-- Test Harness to compare impact of small High Frequency Operations: eg: Things called once for each row in a query.

-- Method: Remember to subtract the overhead of the MAXLOOPS from each result.

--            Remember to Click Menu: Query->SQLCMD mode to allow the MaxMAXLOOPS Counter.

USE TempDB

SET NOCOUNT ON

-- Change this value to impact ALL tests

:SETVAR MAXLOOPS 200000

-- =========================================================================================

-- ---< Pre-Test: Calculate Overhead of Looping code >---

DECLARE @cntLoop INT

SET @cntLoop = 1

DECLARE @Minutes INT

DECLARE @Result VARCHAR(8);

DECLARE @StartTime DATETIME

SET @StartTime = GETDATE()

WHILE @cntLoop < $(MAXLOOPS) BEGIN

SET @Minutes = @cntLoop % 1441

SELECT @Result = 1 -- Empty assignment

SET @cntLoop = @cntLoop + 1

END

SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Empty MAXLOOPS Overhead';

go

-- =========================================================================================

-- ---< Test 1: Calculate Overhead of Calling existing User Function >---

DECLARE @cntLoop INT

SET @cntLoop = 1

DECLARE @Minutes INT

DECLARE @Result VARCHAR(8);

DECLARE @StartTime DATETIME

SET @StartTime = GETDATE()

WHILE @cntLoop < $(MAXLOOPS) BEGIN

SET @Minutes = @cntLoop % 1441 -- Ensure you test every possible value.

SELECT @Result = [dbo].[uf_MinutestoStringTime] (@cntLoop)

SET @cntLoop = @cntLoop + 1

END

SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'uf_MinutestoStringTime';

go

-- =========================================================================================

-- ---< Test 2: Calculate Overhead of equivalent TSQL Statment >---

-- NB: This returns slightly different results as I believe the existing code gives wrong results between 12:00AM -1:00am & also 12:00pm & 1-pm

-- My results show this takes ~4% of Original ie: 25 Times faster.

DECLARE @cntLoop INT

SET @cntLoop = 1

DECLARE @Minutes INT

DECLARE @Result VARCHAR(8);

DECLARE @StartTime DATETIME

SET @StartTime = GETDATE()

WHILE @cntLoop < $(MAXLOOPS) BEGIN

SET @Minutes = @cntLoop % 1441

SELECT @Result = LTRIM(STUFF(RIGHT( CONVERT( VARCHAR(20), DATEADD(mi, @cntLoop, '12:00 AM')) , 7 ) , 6,0, ' '))

SET @cntLoop = @cntLoop + 1

END

SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'Ltrim Inline';

go

-- =========================================================================================

-- ---< Test 3: Calculate Overhead of equivalent TSQL Statment BUT no space between Time & AM/PM >---

-- NB: This returns slightly different results as I believe the existing code gives wrong results between 12:00AM -1:00am & also 12:00pm & 1-pm

-- My results show this takes ~2% of Original ie: 50 Times faster.

DECLARE @cntLoop INT

SET @cntLoop = 1

DECLARE @Minutes INT

DECLARE @Result VARCHAR(8);

DECLARE @StartTime DATETIME

SET @StartTime = GETDATE()

WHILE @cntLoop < $(MAXLOOPS) BEGIN

SET @Minutes = @cntLoop % 1441

SELECT @Result = (RIGHT( CONVERT( VARCHAR(20), DATEADD(mi, @cntLoop, '12:00 AM')) , 7 ) )

SET @cntLoop = @cntLoop + 1

END

SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'uf inline';

go

-- =========================================================================================

-- ---< Test 4: Calculate Overhead of equivalent TSQL Statment put into a User Function >---

-- NB: This returns slightly different results as I believe the existing code gives wrong results between 12:00AM -1:00am & also 12:00pm & 1-pm

-- My results show this takes ~98.5% of Original ie: Calling overhead on trivial User Functions can have a huge Performance Impact.

DECLARE @cntLoop INT

SET @cntLoop = 1

DECLARE @Minutes INT

DECLARE @Result VARCHAR(8);

DECLARE @StartTime DATETIME

SET @StartTime = GETDATE()

WHILE @cntLoop < $(MAXLOOPS) BEGIN

SET @Minutes = @cntLoop % 1441

SELECT @Result = [dbo].uf_MinutestoStringTime2 (@cntLoop)

SET @cntLoop = @cntLoop + 1

END

SELECT DATEDIFF(ms, @StartTime, GETDATE() ) AS 'uf_MinutestoStringTime2';

go

-- =========================================================================================

Warnings

This approach has its limitations. It tends to be less useful for comparing if a query works better with or without an index. In that case you really need either :-

1. Run the same query against 2 databases, one with the indexes & the other without. OR
2. Run the test, create the index & run the other test. OR
3.  Use Visual Studio for DB Professionals &/or RML Utilities to create & measure the load.
   Fortunately using SET STATISTIC IO ON means you should be able to quickly see the winner without resorting to this timing loop approach.
    

Final Thought
So next time you wonder which is faster :-

1. GETDATE

   () or CURRENT_TIMESTAMP
2. “ IF OBJECT_ID(N'dbo.MyTable', N'U') IS NOT NULL; “ or
   “ IF NOT EXISTS (SELECT * FROM sys.objects where object_id = OBJECT_ID(N'dbo.MyTable') AND Type = N'U');

3. Is there much overhead in putting this code in a USER DEFINED FUNCTION & calling it?
4. Is it better to check if a variable is NULL before needlessly passing it to a function? ie:
   IF @productName IS NOT NULL BEGIN
        SET @productName = LTRIM(@productName) SET @productName = RTRIM(@productName)
   END
   vs
   SET @productName = LTRIM( RTRIM(@productName) )

Now you can quickly recycle this code from here & test it out.
(By the way the answers are; No Difference, 1st Option 60% faster, tiny udfs can be 50 times slower than inline., Later alternative is 2.5x faster) 
 

Random Thought for the Post.

In meetings I often hear people say "There's no such thing as a stupid question".

That might be true, however I think "Excuse me Mr Customs man, do you think the heroin in my padded bra makes my breasts look unnaturally large?" must come close. … Especially if you are a man.

I hope you find this useful. As always I’m very interested in your comments, Good or bad. thx Dave

Recently I was developing a SQL Server Analysis Server cube using Oracle as a data source. Weirdly the the development environment (BI Dev Studio) could see the Oracle schema & let me explore the data. But when I tried to process the cube or one of its dimensions, it died indicating that I couldn’t connect to the Oracle Server. Which clearly I could do as I’d been building & running SQL Reports from the same project & could see the data in the development tool.

If you are experiencing any of the following errors this post may help.

• Bad Username / Bad Password
• ORA-01017: Invalid username/password; logon denied.
• ORA-12154: TNS could not resolve the connect identifier specified.

While trying to resolve this issue I read a whole lot of misleading information on Internet forums. These misguided “solutions” distracted me. I hope this post will make it easier for you.

In Short:

You develop the cube in the Business Intelligence Development Studio (BIDS) tool. It is 32 bit process that connects to both Analysis Server & your Oracle Data source. (Purple arrow) It uses the connection information you defined in the Data Source object you added to the Project.

When you process the cube or a Dimension, BIDS passes the relevant definitions to SQL Server Analysis Server (SSAS). (Purple arrow) The SSAS attempts to connect to the Oracle database to process the data. (Blue Arrow)
 

It makes no difference if BIDS is running on the same machine as SSAS. It is designed to work in any of the topologies shown above. That way if SSAS is on a dedicated Server it can run with the full power of a server. That means SSAS doesn’t use the same credentials as the BIDS process.

SSAS does use the same information in the data source to connect to Oracle as you are using from BIDS but runs them under the credentials of the Windows Account used to start the service. Commonly the Service is started as either “NT AUTHORITY\NetworkService”, "Local System” or “Local Service”. It is highly likely that this account will not have sufficient permission to connect thru your domain to the Oracle Server. Especially in a heterogeneous environment of Linx / Unix / Windows, with a variety of Authentication systems etc.

It is possible that the Account you are using to run SSAS doesn’t have permission to read the Oracle directories containing the TNSNAMES.ORA file(s) which the Oracle Client interface needs to connect to your Oracle database. (test this by temporarily putting the SSAS Account into the local Administrators group)

It is possible that you have the wrong “Bitness” Oracle drivers installed. BIDS is a 32 bit application, on 64 bit windows it will be running in a 32 bit shell. But your SSAS system may be full 64 bit. You may need both 32 bit & 64 bit Oracle drivers installed & configured. (test this by installing a 64 bit Oracle client & seeing that it works correctly)

Trouble shooting technique. 

Test 1: If you are running BIDS & SSAS on the same machine and BIDS is connecting to Oracle OK.

• Try changing the SSAS Service Logon account to the account you used to logon to Windows. (see steps below). If it lets you process the cube, problem solved.
• Now go create a dedicated account on the Windows Domain for your OLAP service & use it. (otherwise when you change your password, OLAP server will not start, as you will likely forget to change it at the same time.  :-) ). 

Test 2: If SSAS is on a different machine.

• As above, check the Service Account, & possibly change it to an account you know can connect over the network to Oracle.
• Also check that you can connect from that server to Oracle. The easiest way is to install an Oracle client tool, eg: Oracle Enterprise Manager, TOAD for Oracle, or similar.

Bum steer

While trawling thru the net, looking for Oracle insight I stumbled on this post. It seemed insightful. It proved to be total garbage. Analysis Service does not make connections by doing something funky with HOME0. You do not need to edit registries or change Oracle connection information an a special way for SSAS. If an Oracle app of the same bitness (32/64 bit) can connect, then SSAS should be able to too.

https://www.tek-tips.com/viewthread.cfm?qid=1016413&page=28
”In its wisdom, Analysis Services uses the Oracle "HOME" string in the registry to find the TNSNAMES.ora.
When you're building the cube and the dimensions, it finds TNSNAMES.ora ok using HOME1, but then when you want to run the cube, it decides to use HOME0 as the Oracle Home directory - and I didn't have TNSNAMES.ora. Why can't Microsoft write the software to use a standard Oracle Home location? “

Details

As some people are shaky on Windows & mention they like pictures. Here are some screenshots that my help you.

Step 1: Analysis Services cube processing failed.

When Processing Dimensions you get an error message indicating the connection couldn’t be made to with the DataSource. You can ignore the rest of the errors as they are caused by this failure.

Picture: ORA-01017: Invalid username/password; logon denied.

 
Picture: ORA-12154: TNS could not resolve the connect identifier specified.

Note: It is likely you received this message after you attempted to process a cube or dimension. As below.

Right Click on the cube or Dimension you want to process.

Click Run to begin the processing operation.

Step 2: Ensure the Windows Domain Account you are using is in the appropriate SSAS Admin group

• To Open Computer Management: Click Windows Start.
• R.Click Computer –> Manage. Expand the Local Users & Groups node.
• Search the Descriptions looking for Instances that say “Analysis Services” if you have more than one, make sure you change the one you are using :-)
• Click “Add to Group…”

• Click “Add” to add a new user to this group.
  In this case I’m adding a service account “MyOLAPServiceAccount”.
• Then click “Check Names” & if it finds it, click OK.

You may also want to try adding the account to the local Administrators, in case it is a file permissions issue (can’t see the TNSNAMES.ORA file). But remember to remove it later. It is much safer to find the directory it needs & give the Service Account read access. Than to give it full Admin rights to the entire box.

 

Step 3: Try changing the Service Account of the SQL Service Analysis Services to logon with your account.

Note: Using your personal account to start any Windows Service is not a security '”Best Practice”. Once you’ve solved the issue, remember to create a Domain account for the OLAP service & ensure SSAS is using it & not yours.

• Windows Start Menu: All Programs => Microsoft SQL Server 200?? => Configuration Tools => Configuration Manager
• Choose Appropriate Instance of Analysis Services
• R.Click Properties.

Change from Built-in account: to This Account & enter you own personal account. (the fictional account I created for this blog was “CENTRAL\DLEAN2") or you could use something more meaningful Account Name like “DomainName\SSASServiceAcount”.
Note: Some sites create one Service account for all SQL Server services on all servers. Others have a separate account for each type of service (eg: Analysis, Reporting, Database). Others create a specific account for each SQL Service on each machine. There are pros & cons with each approach. The right answer will vary depending on the scale & interdependency of your environment, your security needs & your process of managing accounts & passwords.

TIP: Oracle Datasource View configuration

You will notice this post applies to a variety of Connection error messages. You will get a slightly different wording depending on the values you used when configuring the “Impersonation Information” but the end result is the same, you can’t connect.

To check your connection, Open the DataSource you are using in BIDS.

I’ve found both the Oracle & the Microsoft OLE-DB providers work well. You will need to specify your Oracle User name as I do not believe Oracle supports Windows NT Integrated Security (bummer huh).
(Note: I just invented the user name sepuser (September User) for this picture. It is not a special Oracle name)

In the Impersonation Information page. The Inherit option worked fine once I’d resolved the SSAS service account issue. You can also choose the other options but remember they are WINDOWS accounts. You have already typed in the Oracle Account info in the prior tab. This dialog is generic to all different types of OLE-DB providers & different authentication schemes, what I’ve shown works most times in an Oracle world.

I trust this gets you up & running quickly.

Dave

Credits

My thanks to Richard Lees and Edward Melomed for insight that helped me solve this issue.

If you are looking for SQL Add-in’s, Utilities, Sample Code & education, perhaps you will find this useful to you too.

SQL 2008 R2 Links

Microsoft SQL Server – Official Microsoft SQL server Site
      Resources for 2008 R2 – Handy Resources (Marketing, Pricing & Technical)

Microsoft® SQL Server® 2008 R2 Feature Pack (7th Jun 10) includes :-

• SQL Server® 2008 R2 Best Practices Analyzer
• SQL Server® 2008 R2 Upgrade Advisor
• SQL Server® 2008 R2 Books On-line
  
• SQL Server 2008 R2 Report Builder 3.0
• PowerPivot for Microsoft® Excel & Sharepoint
• SQL Server® 2008 R2 Policies
• SQL Server® 2008 R2 Remote Blob Store
• SQL Server® 2008 R2 Native Client
• SQL Server® 2008 R2 Command Line Utilities
• SQL Server® 2008 R2 Shared Management Objects
• SQL Server® 2008 R2 Analysis Management Objects
• SQL Server® 2008 R2 ADOMD.NET
• SQL Server® Service Broker External Activator for SQL Server® 2008 R2
• SQL Server® 2008 Data Mining Add-ins for Microsoft® Office 2007
• Sync Framework 2.0 Software Development Kit (SDK)
• SQL Server® Compact 3.5 SP2
• Windows PowerShell Extensions for SQL Server® 2008 R2
• Datamining Viewer Controls For Microsoft® SQL Server® 2008

Interoperability

• Microsoft® SQL Server® Migration Assistant
• JDBC Driver 3.0
• Microsoft® SQL Server® Driver for PHP 1.1
• Microsoft® Connector 1.1 for SAP BW for SQL Server® 2008 R2
• Microsoft® OLEDB Provider for DB2
• Microsoft® Analysis Services OLE DB Provider for Microsoft® SQL Server® 2008 R2
• Microsoft® System CLR Types for SQL Server® 2008 R2
  

Microsoft Report Viewer 2010 Redistributable Package 
      Lets you embed SQL Reports into your Windows Forms & Web applications.
      Brian Hartman's Report Viewer Blog – Nice value add on adding reports into your application.

Sample Databases for Microsoft SQL Server 2008 R2 (RTM) – Updated Adventure Works, Northwind etc.

SQL 2008 Performance Tools

Not a comple list but a few of my favourites 

Replay Mark-up Language (RML) Utilities
      RML Utilities for SQL Server (x86)
      RML Utilities for SQL Server (x64)
      RML is an awesome utility for analysing load on SQL Server & for creating a load.

SQL Load Test – This is a handy utility for creating Visual Studio Load Tests from a SQL Profiler Trace. While far from a complete solution (as it doesn’t handle a lot of SQL Syntax) it does speed the process somewhat. I hope to enhance it in time.

NodeXL: Network Overview, Discovery and Exploration for Excel – I find this handy to visualise the SQL Object Dependence within a database. ie: Quickly seeing that this SP calls 2 others & they call a function or two that reference views & tables. OR how many objects touch this table & how many reference them all.

I see huge potential in using Visual Studio to perform Load tests, Stress Tests & to ensure quality when you are rewriting TSQL code. Unfortunately it is missing a small number of features &/or hooks that make the whole exercise marginal value. One day, soon I hope to have coded up extensions to fix those feature gaps.

Using Visual Studio 2005 to Perform Load Testing on a SQL Server 2005 Reporting Services Report Server – Old article but helps setup Web Testing on SQL RS

Introducing New Features In The VSTS Database Edition GDR

Apply Test-Driven Development to your Database Projects – This article shows how to write the 2 custom tests that make DB tests kinda useful. (Without them, don’t bother)

Database Unit Testing with Team Edition for Database Professionals – Walk thru on creating a DB Unit Test.

Microsoft® Visual Studio Team System 2008 Database Edition Power Tools – Tantalisingly nice add-in, likely to not work on your system as the GDR release broke much of it.

How to get Database Edition Power Tools to work on Visual Studio Team System 2008 Database Edition GDR

SQL 2008 Samples, Training, Whitepapers

Microsoft SQL Server Community Projects & Samples - Loads of great Code Samples

SQL Server Customer Advisory Team - SQL Server Best Practices – Whitepapers from the SQL Dev Team tasked with tuning the largest SQL Customer implementations.
      SQLCAT Community Projects and Code Samples – Nice set of handy add-ins

SQL Server 2008 R2 Update for Developers Training Kit (June 2010 Update) 
      Videos, Presentations, Demo’s & Hands-On-Labs to get you up to speed on using the new capability of SQL Server 2008 R2. Includes the 2008 stuff too.

General Dev Resources

Tech-Ed Online Great technically deep Webcasts on a vast number of topics.

MSDN Magazine – Good articles on new technologies that all developers should scan.

Development Samples, Training, Whitepapers

All-In-One Code Framework! -  framework and skeleton of Microsoft development techniques through typical sample codes in three popular programming languages (Visual C#, VB.NET, Visual C++).

Visual Studio 2010 and .NET Framework 4 Training Kit – Presentations, Hands-On Labs, Demos on; Entiry Framework, C#, VB.NET, F#, ADO.NET Data Services, WPF, WCF, Parallel Extensions, Windows Workflow, ASP.NET, VS Team System & More.

ADO.NET Entity Framework & LINQ to Relational Data – Samples for EF & Link

Microsoft Patterns & Practices - Enterprise Library - The Microsoft Enterprise Library is a collection of reusable software components (application blocks) designed to assist software developers with common enterprise development challenges.
      Enterprise Library Contribution Project – Extensions people have added to make it even easier.

Silverlight

Microsoft Silverlight – The Official Microsoft Silverlight Site. Good Tutorials, SDK, Downloads, Samples. etc

Silverlight 4 Training – Detailed Whitepapers & Labs to guide you to build business level applications

Microsoft XAML Toolkit CTP - July 2010 - Microsoft XAML Toolkit is a set of libraries enabling easier manipulation and analysis of XAML for .NET 4.0 and Silverlight.

Blacklight - Controls for Designers! Blacklight is a UX focused code sharing project. collection of controls, samples, visual assets and ideas that has been put together by User Experience designers and developers to both show you what the technology is capable of (from a UX point of view),

ASP.NET AJAX Control Toolkit rich set of 40+ controls that you can use to build highly responsive and interactive Ajax-enabled Web applications.

ASP.NET QuickStart – Sample ASP Apps for Web folks

WindowsClient.NET – The Official Microsoft Site for Windows Forms & Windows Presentation Foundation

Windows Presentation Foundation (WPF) - WPF Toolkit and the WPF Futures releases.

Expresso 3.0 – Brilliant tool to assist anyone writing Regular Expressions (REGEX). Essential for Data Cleansing, Screen Scraping & a variety of SSIS uses. (free tool from Jim Hollenhorst of Ultrapico)

Excel Programming

VBA for Excel (Macros) – Nice set of 33 lessons that cover how to get VBA Macros working in Excel.

Excel VBA Macro Codes Tips & Tricks

Julian's Excel Macro (VBA) Tips for Beginners

---< END >---

Thought for the post:
If a Bedding Store is having a 50% off sale, & you buy a Double bed, will they deliver a Single bed?
If you buy a single bed with a “2  for 1” coupon & they deliver a Double bed, did you get what you paid for?

Note: This might not be true, but I think of it every time I hear the word iPad. Now you can too.

If I worked for Apple it would only be a matter of time before they shipped a range of new products :-

• iGlasses for the Elderly
• iWear for the rest of us
• Fashion iWear for the Trendy
• spiGlasses for the Captains of the Pirate ships, oh and for 007 James Bond
• iSpy (with my little eye), for kids whose parents take them on long trips & can’t afford to buy technical junk to keep them from talking to any other human.
• iCanSeeYou for people who knock on doors of residents who pretend they aren’t home.
• iKentinaturner similar to an iPod but it only plays “Nutbush City Limits” & “Proud Mary”.

(So now you know what I think about while watching 100’s of Millions of rows load into a PowerPivot spreadsheet. How cool is that? more on that later)

While you are waiting for it to be available for download via MSDN or delivered to you on a DVD fulfilment. Check out The SQL Server R2 Digital Tour

What will you find there? Links to a whole host of resources, including

1. The SQL 2008/R2 Resource Kit
2. SQL Server 2008 R2 Update for Developers Training Kit
3. a Small eBook listing all the new features, so I will not repeat it here.
4. Webcasts (inc the following)

• An Introduction to SQL Server 2008 R2
• SQL Server 2008 R2 Overview
• Introduction to SQL Azure
• Lower Costs through BI Consolidation
• Migrating from Oracle to SQL Server
• SSRS and Report Builder 3.0
• PowerPivot for Excel and SharePoint
• Master Data Services
• Business Continuity with SQL Server and Windows Server 2008 R2 Hyper-V Live Migration
• Application and Multi-Server Management
• Parallel Data Warehouse    
  Develop Rich Reporting Solutions
• StreamInsight Complex Event Processing
• SQL Server 2008 R2 and BizTalk Server

Don’t forget to check out the new SQL Server 2008 R2 PowerPivot for Excel. The BI capabilities for End Users are truly ground-breaking, & like Microsoft Office 2010 it is also ready to go now.

enjoy,

PS: In case you are still have spare time, check out the Visual Studio 2010 it is really nice. Don’t overlook the features for testing your database.

I had some rot in my wooden deck & needed to cut out some wooden panels precisely on the joists. I tried a chisel & a whole lot of BS. It took me forever just to do one & I botched it. So I put it on the backburner, went back to my computer & did nothing more for months.

This Easter, my wife got very enthusiastic about me finishing the job. And so, with her by my side, I stumbled across this Power tool at the local H/W store (Bunnings).  It has a square saw attachment that you can see sticking out of the front in the picture below. It is so cool, cuts a straight line directly into wood, as fast as a hot knife into butter. Well as fast as you’d drill a hole in wood anyway.

It also has a ton of attachments I don’t care about; to sand, cut pipes, tiles, cut things flush etc.

So if you install Network Ports in skirting boards or have any other reason to cut high precision rectangular holes in wood, plaster walls or similar, this is definitely something you should check out.

Be warned, cutting with this thing is kinda fun. The guy who sold me this was very insistent on my owning a modern cable & pipe detector. My wooden deck was outside so I didn’t have to worry about water or power. So I didn’t get one. But can now see why he thought it wise to own one. This thing would cut thru copper pipes & cables like they weren’t there. ;-)