Ivan Donev's blog

Free scripts and tools for your Production SQL Server

Ivan Donev [MSFT] — Thu, 03 Nov 2016 20:43:00 GMT

Yes, that is right – FREE scripts and tools, created by Microsoft SQL Server Tiger PM Team (blog | twitter) are now live on Microsoft repository in github.com. So, go to https://github.com/microsoft/tigertoolbox and enjoy the free stuff. Some notes on the solutions I’ve already used on a lot of production environments: - SQL Server Performance baseline – probably the best solution I ever worked on the topic of performance baselining. You get a bunch of reports, pre-created for you, that you just need to deploy to a RS instance. The solution uses a distributed system of collecting data, where each monitored instance collects data on its own (in a local database). I’ve hit couple of issues with that solution though, so please be aware – you will need Kerberos configured for your RS server and might need to change some of the queries to utilize the TRY_PARSE() statements, especially on source systems that are with different Windows Locale. [Edit: 11/24/2017] : Thanks to Parikshit Savjani (SQL Tiger PM) - if you are allowed by your organization's security standards, you can store credentials (both Windows and SQL) in the data sources - https://technet.microsoft.com/en-us/library/ms159736(v=sql.105) and https://technet.microsoft.com/en-us/library/ms160330(v=sql.105) - SQL Server System Health Session PowerBI – you get a template of a PowerBI report, where you just change the source instance and immediately get visual representation of your system health session (SQL 2012+). Which means that even if you do not have any other performance collections and you need some history information on your instance – this is the thing for you. Now you have a tool to visit the history of your SQL. - BPCheck – this is actually a must have utility for your environment. It is not something that you run on daily basis, but more like a tool you use to check your best practices and sweep your instance for issues on configuration, performance, indexing, plan caching, etc. Make sure you test before you run it on PRODUCTION though. And make sure you are aware of all the input parameters for the script! There are over 10 different solutions/scripts/utilities at the tigertoolbox, so make sure you check them all! And test before you implement. Of course, you are also free to adapt them to your environment, as every piece of source code is OPEN for you to use!

SQL Saturday Sofia 2016 recap

Ivan Donev [MSFT] — Thu, 20 Oct 2016 09:20:26 GMT

SQL Saturday #538 Sofia just ended. It has been an absolutely great experience!!! The recap of the event, including my slidedeck, you can find HERE. Additionally, I'd like to remind everyone that although the Data loading performance guide has been written back in 2008, it still apply, so if you are doing ETL or you need to load data quickly, please - always refer to it!

Data loading performance guide

And if you are in the business of bulk loading data into columnstores, just look over the below material.

Columnstore indexes data loading

Visiting Prague’s PASS BI and SQL Server chapter

Ivan Donev [MSFT] — Fri, 07 Oct 2016 20:41:00 GMT

Last week I was in Prague working with several customers on SQL Server performance tuning and baselining (and some extra fun with setting up Kerberos authentication for Reporting Services 2016). So it was a week full of joy :-) But what made the week special, was that I was given the opportunity to speak in front of the local PASS BI and SQL Chapter. And as far as I know, this is one of the latest additions in the PASS community and the energy around it is amazing. The guys organizing it are putting a great deal of their personal time to make it happen and I can definitely see the results. As I am doing similar things in Bulgaria, I know what this means. And the opportunity I was given to share some of my knowledge and experience was just great. The event itself was held in Microsoft's office and the room was full. Not only that, but everyone that came wanted more and more …. So we did a 2 hour meetup, full of deep dive in indexing, best practices and techniques on monitoring index effectiveness, overcoming limitations and having fun with implicit conversion. The great thing of being in Prague on a SQL Event is that there is always a great pub nearby :-) And I cannot wait until the next time I meet that awesome SQL community in December on SQL Saturday Prague! So, if you are nearby or just want to visit Prague and combine it with some SQL Server, register now! Thanks again Pavel, Tomas, Jaro and everyone else involved! See you in December :-)

It’s October, it’s time for SQL Saturday in Sofia

Ivan Donev [MSFT] — Mon, 03 Oct 2016 07:34:00 GMT

As a well established tradition, this year’s SQL Saturday #538 Sofia is again in October. Just last week the organizers published the event schedule and as usual, there will be a ton of sessions to choose from and a lot to learn. I had the pleasure to participate in the session selection and I must admit I had tough time choosing the best ones. But in the end, I think we did a good job in selecting the best of the best in accordance to the trends and speaker profiles. So now we have 4 tracks, 24 presentations, more than 20 speakers amongst which Microsoft engineers, MCMs, MVPs and most importantly – passionate SQL Server enthusiasts. And if you are still not convinced if you want to attend it, take a look at the schedule. More than 300 already registered and there are still couple of weeks until the event itself (15th of October), so hurry up and take one of the last tickets available! I will also have the pleasure to share my passion and knowledge on SQL Server. I had difficult time in deciding what topics to submit and what I want to share with you. In the end, I will be talking on data loading best practices – a topic I recently dug really deep with one of the customers (and SQL Server enthusiasts) I am working with. So…REGISTER NOW and see you all there (I will be there all day long, so if you just want to talk anything SQL Server-ish, I will be your guy )

Estimating storage requirements for encrypted columns–AE learning series part 04

Ivan Donev [MSFT] — Tue, 06 Sep 2016 07:28:33 GMT

As we discussed in earlier posts and as you might already be aware, AlwaysEncrypted columns require additional storage after they’re encrypted. Of course, that means that you need to plan properly as this might cause effects to your database, such as data file growth, fragmentation, etc. So, how exactly can you calculate the space, that your encrypted column would require? There is a formula, that gives you a precise number of bytes for each encrypted cell. That formula is based on the cypher algorithm, used to encrypt the value.

1 + 32 + 16 + (FLOOR(DATALENGTH(cell_data)/16) + 1) * 16

Example: source column is DECIMAL(29,2) – as the precision here is 29, so the bytes required to store the value are 17.

1 + 32 + 16 + (FLOOR(17/16) + 1) * 16 = 81

There is also a reference table on fixed data types in our MSDN article on the subject - Always Encrypted Cryptography Based on this formula, you can now evaluate your additional requirements and be prepared.

SQL Server instance not starting–failing with ex_handle_except encountered exception C06D007F

Ivan Donev [MSFT] — Fri, 26 Aug 2016 19:18:00 GMT

Couple of days ago I was on one of my regular customer engagements and as always, there was some proper fun with SQL Server that I had to get my hands dirty with. So the situation is the following: a highly consolidated environment of SQL Server 2012, 1014 and 2016 standalone instances with Always On Availability Groups, syncing to a secondary site. So far so good. Everything was in production, working without any issues until one day, there was a requirement to install another pair of instances for a new application. The requirement was for a SQL Server 2012 with latest SP (in our case - SQL Server 2012 SP3). The customer starts with the installation of RTM and everything goes well. He then proceeds with applying SP3 and all of a sudden - the patch process hangs and ultimately fails, leaving the instance in an inconsistent state. The fun thing was, that this entire installation and patching went perfectly fine on the second machine. During our investigation, we noticed the following symptoms:

During the patch, the SQL Server instance was not able to start and the Windows log was filled with exceptions like below

The SQL Server error log contained the following error

So as it seems, due to the patching process, there was something preventing the instance from starting up. And that something was causing a stack dump. So…I basically had three options:

Open a case to Microsoft Support so they can analyze the stack dump
Try to analyze the dump myself (that is fun, but also time consuming and not something that is fun while onsite with a customer)
Scan whatever knowledge base/internet article I can find

I tried to start with the third option first. Naturally I found number of articles with the exception code, pointing to a stack dump generation and here and there some third party tools, messing the SQL Server memory stack. My customer had only antivirus there, so naturally I went for stopping it immediately, but that did not help. One of the many good things when you work for Microsoft is, that you get to access our internal database with all the customer support cases, so that was my next attempt. Besides all the cases with some third-party tools, preventing the startup, I noticed couple of situations, where the SqlServerSpatial110.dll causing some issues due to wrong versions. WRONG VERSIONS?!? Yep, I immediately went for checking this DLL. As it turns, there is one DLL for every SQL server version, installed on the machine, which is stored at C:\Windows\system32 folder: As it turned out, the SqlServerSpatial110.dll for some reason was the version, coming from SQL Server 2014. Great! So, I found a SQL 2012 SP3 version of that DLL (2011.110.6020.0), replaced it and everything was going well. I am not sure at this point if this would help anyone else, having the same exception, however I guess if you ever hit it, it is worth trying!

Why SQL 2016 rocks – the series (part 2)

Ivan Donev [MSFT] — Mon, 23 May 2016 07:23:00 GMT

Why SQL 2016 rocks – the series (part 1)

Ivan Donev [MSFT] — Fri, 20 May 2016 21:06:17 GMT

SQL Server 2016 rocks! There is no doubt about it. At least not with me! I have so many cases where just by upgrading to the new engine you get performance improvement in the range of 10s of percents. And this is even before we hit general availability.

The things that are improved are so many, that for me it was really difficult and time consuming to follow everything. So I bookmarked, and bookmarked and bookmarked article after article. So in the end I end up with a huge list of things I went through or I still have to go through. And I decided – why not to share those resources. You probably have them, you probably not, but I will try to do a simple catalogue, so you can easily follow subject by subject. One note – I will mostly share articles by SQL Server CSS team or the different Program Managers, responsible for the feature development of the product. And I am sure there are a lot more articles by the community.

So let’s start. The first topic is Operational analytics which under the hood is columnstore technology allowing you now to have updatable NCCI over traditional indexes and that NCCI can even be filtered! In the last week I was able to test it over an actual data scenario (although only sample of the data in the test environment), but the results were really promising – a simple NCCI over an OLTP table of helpdesk transactions table gave of improvement of 25%. So I am excited. And here are the articles and demos if you want to further dig deep.

And there is the list:

Real-Time Operational Analytics: Simple example using nonclustered clustered columnstore index (NCCI)

Real-Time Operational Analytics: Compression Delay Option for Nonclustered Columnstore Index (NCCI)

Real-Time Operational Analytics: Filtered nonclustered columnstore index (NCCI)

Enjoy reading!

Permissions required to run Query Store in SQL Server 2016

Ivan Donev [MSFT] — Wed, 10 Feb 2016 12:14:00 GMT

If you wondered what is the minimum permission set you need to have in order to use Query Store in SQL Server 2016, then here is the answer: you need VIEW DATABASE STATE. I tried overcoming it by giving read permissions to all the stored procedures, DMVs and DMFs but to be honest – I did not expect it to work…and it didn’t :-) So, how to grant VIEW DATABASE STATE permission – well the quickest way is T-SQL:

USE MyQDSDB
GO
GRANT VIEW DATABASE STATE TO [QDS_USER];

VIEW DATABASE STATE will not provide permissions to force execution plans, flush the store, reconfigure it, etc.! For that, you will need to add the user to the db_owner fixed database role. Note: This post is tested in CTP 3.2 of SQL Server 2016 and might not be valid when the product goes RTM. Edit(20160213): Thanks to ErikEJ for pointing at the error in my script.

Plan cache pollution or how important it is to properly define parameters in code

Ivan Donev [MSFT] — Sat, 09 Jan 2016 02:26:00 GMT

Recently I was involved in couple of cases concerning plan cache taking more than 20% of the SQL Server Buffer Cache and although there is no fixed threshold/best practices value, I noticed that there were a lot of identical statements getting cached. Effective plan caching is important for optimal performance on SQL Server and I will not get into details why. If you want to further read on plan caching, there is a great whitepaper on MSDN – Plan Caching and Parametrization

Back to my scenario – the plan cache was looking very similar to this:

So I have absolutely the same query, cached three times with one difference only – the parameter definition of the CourseID parameter. So what was the reason to have this behavior?

In order to reproduce it, I started with a very simple console application, using Visual Studio Samples. Based on that code, I created couple of methods, so I can reproduce different behaviors. What I noticed is that this behavior is only applicable on parameters with variable length (varchar, nvarchar, etc.), that do not have the size defined in the code of the application itself. Here are the examples:

Using SqlParameter class without specifying the size

Using AddWithValue() Method

Using the SqlParameter class with specifying the size

What happens is that if you do not specify the size of your parameter and you are executing your queries adhoc (not using stored procedures), the client driver (in that case the ADO.NET library) is evaluating the size of the parameter at runtime and passes it to the SQL Server instance. So if you have 10 inputs with different sizes, then you will have 10 different cached plans.

If you notice in the cached plans, I have 4 different plans and only 1 of them is with nvarchar(11) – this is the one, executing the method, where I used SqlParameter class with explicitly defining the size of the parameter. Exactly this plan always gets reused (usecounts=4). The others will only be reused if the input parameter at runtime is with the same size of any of the cached plans. But in real world, that would be rarely the case.

Note that OptimizeForAdhocWorkloads parameter will not help in this case!

Conclusion: if you are sending dynamic/adhoc queries from your application to the SQL Server instance, make sure to properly define the size of your variable parameters. If you are using int for instance, this behavior will not be in place