(The new solution can be downloaded here)

(The detailed Information of the solution is mentioned here)

After having seeing the solution settled in numerous projects and getting great feedback from you, I added the following request to the solution.

Out-of-transaction support for Procedure Logging

While you might run the logging within your own crafted transactions, there might be situations where the transaction is rolled back due to a failure or within one of your test cases. In that case and because that the logging framework executes directly within the transaction context of the injected procedure, the log records won’t be preserved either. For this, I added the support of bringing in a linked server which is doing a non-participating transaction. The additional parameters which are outlined as below:

• @LogServerName: Defines the linked server name where the log data is pushed to, the server need to exists upon the injection of the procedure code.
• @LogDatabaseName: Defines the database name where the log data is pushed to, the database need to exists upon the injection of the procedure code.
• @PassCurrentProcessUser: If you are using a linked server not with delegated credentials and are using a fixed mapped user and password, the identity would change when the logging is persited. Therefore you can specify if an extra parameter value should be passed, identifying the original caller of the procedure.

While this is open source, you can change the code on your needs. If you though have a brilliant idea on how to make things better, let me know and we will get this back in with credits to yours.

-Jens

A very common request and question is (as I have to confess that this is confusing) is which version of SQL Server (SSMS - SQL Server Management Studio, BIDS - Business Intelligence Development Studio, VS- Visual Studio) does works together with which version of SQL Sever which target version and which project type. Here is a complete list to be updated through the versions of SQL Sever and the lifecycle of Visual Studio:

-Jens

Thanks to all the users, contributors and reviewers of the backup and restore procedure I once write for SQL Server databases.

If anyone does not know about this, more information can be found here (https://blogs.msdn.com/b/jenss/archive/2011/09/08/bulk-backup-and-restore-at-the-tip-of-your-finger.aspx)

I revamped the procedure to be agnostic to SQL Server >2012 as well as the schema changed in the meantime.

The new version of the code can be downloaded here.

-Jens

PS: Thanks to:

• Christopher Wang
• Josh Obudzinski

The most recent version can be found here

(The detailed Information of the solultion is mentioned here)

I created a minor update for the logging functionality in order to find the right entry point for injecting the code.

It turned out that finding the occurence of the beginning of the body text does not work if supplement characters are present after the “AS” keyword, stopping the whole process of applying logging to all procedures. I now check the occurence and throw an error if the statement is not formed as expected, but will proceed with remaining procedures.

-Jens

Update( 30.06.2014):

Beside the below mentioned information, I have written an updated compatibility sheet for the avaialable combinations or Visual Studio / SQL Server and the BI templates which is published here.

We waited and hoped and now we finally have them. SQL Server BI project templates in Visual Studio 2012.

This includes

• Project templates for
  • SSAS Analysis services projects (.dwproj)
  • SSRS Reporting services projects (.rptproj)
  • SSIS Integration services projects (.dtproj)

More Information about this can be found here: https://blogs.msdn.com/b/sqlrsteamblog/archive/2013/03/06/sql-server-data-tools-business-intelligence-for-visual-studio-2012-released-online.aspx

The direct download link is

Microsoft SQL Server Data Tools - Business Intelligence for Visual Studio 2012 (https://www.microsoft.com/en-us/download/details.aspx?id=36843)

Be aware of the following:

• Select “New instance” on the installation dialog if you have SQL Server x64 installed. Otherwise, you probably get the error "The CPU architecture of installing feature(s) is different than the instance specified. To continue, add features to this instance with the same architecture."

A big thanks to the team making this happen !

-Jens

Testing with realistic parameter sets

Ever wanted to test your SQL Server Reporting Services infrastructure against a realistic workload? Most often the problem is where to get the realistic data (parameter sets, a good weight of the different rendering formats) from.

You already have the data

If you deployed an environment, you already hold the solution in your hand. You can grab the information from the execution log of the Reporting Services database and generate requests to the new environment. Doing this you need access to the view dbo.ExecutionLog3 which can be found in the ReportServer content database of SSRS. Without going into too much details as already described in the MSDN article https://msdn.microsoft.com/en-us/library/ms159110.aspx you query some of the attributes stored upon every execution of a report, compose your WebRequest and fire it against the Reporting Services WebService.

You already have the tool

With this you will be able to replay certain reporting service data workloads with historic report executions. To make it just easier for you, I created a powershell script being able to point to any database having the dbo.ExecutionLog3 view and do the work. If you don´t have direct access to the ReportServer meta data database, you can also point to any database containing a copy / materialized view of the table, making it possible for administrators to give you access to the data but not the complete database.
As you see, you have some parameters to start the application, the parameters are also described in the SplashScreen of the application itself.

THe following paramaters can be used:

As the application can be simply started without having any prerequisites, the only requirements is to have powershell installed (most of the clients do have powershell) as well as access to the mentioned Reporting Services ExecutionLog3 table / view. With that, the application can be even started in multiple sessions and computer putting some workload on your servers 

This is a version 0.2.0, so work is still in progress, if you have any questions or any useful additions to it, let me know.

(Download the version here)

-Jens

While becoming familiar and a really big fan of PowerShell, I faced the following issue. During a migration project doing bulk migrations, I needed information Messages (coming to StdOut) as well as error message (coming to StdErr) displayed to the Console. Well that is quite easy as being the standard behavior. The challenge was, that error in addition needed to be duplicated in an error file, that in case of an error you can simply grab all the non 0kb files and do through them in order to investigate the errors.

It sorted out that this wasn´t easy at all as the output once being redirected cannot be easily duplicated.

There were actually solution (which I could post as well if you are interested) including the tee object as well as the option to create a wrapper script, redirect the error output, read it at the end of the execution and write it to the console. But this wouldnß´t be as natural as having the error message at the exact place where it happens (which could somewhere in between).

My colleague Dongbo Wang then pointed me in the right direction. For an easier repro as well as giving him the credits for this idea, here is what we came up with:

The sample execution file contains several command, simulating normal messages as well as an error message:

Write-Output "Before the error" 

Write-Output "================" 

Write-Error "Here is the error" 

Write-Output "================" 

Write-Output "After the error"

For this sample we should be able to see the error message after the "Before" message and Before the after message.

So save the above made statements to a script file psscript.ps1 and execute it with the following command:

.\psscript.ps1 2>&1 | % { if ($_ -is [System.Management.Automation.ErrorRecord]) 

{ $_ | Out-File -Append .\Errors.txt } $_ }

So what happens here ? The messages (being objects) are piped and checked for their message type; if being of type [System.Management.Automation.ErrorRecord] they are collected and appended in a loop to the specified output error file. So what we get here is the output of

and the error message within the created text file Errors.txt:

-Jens

I will write everyone who directed me directly again, but for those watching the blog, the ressources linked in the blog entries are online again..finally. Thanks for all your patience !

-Jens

(You find the script at the end of the blog post)

People who inherit databases encounter often the same problem. They either don´t have the time “learning” the internals / logic of the existing objects within that database, or the learning curve and the time needed is too high for getting started. So cleaning up databases or checking for internal consistency is avoided in order “not to touch a running system”.

So, simple things get out of focus like

• Is the object used anymore by some internal database business logic ?
• Is the object being referenced by any objects in the database ?
• Does the object have any reference in the database to objects that do not exist anymore ?
• When was the table accessed the last time ? Am I able to delete it as not being referenced for a decent amount of time ? (Will be covered in one of the next blog posts)

To be clear, due to the existing dynamic nature of TSQL, not all questions can be answered exactly. But for the object information being modelled by the engine, the questions we have to ask the engine and the answers can be quite easy. I will concentrate on some of the most asked questions here and provide some scripts for you in order to try them on your databases as well, finding the information easily.

Hello ? Does anyone care about me ?

This is a quite easy thing. Beginning with SQL Server 2008, there are special views showing the interactions between existing database schema elements like tables / views / procedures. Without going into detail, I would encourage you to take a look at the system view sys.dm_sql_referencing_entities (MSDN) as in the following sample:

Yay, easy, right ? I see that the SomeTable table is referenced by the view SomeView and that I should not easily drop it before checking with the creator / owner of the view.

Here come the part with the dynamic TSQL culprit. As you can compose your TSQL statement on the fly and it would be compiled at runtime, the view will not find any references to statements like EXEC(‘SELECT SomeCol FROM dbo.SomeTable’) as string will not be parsed for correctness. So checking with the table is only one step of checking for the references. If you are sure that nor internal logic of the database (like procedures) are doing not dynamic stuff, as well as not external APIs are creating access to the object, you should be cautious to delete it at this point.

Me ? I don´t care about anyone else !

Well, let´s see if that is true. For this, the existing view sys.dm_sql_referenced_entities (MSDN) can be used. It will show all the objects that are being referenced by the passed in object. For our sample above, this would turn out to be:

You see that the referenced table itself as well as the used columns are mentioned in here.

So lonely here… Where have all the objects gone ?

Now you know how to find the referenced as well the the referencing objects. But what if I pull the rug / objects from under the referencing objects ? For schema bound objects (objects created WITH SCHEMABINDING) the answer is easy.. you can´t. But due to the restrictions you have with this option most of the objects in the real world are created "loosely" coupled without schema binding. If someone removes the referenced object from the referencing object, noone would ever know unless you execute / query the referencing object at runtime.

For a view you could find that easily using the sp_refreshview procedure. This will show you the following error:

But what about procedures, functions, triggers ? You can mark them for recompilation with sp_recompile, but this simply means that the query plan is thrown away and needs to be reevaluated before the next run.

Fortunately you have already learned one of the methods above to figure these things out. You can use the views mentioned above, as they will throw an error if any inconsistencies will be found. Wrapping this in a loop to go through all the objects as well as wrapping this into a TRY / CATCH block will get you all of the information you need.

At the end going through the views / procedures / functions and triggers, the script will output something similar on your databases:

Not only that the script will give you the Information about the objects that have wrong references, it will also give you the script to do a specific query finding the exact Information:

(The result)

(The Messages)

If you have any questions, let me know. The script is attached to the blog entry. Have fun and let me please know your results ! Looking forward to your experience with this ! Simply drop me a mail through the comment system or JensS(AtSymbolHere)Microsoft.com

-Jens

FindMissingReferences.sql

One big advantage over recent database projects is the ability to do headless build on a build controller.

There is no straight forward definition of a headless build. In summary you can describe it that only the needed prerequisites are installed for compiling and translating the bits. While for some compilations specific applications will have to be installed or even cannot be build without the IDE, headless builds use the builds machine build processor (for MSFT mostly msbuild.exe) and pulls only the needed .targets / satellite assemblies.

Doing headless builds need a specific set of assemblies installed and registered assemblies on the build controller. As there can be some problems associated preparing the prerequisites, this (and former) blog posts will emphasize some of them or at least those were you won´t be able to find an easy solution by yourself.

A good starting point for installing the prereqs is the following blog post. It provides you with the needed Basic for the best walkthrough.

DACFX Sept 2012 Updates Break Headless Build

Though, there can be some culprits and error messages you can run into, which do not seem to be obvious in the error messages at all. The following series will contain some of the problem you might encounter and will not be able to find simple solutions on the internet for.

Referencing dacpacs

You encounter the following error:

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets (465): File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\SQLDB\Extensions\SqlServer\110\SqlSchemas\master.dacpac" does not exist.

This works on your Visual Studio machine as you have full blown set of files installed on the machine. On build controllers / the Team Foundation Server you normally don´t find them if you don´t have Visual Studio (and I don´t mean the Shell) installed. The source of the mentioned directory is constructed from your .sqlproj file containing the references:

  <ItemGroup>
    <ArtifactReference Include="$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\110\SqlSchemas\master.dacpac">
      <HintPath>$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\110\SqlSchemas\master.dacpac</HintPath>
      <SuppressMissingDependenciesErrors>False</SuppressMissingDependenciesErrors>
      <DatabaseVariableLiteralValue>master</DatabaseVariableLiteralValue>
    </ArtifactReference>
    <ArtifactReference Include="$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\110\SqlSchemas\msdb.dacpac">
      <HintPath>$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\110\SqlSchemas\msdb.dacpac</HintPath>
      <SuppressMissingDependenciesErrors>False</SuppressMissingDependenciesErrors>
      <DatabaseVariableLiteralValue>msdb</DatabaseVariableLiteralValue>
    </ArtifactReference>
  </ItemGroup>

You see that a variable called $(DacPacRootPath) is used within the reference. This will be resolved by the above mentioned .targets file (C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets) by the following conditions:

<!--Calculation of the system dac install path--> 
< PropertyGroup> 
   <DacPacRootPath>$(VsIdePath)</DacPacRootPath> 
< /PropertyGroup>

which translates to the V(isual)S(tudio)IDEPath (VSIdePath)

<!--Calculation of the VS install path-->
 <PropertyGroup Condition="'$(VisualStudioVersion)' == '10.0'">
   <VsIdePath>$(Registry:HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\10.0@InstallDir)</VsIdePath>
   <VsIdePath Condition=" $(VsIdePath) == ''" >$(Registry:HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\VisualStudio\10.0@InstallDir)</VsIdePath>
 </PropertyGroup>
 <PropertyGroup Condition="'$(VisualStudioVersion)' == '11.0'">
   <VsIdePath>$(Registry:HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\11.0@InstallDir)</VsIdePath>
   <VsIdePath Condition=" $(VsIdePath) == ''" >$(Registry:HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\VisualStudio\11.0@InstallDir)</VsIdePath>
 </PropertyGroup>

  

The target is first searching for the existence of the regkey in the x64 path, id empty in the x86 path (WowNode) (at least for VS2012 = 11.0)

If you don´t have the regkey set, you can set it to the appropriate one like on your dev machine, pointing to e.g. “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE”. Under this directory you can create the file path as in the error message or simply copy the appropriate Folder to your build machine containing the needed .dacpacs.

After copying, MSBuild will be able to find the references and at least for this path would be able to resolve correctly.

Another option would be to always copy your referenced system dacpacs to the local folder.

-Jens