I had a request to update my Visual Studio 2008 GDR Database projects CRUD generator to Visual Studio 2010. Not much has changed except for some references and directory locations. The flow of the add-in is identical to the 2008 version. Below are the pertinent links.

Original blog entry

Source Code

Visual Studio 2010 Database Projects CRUD Generator MSI

Please ping me if there are any issues. Thanks!

-- Patrick

In this blog I’ve created a project feature extension to combine Table and View child scripts into the script that defines the parent object. For example, constraints or extended properties for a table. Some developers like to have all the T-SQL scripts defining a particular object in one .sql file. Typically you’d run this after using Import Schema to populate your database project. The package is built using a vsix file as the deployment mechanism and can be downloaded here. The source for this project is located here.

Overview of the Package

Adding the button to Schema View

The package contributes a ‘Merge scripts into parent schema object’ menu button onto the schema view context menu for Tables and Views. When pressed the selected Tables and Views will gather their hierarchical statements into the parent script. If the child script is emptied of content then it is excluded from the project . If your project is under version control you may wish to review the output window and delete these files from your SCC provider.

Overview of the Code
One of our requirements is to add a button to the Schema View context menu.  In order to do that we’ll need to create a Visual Studio package.  To create Visual Studio packages you’ll need to install the SDK located here.  Once installed, you’ll have the option to create a new “Visual Studio Integration Package” project based on the vsix format.   See the file ‘MergeScripts.vsct’ for details on how I added this button on the schema view context menu. 

Implementing the database project feature extension

I also need to hook into the Visual Studio 2010 database project extensibility. To create extensions to a Visual Studio 2010 database project you must

• Create a class which implements one of the extension point interfaces. All these interfaces ultimately derive from Microsoft.Data.Schema.Extensibility.IExtension.
• Publish your extension via a file named <MyFile>.Extensions.xml file which can be placed in the %ProgramFiles%\Microsoft Visual Studio 10.0\VSTSDB, %ProgramFiles%\Microsoft Visual Studio 10.0\Common7\IDE or the %LocalAppData%\Microsoft\VisualStudio\10.0<hive>\Extensions directory.
• Your extensions file must correspond to the following schema:

<?xml version="1.0" encoding="utf-8"?>
<xsd:schema xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns="urn:Microsoft.Data.Schema.Extensions" elementFormDefault="qualified" attributeFormDefault="unqualified" targetNamespace="urn:Microsoft.Data.Schema.Extensions">
  <xsd:element name="extensions">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="extension" minOccurs="0" maxOccurs="unbounded">
          <xsd:complexType>
            <xsd:attribute name="assembly" type="xsd:string" use="optional"/>
            <xsd:attribute name="type" type="xsd:string" use="required"/>
            <xsd:attribute name="enabled" type="xsd:boolean" use="required"/>
          </xsd:complexType>
        </xsd:element>
      </xsd:sequence>
      <xsd:attribute name="assembly" type="xsd:string" use="required"/>
      <xsd:attribute name="version" type="xsd:unsignedByte" use="required"/>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

The database extension created for this project is called MergeScriptsFeature and is published in the MergeScripts.Extensions.xml:

<?xml version="1.0" encoding="utf-8"?>
<extensions assembly="" version="1" xmlns="urn:Microsoft.Data.Schema.Extensions" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:Microsoft.Data.Schema.Extensions Microsoft.Data.Schema.Extensions.xsd">
  <extension type="MergeScripts.MergeScriptsFeature" assembly="MergeScripts, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0cae59df434c3018"  enabled="true" />
</extensions>

MergeScriptsFeature has the following declaration:

[DatabaseSchemaProviderCompatibility(typeof(SqlDatabaseSchemaProvider))]

class MergeScriptsFeature : IDatabaseProjectFeature, IOleCommandTarget

The DatabaseSchemaProviderCompatibility attribute identifies the database schema provider affinity. In this case “SqlDatabaseSchemaProvider” is used indicating that this feature will work with all SQL database projects.

When a database project is opened it queries for all extensions compatible with the current database schema provider which also implement IDatabaseProjectFeature. Instances of each of those features are instantiated and initialized with the IDatabaseProjectNode for that project. With this node, features can listen to events – in my case I listen to ProjectClosed and FileIconRequest.

The MergeScriptsFeature will need to chain into the IOleCommandTarget implemented by the Database Schema View. To do this the feature retrieves the ‘SchemaModelViewerService’ from the agnostic database package – called ‘DataPackage’. Once that service raises the DatabaseSchemaViewInitialized event (or if the Schema View is already displayed) the MergeScriptsFeature IOleCommandTarget is chained through the method IDatabaseSchemaView.RegisterForLimitedCommandRouting.

When the merge button is pressed MergeScriptsFeature.DoMerge() is called. This routine creates a CommandFactory and proceeds to buffer up the source code control and file operations necessary to do the actual merge. Once all these commands are buffered they are ordered and executed in CommandFactory.Execute().

Installing the Package

Installing a vsix package can be done on a per-user basis or for all users on the machine. To install as a specific user just double-click on the .vsix file. This will unzip the contents and place them into “%LocalAppData%\Microsoft\VisualStudio\10.0Exp\Extensions\Microsoft Corporation\MergeScripts\1.0”. The next time Visual Studio is launched the contents of this package (like menu definitions) will be merged into the Visual Studio shell. To verify the vsix extension is installed bring up the Extension Manager available under the main menu “Tools.Extension Manager”. You should see the MergeScripts package.

To install the vsix package for all users (called a Trusted Extension) you need to have administrator rights on the machine. You can then install using the “VSIXInstaller /admin” command. This will install the extension into %VSInstallDir%\Common7\IDE\Extensions.

Conclusion

I know an extension like this (or an option during Import Script) has been requested multiple times on our forums. Until we get around to adding the option into Visual Studio I hope this extension will service your needs. If you have any questions please visit our forums here or contact me via this blog. Thanks!

-- Patrick

--*CREATE TABLE [dbo].[$(MyVar)]
--*(
--* column_1 int NOT NULL, 
--* column_2 int NULL
--*)

Into valid Transact-SQL

CREATE TABLE [dbo].[MyReleaseTableName]
(
column_1 int NOT NULL, 
column_2 int NULL
) 

I’ve updated the add-in to work with Visual Studio 2010. The essential details haven’t changed so please refer to my past post. If you wish to recompile the source you’ll need Visual Studio 2010 (professional or ultimate) and the Wix Toolset.

The source code for the Visual Studio ProcessSqlTemplate MSBuild Task and Add-in is here.

The ProcessSqlTemplateAddIn MSI is located here.

• Import Script. Unfortunately this will shred your scripts for schema objects and move any conditional logic into the scriptsignored file.
• Add Existing Item. You could create a Visual Studio 2010 project and use “Add Existing Item” to bring your .dbp scripts into your new project. The downside with this is 1) it is time consuming and 2) you lose the workspace the version control provider needs – therefore loosing history of your files.
• “Include In Project”. Create a new Visual Studio 2010 project and drag your .dbp project folder structure on the top of the .dbproj file. After your finished select “Show All Items” from the Visual Studio 2010 Solution Explorer and then “Include In Project” for your .dbp files. Unless you create the new .dbproj project over the top of your .dbp project you’ll lose version control history.

DbpToDbproj.exe

To help Database Project (.dbp) customers with the upgrade to Visual Studio 2010 I’ve written DbpToDbProj. The source and executable is located here.

This project builds a command-line executable with parameters allowing you to create either a Visual Studio 2010 2005DSP or 2008DSP project, and indicate if the file structure for the ‘Schema Objects’ folder is ‘by schema’ or ‘by type’.

   1: c:\Temp>DbpToDbproj.exe /?

   2: Database projects that have the file extension .dbp have been deprecated in Visual Studio 2010

   3: DbpToDbproj creates a database project with the .dbproj extension to replace the dbp.

   4:  

   5: DbpToDbproj [drive:][path]filename [/target:[2005|2008]] [/DefaultFileStructure:[ByType | BySchema]]

   6: [drive:][path]filename

   7: Specifies drive, directory and filename for the dbp file to be converted.

   8:  

   9: /target:[2005|2008]

  10: Specifies whether the .dbproj to be created is a Sql2005 or a Sql2008 project.

  11:  

  12: /DefaultFileStructure:[ByType | BySchema]]

  13: Specifies whether the directory structure of the .dbproj should be by object schema or type.

Specifies whether the directory structure of the .dbproj should be by object schema or type.

As an example, here’s a .dbp project from Visual Studio 2008:

Running DspToDbproj in the same directory as your existing .dbp project will create a file called ‘<dbpfilename>.dbproj’.

   1: C:\Temp\dbpProject\dbpProject>DbpToDbproj.exe dbpProject.dbp

   2: Converting...

   3: C:\Temp\dbpProject\dbpProject\dbpProject.dbp

   4: ...to...

   5: C:\Temp\dbpProject\dbpProject\dbpProject.dbproj

   6: Successfully converted dbp to C:\Temp\dbpProject\dbpProject\dbpProject.dbproj

When the .dbproj file is opened you’ll receive an upgrade prompt.  The reason for this is that my command-line utility does not interact with your Version Control system.  Instead I place an MSBuild property in the dbproj file called “PostUpgradeAddToSCC”.  This property is a flag to the database project system that upgrade has added new files and that these new files should be added to SCC.  This only happens after the upgrade process so I had to make the system think an upgrade was necessary.  If you don’t use version control you can edit Program.cs and remove the line where I create the “PreviousProjectVersion” MSBuild property.  The absence of this property will prevent the upgrade wizard from appearing.

After opening the .dbproj file the following project will appear in the solution. Notice that the script files are maintained as they were in the .dbp project. Each of these script files have a build action of ‘Not In Build’, meaning they do not contribute to the dbproj model. It also creates the standard “Schema Objects”, “Data Generation Plans”, and “Schema Comparisons” folders you’d get with any other Visual Studio 2010 project. If you’re using the new Database Project (.dbproj) project for purely version control you can always delete these directories – otherwise you’ll still be able to import from an existing database or a script.

DspToDbproj will preserve the connections found in the .dbp by writing them into a file called Connections.txt. From these connection strings you should be able to recreate the connection in the Server Explorer manually.

Conclusion

Hopefully this will make the conversion to Visual Studio 2010 a little easier for everyone with .dbp projects. Once again the source and executable is located here. If you have any questions please visit our forums here or ping me through this blog.  Thanks!

-- Patrick

In the application I was debugging it was making use of binary deserialization to reconstruct a graph of objects. Many of these objects needed some additional context so a StreamingContext object was created with an additional context state. Here’s a sample of what the code was doing:

   1: using (FileStream fs = myfile.OpenRead())

   2: {

   3:     ContextObject ct = new ContextObject();

   4:     ct.Name = "testing";

   5:     StreamingContext context = new StreamingContext(StreamingContextStates.Other, ct);

   6:     BinaryFormatter bf = new BinaryFormatter();

   7:     bf.Context = context;

   8:     SerializableObject obj = (SerializableObject)bf.Deserialize(fs);

   9: }

Everything was working well until I noticed that deserialization would consistently increase the memory footprint (recognize that, in my case, the additional context object was quite large). I wrote a sample application analyzed the problem using WinDBG. The sample program is located here.

Open WinDBG and select “Open Executable”. Select LeakApplication.exe and hit F5. LeakApplication will run waiting for console input. Select ‘Break’ in WinDBG and type “!dumpheap -type LeakApplication.ContextObject”. You’ll see something like this:

   1: 0:003> !dumpheap -type LeakApplication.ContextObject

   2:  Address       MT     Size

   3: 01f6b7b0 0042340c       12     

   4: total 1 objects

   5: Statistics:

   6:       MT    Count    TotalSize Class Name

   7: 0042340c        1           12 LeakApplication.ContextObject

   8: Total 1 objects

WinDBG is telling us that 1 LeakApplication.ContextObject may be in memory. In order to determine if this is accurate type “!gcroot 01f6b7b0”. Here are the results:

   1: 0:003> !gcroot 01f6b7b0 

   2: Note: Roots found on stacks may be false positives. Run "!help gcroot" for

   3: more info.

   4: Scan Thread 0 OSTHread 12f0

   5: ESP:20ef20:Root:01fcb148(System.Runtime.Serialization.Formatters.Binary.BinaryFormatter)->

   6: 01f6b7b0(LeakApplication.ContextObject)

   7: ESP:20ef3c:Root:01f6b7b0(LeakApplication.ContextObject)->

   8: 01f6b7b0(LeakApplication.ContextObject)

   9: ESP:20ef40:Root:01f6b7bc(System.Runtime.Serialization.Formatters.Binary.BinaryFormatter)->

  10: 01f6b7b0(LeakApplication.ContextObject)

  11: ESP:20ef54:Root:01f6b7b0(LeakApplication.ContextObject)->

  12: 01f6b7b0(LeakApplication.ContextObject)

  13: Scan Thread 2 OSTHread a28

  14: DOMAIN(002F1DE8):HANDLE(Pinned):4013fc:Root:02f51010(System.Object[])->

  15: 01f6ad88(System.Collections.Generic.Dictionary`2[[System.Runtime.Serialization.MemberHolder, mscorlib],[System.Reflection.MemberInfo[], mscorlib]])->

  16: 01f6ae5c(System.Collections.Generic.Dictionary`2+Entry[[System.Runtime.Serialization.MemberHolder, mscorlib],[System.Reflection.MemberInfo[], mscorlib]][])->

  17: 01f6cb5c(System.Runtime.Serialization.MemberHolder)->

  18: 01f6b7b0(LeakApplication.ContextObject)

Sure enough the BinaryFormatter is holding onto the LeakApplication.ContextObject. This is a known leak and, unfortunately, the best you can do is to mitigate the problem. In my case I created a small, disposable property bag object to hold onto my larger “additional” context objects. After deserialization simply call Dispose on the property bag and let it NULL out any members, thus clipping the object graph. Your property bag will still leak, but at least your larger object graph will be available for garbage collection.

-- Patrick

For this blog I found two exceptional resources. Dr Ex. has a good blog post about the process for contributing to the Visual Studio fonts and colors dialog located here. There was also a post by Matthew Etter located here about creating a derived RegistrationAttribute to construct the necessary Visual Studio keys. Going from those postings to an actual working contributor is still a bit of a challenge though.

The sample involves one package which contributes a “FontAndColorCategory” to the Options dialog. It also contributes one color called “MyColor”.

The aforementioned blog posts and msdn help should be able you to navigate and understand the source so I’m not going to repeat anything here. Hopefully a fully working example will expedite your own development efforts.

· The full source for the fonts and color provider can be downloaded here.

-- Patrick

Pre-Build Events

With this mechanism you’re swapping in new source files prior to each build. In the sample solution below I have two Visual Studio Configurations – ‘Sandbox’ and ‘Integration’. For each security object I’ve created 2 new files and navigated to the Properties Window to change their Build Action to ‘Not in Build’.

The pre-build event action is found in the project properties ‘Build Events’ tab:

In the pre-build textbox I’ve inserted a script for swapping in new security objects just prior to the build.

   1: if [$(Configuration)] == [Integration] (

   2:     xcopy /r /y "$(ProjectDir)User1.user.integration.sql" "$(ProjectDir)User1.user.sql" 

   3:     echo copied "$(ProjectDir)User1.user.integration.sql" to "$(ProjectDir)User1.user.sql" 

   4: ) else if [$(Configuration)] == [Sandbox] (

   5:     xcopy /r /y "$(ProjectDir)User1.user.sandbox.sql" "$(ProjectDir)User1.user.sql" 

   6:     echo copied "$(ProjectDir)User1.user.integration.sql" to "$(ProjectDir)User1.user.sql" 

   7: ) else (

   8:     del "$(ProjectDir) User1.user.sql" 

   9: )

Note that I’m using “xcopy /r” which overwrites the target file even if it’s marked readonly. I consider this a reasonable approach because, even if you do have this file in version control, it contains no useful information. The interesting, versionable information actually resides in the .sandbox.sql and .integrationl.sql files.

Advantages

• Object references in the schema model will be validated during the build process
• Deploy will accurately compare the identifiers for security objects between the project and the target database

Disadvantages

• Refactoring the security object will not modify the definition in the configuration specific scripts.
• Schema Compare will compare the identifier defined in the script which is “In Build” to the target. This may result in unexpected differences.
• If you need to create a new Visual Studio configuration for each target SQL server you will be deploying to.

Post-Deployment File

Another approach to creating security objects is to place all your security-based T-Sql in your Post-Deployment Script.

Here’s an example of a Post-Deployment script used in this way:

   1: /*

   2: 

   3: Post-Deployment Script Template

   4: 

   5: */

   6:  

   7: IF (('$(DatabaseName)') = 'TR9_Database_Dev')

   8: BEGIN

   9:     PRINT N'Creating Sandbox Security Objects...';

  10:     IF NOT EXISTS (SELECT * FROM sys.server_principals WHERE name = N'TR9-DEV\TR9_CONTROL')

  11:     BEGIN

  12:         CREATE LOGIN [TR9-DEV\TR9_CONTROL] FROM WINDOWS;

  13:         EXECUTE sp_addsrvrolemember @loginame = N'TR9-DEV\TR9_CONTROL', @rolename = N'dbcreator';

  14:         IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'TR9-DEV\TR9_CONTROL')

  15:         BEGIN

  16:             CREATE USER [TR9Control] FOR LOGIN [TR9-DEV\TR9_CONTROL];

  17:             EXECUTE sp_addrolemember @rolename = N'TR9_Control', @membername = N'TR9Control';

  18:         END

  19:     END 

  20:     IF NOT EXISTS (SELECT * FROM sys.server_principals WHERE name = N'TR9-DEV\TR9_USERS')

  21:     BEGIN

  22:         CREATE LOGIN [TR9-DEV\TR9_USERS] FROM WINDOWS

  23:         IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'TR9-DEV\TR9_USERS')

  24:         BEGIN

  25:             CREATE USER [TR9Users] FOR LOGIN [TR9-DEV\TR9_USERS];

  26:             EXECUTE sp_addrolemember @rolename = N'TR9_Users', @membername = N'TR9Control';

  27:             EXECUTE sp_addrolemember @rolename = N'TR9_Users', @membername = N'TR9Users';

  28:         END

  29:     END 

  30: END

  31: ELSE

  32: BEGIN

  33:     PRINT N'Creating Integration Security Objects...'

  34:     IF NOT EXISTS (SELECT * FROM sys.server_principals WHERE name = N'TR9-INT\TR9_CONTROL')

  35:     BEGIN

  36:         CREATE LOGIN [TR9-INT\TR9_CONTROL] FROM WINDOWS;

  37:         EXECUTE sp_addsrvrolemember @loginame = N'TR9-INT\TR9_CONTROL', @rolename = N'dbcreator';

  38:         IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'TR9-INT\TR9_CONTROL')

  39:         BEGIN

  40:             CREATE USER [TR9Control] FOR LOGIN [TR9-INT\TR9_CONTROL];

  41:             EXECUTE sp_addrolemember @rolename = N'TR9_Control', @membername = N'TR9Control';

  42:         END

  43:     END 

  44:     IF NOT EXISTS (SELECT * FROM sys.server_principals WHERE name = N'TR9-INT\TR9_USERS')

  45:     BEGIN

  46:         CREATE LOGIN [TR9-INT\TR9_USERS] FROM WINDOWS

  47:         IF NOT EXISTS (SELECT * FROM sys.database_principals WHERE name = N'TR9-INT\TR9_USERS')

  48:         BEGIN

  49:             CREATE USER [TR9Users] FOR LOGIN [TR9-INT\TR9_USERS];

  50:         EXECUTE sp_addrolemember @rolename = N'TR9_Users', @membername = N'TR9Control';

  51:         EXECUTE sp_addrolemember @rolename = N'TR9_Users', @membername = N'TR9Users';

  52:         END

  53:     END 

  54: END

Once again, since the objects do not exist as part of the project system model they cannot participate in the schema differencing process which occurs prior to deployment. Any errors in the objects or your script will not appear until you deploy.

Advantages

• All of your security objects are defined in one script.

Disadvantages

• Since the security objects are not defined in the database schema they will not participate in Refactoring or Schema Compare.
• You may need conditional in the script for each target SQL server you will be deploying to.  If your security objects are similar enough you may be able to use SqlCmd variables to instead of the IF/ELSE scheme I used.

Visual Studio Extensibility

Now I’ll introduce you to the Visual Studio Add-In ProcessSqlTemplateAddIn and associated MSBuild Task. These components use Visual Studio Extensibility to pre-process your T-Sql scripts and bind the value of SQLCMD variables to the respective values defined in the active project configuration. When you select a project configuration the MSBuild task scans the project for template script files. Any variables that are discovered in the templates are replaced with the respective value.

Install the Add-In and MSBuild Task using this msi. The MSI will install the following components:

• %ProgramFiles%MSBuild\Microsoft\VisualStudio\v9.0\TeamData\ProcessSqlTemplate.Task.Targets
• %ProgramFiles%Microsoft Visual Studio 9.0\Common7\IDE\PublicAssemblies\ProcessSqlTemplateAddIn.dll
• %ProgramFiles%Microsoft Visual Studio 9.0\VSTSDB\ProcessSqlTemplate.Task.dll
• %ProgramData%\Microsoft\MSEnvShared\Addins\ProcessSqlTemplateAddIn.AddIn

After installing, create a Server Project and in that project create a file called ‘Login1.login.template.sql’. The “template.sql” at the end of the filename indicates to the Add-In that this is a template script. Copy the following text to that file:

   1: --*CREATE LOGIN [$(SVR)\TR9_CONTROL] FROM WINDOWS

   2: --*GO

   3: --*CREATE LOGIN [$(SVR)\TR9_USERS] FROM WINDOWS

The “--*” indicates that this text is part of the template.

Next edit your project properties to create two new .sqlcmdvars file – one for your Sandbox and one for your Integration configuration. In the Sandbox .sqlcmdvars file define $(SVR) as “SVR-DEV”. In your integration .sqlcmdvars file define $(SVR) as “SVR-INT”.

Now we need to chain our new MSBuild task into the build process. As part of the ProcessSqlTemplateAddIn MSI a targets file was installed. This target file contains the definition of the ProcessSqlTemplate MSBuild task. Unload the project and insert the following Import statement into your project file after the ‘Microsoft.Data.Schema.SqlTasks.targets’ import statement:

<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v9.0\TeamData\ProcessSqlTemplate.Task.Targets" />

Reload the project and you’ll see that switching configurations will alter the contents of your ‘Login1.login.template.sql’ script according to the value of your sqlcmdvars for that configuration. A sample project is included here.

Before I leave this top there’s one advantage I’d like to explain. Because this approach is bound to your configurations SqlCmd variables file you are able to provide these variable values on the MSBuild commandline. To do this create a new ItemGroup in your project file with the name of the SqlCmd variable you may want to override:

   1: <ItemGroup>

   2:     <SqlCommandVariableOverride Include="SVR=$(SVR)" Condition="'$(SVR)' != ''"/>

   3: </ItemGroup>

With this ItemGroup in place I can now execute MSBuild supplying the configuration as well as an override for the $(SVR) sqlcmd variable. There’s a good forum post on this topic here.

msbuild /t:rebuild /property:Configuration=Sandbox;SVR=Patrick TR9_Database\TR9_Database.sln

Advantages

• The database developer only needs to implement one SQL script defining the security object.
• The identifier for the security object is represented as a variable that is bound to the correct value when the project is built.
• Deploy will correctly compare the project’s schema definition with the target database.
• It makes use of SqlCmd variables which are overridable on the command line.

Disadvantages

• The Add-In and MSBuild task need to be installed on each Developer’s computer as well as the TFS Build server.
• The .dbproj file must be modified to include the MSBuild target.
• Refactoring will not modify the schema template definitions.

Visual Studio Extensibility Source Code

The source code for the Add-in, MSBuild task and the installer is located here. To compile this you’ll need Visual Studio 2008, the Visual Studio 2008 VSSDK, and Votive 3.0 (I used 3.0.4707.0).

The MSBuild addin is located in the ProcessSqlTemplateAddIn project. Visit the source file Connect.impl.cs for the most interesting details of the implemention. Note that Connect implements IVsUpdateSolutionEvents and IVsSolutionEvents. These events help the addin listen to database projects opening/closing as well as when their configuration changes. When a configuration does change ProcessAllTemplates is called which calls on the MSBuild task to do the template processing.

The MSBuild task is located in the ProcessSqlTemplate.Task project. Take a look at ProcessSqlTemplateTask.Execute for its entry method.

Conclusion

In this blog we’ve discussed three possibilities for deploying server objects to different Sql Server instances from a single Visual Studio Solution. To install the msi or view the source code you’ll need Visual Studio Team System 2008 Database Edition GDR.

The source code for the Visual Studio ProcessSqlTemplate MSBuild Task and Add-in is here.

The sample project demonstrating how to use the Add-In is here.

The ProcessSqlTemplateAddIn MSI is located here.

Overview of the Add-In

The zip reference above contains an MSI called InstallCRUDGenerator.msi. Once you install, make sure the add-in is loaded by selecting the “Tools.Add-in Manager” menu button:

Once loaded any scripts containing tables will display a “CRUDGenerator” button in your Solution Explorer context menu:

If you select this the CRUD Generation Dialog will appear:

This dialog allows you to select your source table script and then view the generated CRUD stored procedures. If you’re happy with the generated procedures select “Save Procedures to Project” and 4 new stored procedures will be created in your project.

Now for the details! The text box in the middle of the CRUD Generation Dialog contains your T4 template. At the start of this blog I gave you a good T4 link so I won’t go into details on the scripting language itself, but let’s adjust the Create procedure seen in the image above. First let’s add my company name to the comments for this stored procedure. The comment is generated on lines 10 through 13 of the T4 Template:

   1: /*

   2: CREATE procedure for <#= tableFullName #>

   3: Generated <#= DateTime.Now.ToString() #>

   4: */

Let’s change this to

   1: /*

   2: Copyright (c) Microsoft Corporation. All rights reserved.

   3: CREATE procedure for <#= tableFullName #>

   4: Generated <#= DateTime.Now.ToString() #>

   5: */

.. and hit the “Generate” button:

Notice now that our copyright appears in the generated stored procedure T-SQL and we did this without modifying our add-in. Very cool!

Feel free to experiment and modify the T4 templates I provide in this add-in. There are 5 templates installed into “%ProgramFiles%\Microsoft Visual Studio 9.0 CRUD Template AddIn”:

· CrudCreate.tt

o Responsible for generated the ‘Create’ stored procedure

· CrudDelete.tt

o Responsible for generated the ‘Delete’ stored procedure

· CrudRead.tt

o Responsible for generated the ‘Read’ stored procedure

· CrudUpdate.tt

o Responsible for generated the ‘Update’ stored procedure

· CrudToolbox.tt

o Useful utilities for walking the Data Dude model. This is where the columns are analyzed to determine how they are used in the templates above.

CrudToolbox.tt contains the most code. It’s here where I walk the Data Dude object model to determine the column and column types for the selected table. If you wish to change this file remember that it too can be altered without having to restart Visual Studio. Just open it in a text editor or another Visual Studio instance, edit and then hit the ‘Generate’ button as we did before.

Overview of the Add-In Code

In this section I’ll provide you with an overview of the CRUD Generator Add-in code. For anyone that’s ever generated a Visual Studio Add-in project you’ll know that Connect.cs is where the add-in initializes. It’s there where I add the ‘CRUD Generator’ button to the Solution Explorer context menu and handle the button click. My handling is a simple pass-off to a class I call the CRUDEngine:

   1: public void Exec(string commandName, vsCommandExecOption executeOption, ref object varIn, ref object varOut, ref bool handled)

   2: {

   3:     handled = false;

   4:     if(executeOption == vsCommandExecOption.vsCommandExecOptionDoDefault)

   5:     {

   6:         if(commandName == "CRUDGenerator.Connect.CRUDGenerator")

   7:         {

   8:             CRUDEngine engine = new CRUDEngine(ProjectFinder.GetFirstDatabaseProject(_applicationObject));

   9:             engine.Run();

  10:             handled = true;

  11:             return;

  12:         }

  13:     }

  14: }

This engine creates an instance of my WinForms dialog and provides that dialog with a class I call the CRUDGenerator:

   1: public void Run()

   2: {

   3:     CRUDDialog dialog = new CRUDDialog();

   4:     dialog.Generator = new CRUDGenerator(_project);

   5:     dialog.ShowDialog();

   6: }

I won’t go over my CRUDDialog since most of it is simple WinForms programming. The CRUDGenerator is where we’ll focus next. This class is responsible for executing the T4 scripting engine and returning the results. For more information on executing T4 programmatically see this msdn page.

The most important thing to note when walking the Data Dude object model is that it is not Serializable (although it is thread-safe). This means you cannot access the object model from another process (i.e. through out of process DTE) or from another AppDomain. Fortunately during the process of executing the T4 template engine you have the opportunity to provide a class which must derive from ITextTemplatingEngineHost. My instance of that host is called VSDBCustomHost and it it I provide the current AppDomain

   1: public AppDomain ProvideTemplatingAppDomain(string content)

   2: {

   3:     return AppDomain.CurrentDomain;

   4: }

The core of this class is it’s RunTemplate method. In that method I create an instance of a VSDBCustomHost and process the template:

   1: public string RunTemplate(ISqlTable selectedTable, 

   2:                         string templateFullPath,

   3:                         out CompilerErrorCollection errors)

   4: {

   5:     string tableScript = GetTableText(selectedTable);

   6:     errors = null;

   7:     if (string.IsNullOrEmpty(templateFullPath))

   8:         return string.Empty;

   9:  

  10:     VSDBCustomHost host = new VSDBCustomHost();

  11:     Microsoft.VisualStudio.TextTemplating.Engine engine = new Microsoft.VisualStudio.TextTemplating.Engine();

  12:     host.TemplateFile = templateFullPath;

  13:     host.IncludePath = Path.Combine(VsUtils.GetVSTSDBDirectory(), @"Extensions\SqlServer\");

  14:     host.VsUtils = _vsUtils;

  15:     AppDomain domain = host.ProvideTemplatingAppDomain(string.Empty);

  16:     string output = string.Empty;

  17:     try

  18:     {

  19:         // Set the table to be generated into the app domain

  20:         domain.SetData("TT_TABLE", selectedTable);

  21:         //Read the text template.

  22:         string input = File.ReadAllText(templateFullPath);

  23:         //Transform the text template.

  24:         output = engine.ProcessTemplate(input, host);

  25:         errors = host.Errors;

  26:     }

  27:     finally

  28:     {

  29:         domain.SetData("TT_TABLE", null);

  30:     }

  31:     return output;

  32: }

At the beginning of this blog is the link to the zip file containing all the source for this add-in. From here I’ll leave you to paruse the source.

Conclusion

Hopefully you’ve now got an appreciation of the power of T4 templates! Remember that if you want to use these templates with Data Dude projects you’ll have to provide your own templating engine host since the Data Dude model cannot cross process or AppDomain boundaries since it is not serializable. Feel free to ping me if you’ve got questions!  Have a good 2009!

Patrick Sirr

Data Dude Programmer

Basic Navigation

I’m assuming you are writing an add-in. If so then your add-in will be initialized with a DTE2 object in its OnConnection method. This provides you with all the open projects in your solution. To find the database projects loop over all projects finding those with a EnvDTE.Project.Kind of “C8D11400-126E-41CD-887F-60BD40844F9E”.

Now that you’ve identified the EnvDTE.Project instance you can gain access to the DataDude model through the properties collection of this project. Get the “DataSchemaModel” property and cast to a Microsoft.Data.Schema.SchemaModel.DataSchemaModel. Note that you’ll need to add references to the DataDude assemblies Microsoft.Data.Schema.dll and Microsoft.Data.Schema.Sql.dll.

   1: public static DataSchemaModel GetModel(EnvDTE.Project project)

   2: {

   3:     object obj = project.Properties.Item("DataSchemaModel");

   4:     return ((EnvDTE.Property)obj).Value as DataSchemaModel;

   5: }

Note that the DataSchemaModel is thread-safe but is not XmlSerializable. So if you’re using DTE from an external process the DataSchemaModel property will return null.

Let’s continue and query the model for all the tables. Use the member DataSchemaModel.GetElements querying for ISqlTable. Note that we are choosing to use ElementQueryFilter.Internal as the argument because we only want those tables which have been defined by the user – not those brought in through an external reference to a .dbschema file. Here’s an example of how to retrieve all the tables in the project:

   1: public static IList<ISqlTable> GetTables(DataSchemaModel model)

   2: {

   3:     return model.GetElements< ISqlTable >(ElementQueryFilter.Internal); 

   4: }

Great! Now we have all the tables. Note that the objects returned by this query support other interfaces. For instance, ISqlTable supports ISqlColumnSource which makes navigating the table columns much easier.

Even though you have access to the model changes to it are prohibited and will throw an exception. This is because the project system’s model is controled through scripts. It is only through these scripts that the model can be modified. Ok, so the next logical question is – “where is this table defined? What script? Where in that script?” Good question! Cast your object (i.e. the ISqlTable instance) to IScriptSourcedElement. If this interface returns an ElementSource then it does indeed live in a source file in the project system. The CacheIdentifier field will be the full path filename in the project system and and the length/offset the location, within that file, of this particular object. Note also, that IScriptSourcedElement gives you access to the ScriptDom allowing you to walk the ASTs if your needs include walking our parse tree. If you download the add-in source below and look at VsUtils.GetSelectedModelElements you’ll see where I use the ElementSource.

Add-In Example

Now I’ll introduce you to a General Distribution Release (GDR) for Visual Studio Team System 2008 Database Edition Add-in which will alter the script in your project to comment out selected schema-owned elements. I’ve included the source code but, in order to compile the install project you’ll need to install Votive 3.0 (I used 3.0.4707.0). This add-in uses the DataSchemaModel features I talked about in the previous section.

To install run CommentOutAddIn.msi (available in this zip). This install will create these files on your filesystem:

%VSINSTALLDIR%\Common7\Ide\PublicAssemblies\VSTSDBCommentAddIn.dll

C:\ProgramData\Microsoft\MSEnvShared\Addins\CommentOutElementAddIn.AddIn

After installing run Visual Studio 2008 and bring up the Add-in Manager by selecting Tools.Add-in Manager from the main toolbar. Verify that “Database Project Comment Out Add-in” is available and selected to load.

A new menu will appear in the Data menu called “Comment Out…” whenever at least one Database Project is open in the solution explorer.

Selecting this menu button will bring up the Comment Out dialog. Select the Database Project to search and click the “Find Schema Objects” button. This will search the project for all Tables, Views and Procedures and populate the tree control you see in the image below.

At this point select the schema objects you’d like to comment out and select “Command Out Objects”. At this point the dialog will close and all the selected elements will be commented out from their respective files.

For each file you should see a message similar to this in the output window:

C:\TEMP\DATABASE3\DATABASE3\SCHEMA OBJECTS\SCHEMAS\SALES\TABLES\SPECIALOFFERPRODUCT.TABLE.SQL is being modified so selected schema objects may be commented out.

Here’s the result of the commenting out a table:

   1: /*CREATE TABLE [Sales].[SpecialOfferProduct] (

   2:     [SpecialOfferID] INT              NOT NULL,

   3:     [ProductID]      INT              NOT NULL,

   4:     [rowguid]        UNIQUEIDENTIFIER ROWGUIDCOL NOT NULL,

   5:     [ModifiedDate]   DATETIME         NOT NULL

   6: );*/

Conclusion

The Comment Out Add-in is capable of commenting out Tables, Views and Procs in your script. But more than that, I hope the source code provides some help to for those wishing to write add-ins to tailor to their specific needs. Enjoy!

- Patrick Sirr

“DataDude” Senior Programmer

To install run InstallImportScripts.msi (available in this zip). This install will create these files on your filesystem:

%VSINSTALLDIR%\Common7\Ide\PublicAssemblies\ImportScriptsRecursively.dll

C:\ProgramData\Microsoft\MSEnvShared\Addins\ImportScriptsRecursively.AddIn

After installing run Visual Studio 2008 and bring up the Add-in Manager by selecting Tools.Add-in Manager from the main toolbar. Verify that “Database Project Script Import” is available and selected to load.

A new menu will appear in the Data menu called “Import Scripts…” whenever at least one Database Project is open in the solution explorer.

Selecting this menu button will bring up the Scripts Selector dialog. Use this dialog to

· Select the project which will receive the imported database schema objects (the target project)

· Select the directory to search for .sql files

· Select or deselect individual files or folders from the import process. Note that, by default any scripts named Script.PostDeployment.sql, Script.PreDeployment.sql or IgnoredOnImport.sql will not be selected.

· Select the default encoding for the .sql files. If you run the addin and no database schema objects are imported verify the encoding of your files match the encoding you’ve selected in the Scripts Selector dialog.

· Choose among the various import script options available on the Import Script Wizard.

· If you choose to concatenate your scripts into one large file memory pressure will increase but performance will be vastly increased.

Once you select ‘Run Import’ the Output Window will be activated and messages will appear providing you with feedback on the import script process. Each file will produce output similar to the one below. The output will include 1) a progress indicator (in the case below 4/700) and 2) the DTE command which can be executed into the Command Window to re-import that particular script (in the case below Project.ImportScript /FileName "C:\temp\AdventureWorks\AdventureWorks\Schema Objects\Database Level Objects\Security\Schemas\Person.schema.sql" /Encoding UTF8).

Executing 4/700 : Project.ImportScript /FileName "C:\temp\AdventureWorks\AdventureWorks\Schema Objects\Database Level Objects\Security\Schemas\Person.schema.sql" /Encoding UTF8 Started importing file: C:\temp\AdventureWorks\AdventureWorks\Schema Objects\Database Level Objects\Security\Schemas\Person.schema.sql File name C:\temp\AdventureWorks\AdventureWorks\Schema Objects\Database Level Objects\Security\Schemas\Person.schema.sql (size: 270) Parsing SQL script Total number of batches in script: 2 Total number of statements in script: 2 Finished importing file: C:\temp\AdventureWorks\AdventureWorks\Schema Objects\Database Level Objects\Security\Schemas\Person.schema.sql

Conclusion

The Import Scripts Add-in allows you to import multiple scripts at once. This should be a useful addin when upgrading from CTPs (which do not support upgrade) or importing from a script archive into a database project for the first time.

- Patrick Sirr

- DataDude Programmer