Bill Gibson's Blog

SQL Server Data Tools - From Oslo to Juneau.

Bill Gibson MSFT — Mon, 12 Dec 2011 11:15:00 GMT

Time to blog again! The road to Oslo turned out to have many twists, turns and eventually roadblocks. So for a little over a year now I’ve been working on database development tools with SQL Server Data Tools – known earlier as ‘Juneau’. Seems to be a far north theme here! We’re readying the first release of SSDT as part of SQL Server 2012 so life is exciting (hah!) We’ve just shipped a tech preview, CTP4, which you can download here. The tools target SQL 2005 and later so you don’t have to install SQL 2012 to use them. And they work with any edition of SQL Server – from Express to Enterprise. If you don’t have, or want to install, SQL Server you can use Local DB, the lightweight SQL Server runtime that installs with the tools. This is perfect for desktop debugging or just practising your SQL. And of course the tools also target SQL Azure in the cloud. Think of SSDT as the next generation of the database projects that currently ship in the high-end Visual Studio SKUs. Moving the tools from Visual Studio to SQL Server means any SQL Server user can use them – no charge. They still install and run in VS – either the installation you have already or a lightweight shell SSDT will install for you. And being in SQL Server means we can really push the envelope and work closely with the engine teams to make the tools richer and more integrated with the database. SSDT uses a model-based declarative approach which is what makes it so interesting to me – I'll post more on this later. As for me, I’ve been working on the model-based schema comparison tool - more on that in another post also.

For more info on SSDT check out the SSDT main page and SSDT team blog. But if you just can't wait go straight to the download page.

Oslo CTP update; Data Modeling Design Patterns in M

Garry Trinder — Wed, 28 Jan 2009 17:59:00 GMT

As modelers, one of the things we're doing all the time is looking for patterns - trying to distinguish what in each model is truly unique to the domain from that which is more broadly applicable, and then either using or adapting existing patterns or harvesting new patterns to put in our back pocket. The January Oslo CTP update (woohoo!!) on the Oslo Dev Center includes an initial set of data modeling design patterns. These patterns, described and illustrated in M, include patterns for modeling enumerations and relationships, including closed type-based enumerations, open extent-based enumerations, and one-to-many, many-to-many and one-to-one relationships.

While this may seem like pretty basic stuff you'll likely find some or all of it useful, particularly if you're new to M and coming at it and repository modeling with an OO rather than relational mindset. The patterns all have code samples so you may find it useful to have the patterns open while you're coding.

We'll add more patterns as our experience grows and the language evolves and is more fully implemented. In the pipeline are patterns for vertical partitioning, including patterns that help when mapping a generalization hierarchy in a conceptual model to a relational design. You can see some early exploration of these kinds of patterns in the domain schemas included in the CTP.

As always, we'd love to get your feedback. If you have any comments on the patterns and the usefulness of this kind of material let us know. And if you have any broadly useful patterns in your back pocket that might map nicely into M they would be great to hear about also.

Domain Modeling

Garry Trinder — Tue, 27 Jan 2009 23:27:00 GMT

My focus within the Olso team is on domain modeling – creating models for specific problem domains using the Oslo modeling platform’s languages and tools. Let me describe why we think of this as more than just data modeling.

At the center of an Oslo domain is a data model authored in M. The primary purpose of this model or schema is to describe repository storage, defined in terms of extents. This domain schema defines data integrity constraints and may also define reusable data definitions or types, as well as computed values that describe interesting ways to retrieve and compute data from the model. The schema is compiled into SQL, resulting in tables, views, constraints and functions which are loaded into a SQL Server database primed with some simple Repository patterns. The result is a simple and easy-to-query and otherwise normal SQL database, which can be queried using any SQL data access API or tools.

Surrounding this data model and considered part of the domain may be any number of domain specific languages, which if developed using Oslo may be textual, authored in MGrammar, or visual, authored with Quadrant. These DSLs define ways in which domain information (instance data) can be created or presented. Any number of textual and visual languages may be defined for the same domain, each customized to provide some specific perspective. A domain can also reference other domains, so models and languages may also reference (and in due course, embed) other models and languages.

Accompanying each DSL may be utilities that map or import/export language 'documents' to and from repository storage. For textual DSLs described in MGrammar simple transformations can be expressed in the grammar itself - more complex transformations need to be described separately. A key Oslo scenario will involve cataloging and potentially creating/editing pre-defined artifacts that might otherwise be created and/or consumed externally by other tools. Collectively, the domain schema, textual and visual DSLs, and their accompanying import/export tools define what we think of as a domain from an Oslo standpoint and will typically be evolved as a unit. This whole-domain perspective is an important part of the way we think about modeling with Oslo and strongly influences how you can expect the Oslo platform to be developed. You can see some early examples of domain schemas included with the CTP on the Oslo Dev Center. Take a look if you can. Next up, data modeling design patterns in M...

Getting Back in the Saddle (although no longer on a Whitehorse)

Garry Trinder — Tue, 27 Jan 2009 21:12:00 GMT

Long time no blog! In the meantime I've changed jobs within Microsoft; I've been working for the last several years on the modeling platform known as “Oslo” which 'came out' at PDC in October. You can find out more about Oslo at the Oslo Dev Center.

I was amazed to find my blog still accessible after being dormant for so long. While it was with some nostalgia that I freshened up the links to the Whitehorse tech notes below, you should follow this link to get the latest on Visual Studio Team System Architect Edition.

Whitehorse Architecture TechNotes Published on MSDN

Garry Trinder — Tue, 06 Dec 2005 09:40:00 GMT

Phew! A set of Visual Studio Team System Tech Notes have now been published on MSDN, including some fifteen new articles I've written on topics related to the Whitehorse architecture tools. These should all be easy to read and are inter-linked, to make them an easy to browse. Hopefully they will provide some useful background and some new insights into the Distributed Systen Designers.

Below are links to each of my notes, ordered in what is probably a good sequence in which to read them.

TN_1104: Understanding the System Definition Model

TN_1114: The Four Layers of Systems in the System Definition Model

TN_1100: Understanding Applications and the Application Diagram

TN_1105: Why Class Libraries are not shown on an Application Diagram

TN_1109: Understanding Systems and the System Designer

TN_1101: Understanding and Using the Default System

TN_1110: Using Systems to Represent Services in a Service Oriented Architecture

TN_1112: System Portfolio Management

TN_1102: Designing Substitutable Web Services

TN_1113: Representing Connections to Manual, Physical and Other Kinds of System

TN_1111: Top-down System Design

TN_1108: Connecting Applications to Web Services via Class Libraries

TN_1106: Copying Endpoints vs Creating Endpoints from WSDL

TN_1103: Web Service Endpoint Name Propagation

TN_1107: Understanding, Using and Creating Toolbox Prototypes

Why do ASP.NET WebService and ASP.NET WebApplication look alike on the Application Designer?

On shadow applications and agile development

Garry Trinder — Thu, 01 Dec 2005 16:49:00 GMT

Randy Miller has published a paper on agile development that discusses the use of 'shadow' applications in an agile modeling and development context, in which he talks in somewhat veiled terms about the use of the Whitehorse design tools. The term shadow application emerged from discussions Randy and I had at Tech Ed earlier this year as we developed the top-down design approach described in my earlier article. In the article I show how you can use a shadow application as a placeholder for the behavior of a system and thus design systems top-down. In his paper, Randy take this idea further and suggests the shadow application is actually written or generated, consistent with the code-oriented agile development philosophy.

Logical vs. Physical Architectural Modeling Concerns

Garry Trinder — Fri, 11 Nov 2005 12:33:00 GMT

We've been having hallway discussions about some of the dimensions of modeling application architectures. SDM adopts a particularly physical perspective, as it focuses on the deployment packaging stack of concerns. Its focus is on resources (things like assemblies, config, XML files etc.) that are composed into independently deployable applications, that are composed into systems, which are then composed into still higher level systems. SDM, does allow you focus on tangible software entities and express their behavior - very CBD-like. But of course there are other perspectives, and particularly troublesome to some is the absence in the current designers of a logical perspective, in which you can define what an application does, independently of describing how it does it or how that implementation is packaged. The discussion turned to the role of namespace in that logical perspective, specifically to the question, are namespaces the primary logical organizing perspective. I wrote some comments in an email on the subject and have elaborated on them here and invite comment. While namespaces are clearly not a packaging concept as they can cut across assemblies, they are not necessarily the best or only way to organize the elements in a logical architecture. Instead the notion of a cross-cutting and entirely logical organization of function is proposed. The term "module" was suggested some months ago by a colleagure for this notion. Thus a module is an arbitrary collection of function. A module could be a collection of methods, one class, many classes; may be coincident with an assembly, cross assemblies or contain assemblies. Modules can also describe data. Early on in your design, you might think about modules but go no further than giving a name and describing the role. A classic block structure architecture diagram might be used to show the way modules interact, either depicting a specific instance of an architecture or a pattern before any details exist of the behavior of each module. You might assign namespaces to modules – although the naming of types/classes, while important, is not intriniscally linked to the logical structuring of software. Namespaces might reflect the logical structure and you might be tempted to conflate the two, such that you use namespaces when really you arethinking about modules. At some point in your design process as you refine your module designs and start to define classes, creating namespaces will become important. And at some point in your design packaging considerations will become important, when you will specify applications, frameworks, and systems. With your packaging decisions you harden the partitioning of the functions in your modules and make communication and binding decisions.

Thus there are module, namespace and packaging dimensions, These represent orthogonal but related concerns which inform each other. Different design strategies will have you tackle these aspects: physical structure (systems, applications, frameworks), logical structure (modules) and implementation (classes with namespaces) in different amounts at different times – being able to reason about all three as and when you choose will be the most flexible and should be what we aim for in tools. Associating a namespace with a module or even a package may make good sense when the purpose of the module/package and the goals of the namespace align but the two are not intrinsically linked.

ARCast on model-driven development

Garry Trinder — Thu, 03 Nov 2005 22:08:00 GMT

An interesting initial round on this model-driven development podcast, including comments from Jack Greenfield from our team. Will be fun to watch this one develop... As someone who was involved in what we believed to be one the big successes of the 80's and 90's so called CASE era, I always find it a little irritating to hear about all the failures, so it was interesting to hear some positive comments about the lessons to learn from that period. From my experience, what we were doing at Texas Instruments was successful for a long time, in large part because the problem domain we tackled was limited and the patterns well recognized-- we were targeting large scale integrated database transaction processing systems. In effect, while we never described it in these terms, what we developed was a domain specific language - or at least a family of languages. And as one of the contributors mentioned, the approach relied on the assumption that you never modified a line of generated code - there were lots of user exits but you never touched the code. We even implemented a feature that deleted the source code after it had been compiled to save space :-) The approach we used then would be much harder to make successful now, when frameworks are so much richer and the problem domains and application model so much more varied.

Kick the VSTS tires...

Garry Trinder — Thu, 03 Nov 2005 01:59:00 GMT

Now you can try out Team System in a hosted environment without the hassle of downloading and installing. Check out Rob Caron's post for details.

Going dark...

Garry Trinder — Sun, 30 Oct 2005 09:51:00 GMT

I've not lost enthusiasm for this blogging thing, its just that I've been asked to convert some of my blog entries into MSDN TechNotes and write others afresh to accompany the Visual Studio launch, so I've been busy with notes/articles in work on the following topics among others: Understanding the SDM Type System
The Four Layer Model of Systems in SDM
Understanding Applications in Application Designer
When and how to use the Default System
The Role of System Designer
Using Systems to Define Services
Designing Substitutable Services
Understanding Creating and Using Toolbox Prototypes
Top-Down Design (published here already)
Why no Class Libraries on the Application Designer (blogged here already)

I will post the finished articles here and/or links to them on MSDN.