Organizing Patterns

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. Please see the patterns & practices guidance for the most current information.

Version 1.0.0

Complete List of patterns & practices

"Each pattern then depends both on the smaller patterns it contains, and on the larger patterns within which it is contained." - Christopher Alexander in The Timeless Way of Building

An innovation in one area of technology often fuels a breakthrough in another area. Radar technology turned into a cooking device: the microwave oven. The Internet itself was originally designed as a military communications network with resilience against single points of attack and has since turned into the world's largest repository of knowledge. Similarly, patterns, originally applied to building and town architecture, were quickly embraced by the software development community as a means to describe complex software systems.

Today there are dozens of patterns related to software with more emerging daily. This abundance of patterns creates a new set of challenges. How can a developer identify those patterns that are most relevant to the task at hand? Is the collection of patterns sufficient to describe complete solutions?

This chapter answers some of these questions by demonstrating how to:

Identify relationships between patterns.

Group patterns into clusters.

Identify patterns at various levels of abstraction.

Apply patterns to multiple aspects of a solution.

Organize patterns into a frame.

Use patterns to describe solutions concisely.

Pattern of Patterns

One reason the object-oriented programming community embraced patterns so emphatically is because patterns describe relationships. The base element of object-oriented programming is a class. However, a single class is not very meaningful apart from its relationship to other classes that make up the solution. In the same way, the data world is full of elementary items and their relationships. These could be entities and attributes, database tables related by foreign keys, original data and copies of it, and so on.

A pattern about an elementary item is valuable in its own right. But patterns that pull other patterns together are even more powerful. These pattern clusters turn the sea of individual patterns into a much more manageable collection of patterns by highlighting the relationships between the clustered patterns. For example, to implement Master-Master Replication,you first need to build an exact copy of one of the masters, and you could use Master-Subordinate Snapshot Replication to do that. Then you would have to manage the integrity of the two masters, and you could use Master-Master Row-Level Synchronization for this purpose. So these three patterns form at least part of a useful cluster.

Patterns participate in other relationships, too. For example, some patterns are refinements of others. Master-Master Row-Level Synchronization is a specific application of the concept of Master-Master Replication.

To begin organizing patterns according to relationship, visualize a set of patterns as small circles (see Figure 1):

Figure 1: A set of patterns

If you draw a line between each pair of patterns that share some relationship, you get a picture like this:

Figure 2: Pattern relationships represented as lines

The somewhat random collection of circles becomes a connected web of patterns. When you look at a pattern, you can now identify closely related patterns and review those as well. You can also identify "neighborhoods" of closely related patterns and see how they are related to other, more remote patterns.

Pattern Clusters

Charting the relationships between patterns helps you navigate from one pattern to a set of related patterns. However, it does not yet tell you where to start. If you are building a data replication solution, should you look at Master-Subordinate Snapshot Replication first or Master-Subordinate Transactional Incremental Replication? A cluster gives you hints on what to read first by clearly identifying a root for the set. Pattern clusters are groupings of patterns that relate to a specific subject area. So in this example, you might need to read either Master-Subordinate Replication or Data Replication or Moving Copy of Data depending on how much you've already thought through the issues at the architecture or design levels.

Figure 3: Pattern clusters

This first release of Microsoft Data Patterns identifies a Data Movement patterns cluster, with the Move Copy of Data pattern at its root. Industry luminaries and customers identified these patterns as having critical importance right now.

Different Levels of Abstraction

Dividing patterns into clusters makes them more manageable. However, recognizing levels of abstraction by dividing the patterns in a hierarchy from general to more specific detail is also very useful. It not only categorizes patterns so that people with different roles can find the patterns that correspond most closely to their area of interest, but also helps you decide which patterns to consider first. A cluster is then a hierarchy of patterns with the root pattern at the top.

A good way to categorize the patterns is to divide the pattern graph into the three levels shown in Figure 4.

Figure 4: Levels of abstraction

Architecture Patterns

The data architecture patterns express the highest-level design patterns for data topics. They are completely product-independent and are not meant to be directly implemented.

Design Patterns

The data design patterns provide a greater level of design detail, while still remaining product-independent. However, they are usually detailed enough to serve as the basis for a product-specific implementation pattern.

Implementation Patterns

The patterns community refers to more detailed, programming-language-specific patterns as idioms. This definition works well for software patterns. However, the scope of this guide is not just software, but software-intensive systems, including the deployment of the software onto hardware processing nodes to provide a holistic business solution. Therefore, Microsoft modifies the definition of an idiom given in Pattern-Oriented Software Architecture (POSA) [Buschmann96] to reflect the broader scope and relabels these patterns as implementation patterns:

An implementation pattern is a low-level pattern specific to a particular platform. In the case of the data patterns, we describe how to implement data design patterns with SQL Server.

Viewpoints

Although the levels of abstractions help to address different user groups, they do not reflect the fact that a software solution encompasses other viewpoints and specializations. Each viewpoint itself can also focus on different levels of abstraction. Therefore, Microsoft patterns depict the following viewpoints as vertical slices across the pattern graph: database, application, and infrastructure. There is often a significant gap between the application and infrastructure viewpoints. Concepts, abstractions, and skill sets are sufficiently different to warrant the insertion of a buffer between the two that helps to bridge the divide. This viewpoint is called the deployment viewpoint.

This line of reasoning results in the four viewpoints shown in Table 1.

Table 1: Patterns Viewpoints

Database	The database view describes the persistent layer of the application. This view looks at such things as logical and physical schemas, database tables, relationships, and transactions.
Application	The application view focuses on the executable aspect of the solution. It includes such things as domain models, class diagrams, assemblies, and processes.
Deployment	The deployment view explicitly maps application concerns to infrastructure concerns (for example, processes to processors).
Infrastructure	The infrastructure view incorporates all of the hardware and networking equipment that is required to run the solution.

Figure 5 overlays these viewpoints as vertical lines over the pattern graph and the levels of abstraction.

Figure 5: Adding viewpoints

Keep in mind that these four areas describe different viewpoints of the overall solution. Therefore, unlike the levels of refinement, these viewpoints do not describe a hierarchy, but provide four different ways of looking at the solution. You can compare these viewpoints to different types of maps. One map of a region may depict traffic networks such as roads and freeways, while another map of the same area shows the topography. Still another map may show state and county borders. Each map has its own vocabulary. For example, lines in the topographical map represent elevations, while lines in the traffic map represent streets. Nevertheless, all maps describe the same subject: a specific geographic region.

For the sake of simplicity, Figure 5 does not show the cluster boundaries. However, the clusters, the layers of abstraction, and the viewpoints exist in parallel. They represent different ways to access the same set of patterns.

The Pattern Frame

The combination of three levels of refinement on the vertical axis and the four viewpoints on the horizontal axis results in a grid-like organization of the pattern graph. Figure 6 shows this arrangement, which is called the Pattern Frame.

Figure 6: The Pattern Frame

The Pattern Frame is included with each individual pattern description as a point of reference and as a navigational aid.

Constraints

Because there are so many potential patterns in the data realm, it is necessary introduce a means to constrain the set that is being developed at any point in time. Figure 7 shows a simple view of electronically held data in an Enterprise context.

Figure 7: Principle enterprise data types

Terminology may vary, but generally within an Enterprise there are four main types of data stores:

Operational data, which supports the daily customer-facing business processes.

Informational data, which supports business reporting, analysis, and planning.

Knowledge, which is the explicit storing of best practices and other forms of intellectual capital.

Metadata, which is data about data, or the knowledge of what data is stored in the other three stores, how it is structured, and how it is used.

Different types of users interact with all these types of data stores, either to use them for business purposes or to administer and maintain them. They do this with tools, application packages, or customer applications.

These stores also interact as indicated by the arrows in Figure 7. Typically, operational data is copied to other operational stores, to informational stores, or to knowledge stores. Informational data is also copied to other informational stores and to knowledge stores.

In the context of this overall data ecosystem, this initial data patterns release focuses on how to move copies of data within the operational stores in Figure 7, and in certain contexts, also between the operational and informational stores.

The topic of moving copies was chosen because of the complexity of the problem today. This complexity has compounded as enterprise data processing has evolved from a single copy of data in a centralized database with offline nighttime batch work to distributed systems requiring constantly available data copies, and to data warehouses, Extract, Transform, and Load (ETL) services, and Relational Database Management Systems (RDBMS). The first set of patterns presented in Chapter 3, "Data Movement Patterns," is designed to help you sort out the complex choices involved in moving data in the context of replication. These patterns were selected based on workshops and reviews with customers, partners, and industry luminaries, who identified this set as being of critical importance now.

Within this constrained view of data replication, the Pattern Frame organizes the collection of patterns into meaningful subcategories, such as how the data is organized, how it is used by applications, how it is deployed, and what sort of infrastructure it runs on. These subcategories can each be viewed from the perspectives of architecture, design, or implementation.

Pattlets

A pattlet describes a solution to a problem, but does not contain a detailed description of the context, problem, or forces that may impact the solution. Pattlets are required because elaborating on all patterns in the graph takes a significant amount of effort and time. Patterns need to evolve as the collective understanding of them evolves.

Deferring patterns until later, however, leaves holes in the pattern graph, which could result in related patterns suddenly becoming disconnected. To preserve the integrity of the relationships inside the pattern graph, this guide includes the patterns that were not included in the first release as pattlets. Pattlets are actual patterns that have not yet been documented in detail.

Summary

This chapter demonstrated how patterns provide a vocabulary to efficiently describe complex solutions without sacrificing detail. Effectively, the patterns form a new language with which architects and designers can communicate their thinking.

Because of the large number of patterns involved in building enterprise solutions, it can seem difficult to learn this new language. The guide structures the patterns into smaller, more closely related sets of patterns. This allows you to get started by using a smaller set of patterns, depending on your specific interest or the stage of the project.

In this chapter, four mechanisms have been introduced to help you navigate the patterns:

Relationships. Relationships between patterns help you to identify patterns that are closely associated to the pattern you are using.

Clusters. Clusters group patterns that belong to a common subject area

Levels of abstraction. Levels of abstraction allow you to describe concepts in a manner that is consistent with the level of detail of your discussion.

Viewpoints. Viewpoints help you select the vocabulary that is relevant to a team's particular role.

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.