Integrating Layer
| Retired Content |
|---|
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. |
.png "patterns & practices Developer Center")
Integration Patterns
May 2004
Summary: This chapter describes the different strategies for designing an integration layer and the tradeoffs involved in choosing an alternative. An integration layer can automate complex business processes or provide unified access to information that is scattered across many systems.
Contents
Level of Automation
Level of Abstraction
Maintaining State
Coupling
Semantic Dissonance
Choosing an Integration Layer Type
Integration Layers Patterns
"Any problem in computer science can be solved with another layer of indirection."—David Wheeler (chief programmer for the EDSAC project in the early 1950s)
As described in Chapter 1, connecting multiple systems requires an infrastructure that moves data between the systems. However, you often want the solution to do more than just pass data around. You want to add an additional layer of functionality on top of the functional assets that reside inside the existing applications. This layer can automate complex business processes or provide unified access to information that is currently scattered across many systems.
How should such an integrating layer be designed, and what choices do you have? Unfortunately, there is no single right answer for all enterprise architectures. This chapter discusses some of the key considerations to help you to understand the tradeoffs associated with the various alternatives and to find the most suitable approach.
Level of Automation
A fully integrated enterprise seems to be any CIO's idea of perfection. Complex interactions between systems are orchestrated through precisely modeled business process definitions. Any data format inconsistencies are resolved through the integration layer. Relevant summary data is presented to executive dashboards with up-to-the-minute accuracy. Such visions are surely enticing, but should every enterprise set out to build such a comprehensive and inherently complicated solution?
It is not likely that anyone would build such a solution because of the time and money it takes to build a complex enterprise integration solution. Therefore, deciding how far to go is an important step when planning an integration solution. Automation brings efficiency and can help eliminate inconsistencies in current business practices. However, complete automation can require an enormous upfront effort that delays the tangible benefits to the business. A simpler solution might only achieve a portion of the business benefit, but it achieves that benefit much sooner and with much less risk. For example, displaying information from multiple systems in a single screen that is divided into individual panels can increase productivity, and it does not require the system to resolve all the differences between existing systems.
Level of Abstraction
Object-oriented design principles teach the benefits of abstraction. Abstracting one system's internals from other systems allows you to change one system without affecting the other systems. This ability to limit the propagation of changes is a key consideration for integration solutions where connections can be plentiful and making changes to applications can be very difficult.
You achieve abstraction in a number of ways. For example, you can achieve abstraction by passing self-contained documents between systems. Passing a document does not instruct another system to perform a specific function, but it leaves that choice to the receiving system. Thus, if the receiving application processes the document in a different way, the originating system is not affected.
Maintaining State
Most integration solutions aim to streamline business processes such as placing an order or making a payment. These processes rely on the coordinated completion of a series of steps. When an order arrives, the order has to be validated, the inventory has to be checked, the sales tax has to be computed, and an invoice has to be generated. This type of integration requires some mechanism to track the current state of an order. For example, the mechanism might indicate whether the order has been validated yet and which step should be completed next. This state mechanism can be implemented in three ways:
- Manually. State can reside in the user's head or in a manual on the user's desk. Based on the information gathered, a user can decide which step should be performed next. This approach is less efficient and less reliable than more automated solutions, but it is also very flexible and easy to implement.
- Inside existing applications. The state can be kept inside existing applications. For example, after the inventory application has verified the number of items on hand, it can be programmed to send a message to the financial system to compute the sales tax. The financial system in turn might send a message to the fulfillment system to ship the goods. This approach is more efficient but also ties the applications more closely to each other.
- In an integration layer. You can insert a new integration layer to orchestrate the activities across multiple applications and to keep track of state. Such a layer is likely to require additional effort, but it manages all interactions from a central point without applications requiring information about each other.
Coupling
Loose coupling has become the standard of distributed applications. Loose coupling makes integrated systems less dependent on each other and allows them to evolve independently. However, loose coupling is not a panacea. Loosely coupled systems can often be difficult to understand and debug.
Semantic Dissonance
One of the key difficulties in building integration solutions is the notion of semantic dissonance. This term describes a situation where data that appears to be the same may not necessarily mean the same thing. For example, one system might treat a monetary figure for annual revenues as if it includes sales tax, while another system treats the monetary figure as if it does not include any taxes. Likewise, one system might define the Western Region, in the context of revenue breakdown, to include the state of Colorado. Meanwhile, another system may define Colorado as part of the Central Region. These types of semantic dissonance can be very difficult to detect and reconcile. Some rather elegant types of integration rely on the complete resolution of semantic dissonance, which might not be feasible in real-life situations. Other simpler forms of integration, such as Portal Integration, can accommodate ambiguity, but they do it at the expense of precision. For an in-depth discussion of data modeling and the perils of semantic dissonance, see Data and Reality, by William Kent [Kent00].
Choosing an Integration Layer Type
Based on these considerations, this guide identifies three approaches towards integrating layers. Each approach is presented as a pattern (see Figure 1):
- Entity Aggregation
- Process Integration
- Portal Integration
.gif "Ff647962.f03intpatt01(en-us,PandP.10).gif")
Figure 1. Three integration approaches
The following paragraphs briefly describe each approach to integration, in order from least to most complex.
Portal Integration
Portal Integration connects systems to provide a unified view to a user. Portal Integration can dramatically increase productivity because users no longer have to access multiple applications independently. Instead, they receive a comprehensive view across all systems in a visually consistent format that is illustrated in Figure 2. More sophisticated versions of Portal Integration also allow the user to make updates to individual systems or even across multiple systems.
.gif "Ff647962.f03intpatt02(en-us,PandP.10).gif")
Figure 2. Portal Integration provides a unified view across systems
Portal Integration is often easier to implement than other, more automated alternatives. The disadvantage of the simplicity is that much of the business process and many of the rules still reside in a user's head as opposed to in a system. Nevertheless, Portal Integration is often a good first step towards integration.
Entity Aggregation
The main limitation of Portal Integration is that it aggregates information only for end users but not for applications. Entity Aggregation addresses this shortcoming by providing a unified data view to applications, as illustrated in Figure 3.
.gif "Ff647962.f03intpatt03(en-us,PandP.10).gif")
Figure 3. Entity Aggregation provides unified data access across systems
Entity Aggregation provides a logical representation of unified data entities across multiple data stores. Applications can interact with this representation as if it were a single data source. Thus, Entity Aggregation greatly simplifies the development of applications that need to access data across multiple data stores. The tradeoff for this increased integration is that the integration layer must now resolve any instances of semantic dissonance between the individual systems. Resolving such semantic dissonance and creating a unified data layer can be a major undertaking.
Process Integration
Process Integration focuses on the orchestration of interactions between multiple systems, as illustrated in Figure 4. As mentioned earlier, automated business processes such as straight-through processing often drive integration initiatives. It is often advisable to model these processes outside of the applications to avoid coupling the applications to each other. You can achieve this modeling by adding a Process Integration layer that manages the interaction across applications. This layer tracks the state of each business process instance and also enables centralized reporting. For example, this layer enables the Business Activity Monitoring (BAM) feature of Microsoft BizTalk® Server.
.gif "Ff647962.f03intpatt04(en-us,PandP.10).gif")
Figure 4. Process Integration orchestrates activities across systems
The three integration layer types are not mutually exclusive. An integration solution might use Portal Integration for some portions of the solution and Process Integration for others. Management reporting functions may be accomplished by using Entity Aggregation. All the approaches may be used in parallel.
Integration Layers Patterns
The following table summarizes the three patterns just discussed and shows the corresponding implementation patterns.
Table 1: Integration Layers Patterns
| Pattern | Problem | Associated implementations |
|---|---|---|
| Entity Aggregation | How can enterprise data that is redundantly distributed across multiple repositories be effectively maintained by applications? | |
| Process Integration | How do you coordinate the execution of a long-running business function that spans multiple disparate applications? | Implementing Process Integration with BizTalk Server 2004 |
| Portal Integration | How can users efficiently perform tasks that require access to information that resides in disparate systems? |
| Retired Content |
|---|
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. |