Interop

Connect To Mainframe Apps With BizTalk Adapters and .NET

Bashar "Bash" Badawi

Contents

BizTalk Adapters for Mainframes
Making the Connection
Exposing COBOL Apps as Endpoints
Integration Beyond Adapters
Wrapping Up

Imagine you were a bank CEO back in the 1970s. Your customer base was growing rapidly, and keeping paper records was proving to be an expensive and a risky business. Where could you go to automate record-keeping? What were your options for more efficient storage?

As it turned out, most such organizations invested in mainframes or minicomputers, such as the AS/400 and VAX systems—and it was on these systems that many programmers learned their trade. Common Business Oriented Language (COBOL), which was originally developed in 1959, became the dominant development language on these systems.

You might be surprised to know that COBOL is still a common language for business apps. According to a 2001 Gartner report on enterprise programming language adoption, more than 60 percent of the businesses surveyed still used COBOL. Almost every major industry relies on COBOL. Most companies that use mainframe systems for business-critical applications and data storage have continued investing in COBOL development—and probably will do so even after migrating away from mainframes. As recently as 2006, the New York Stock Exchange had moved its stock options clearing and billing systems to Windows®-based servers, but continued running its COBOL applications in recompiled form.

In the early days, mainframe connectivity was limited to terminals connected via coaxial cable to a controller that was, in turn, connected to a front-end processor (FEP), much like current networks. The network protocol was typically SNA (System Network Architecture), though mainframes later came to support TCP/IP as well. Another popular choice for wire-level connectivity was IP-DLC and High Performance Routing. The original design of SNA did not support routing, so special routing hardware or software was used to deliver the packets to the FEP. Since many COBOL applications were SNA protocol-specific, IP-DLC was developed as an alternative in which IP is the main carrier and SNA is somewhat encapsulated within it.

Within SNA there are sub-protocols based on the concept of a logical unit (LU)—an end user or application that interacts with the network. LU 0 was used for custom protocols, LU 2 for a 3270 data stream targeting terminal-based applications, LU 3 for printing from the mainframe, and LU 6.2 for application-to-application communication. Integrating with LU 2/3270 (the green screen often associated with mainframe interfaces) is commonly known as screen-scraping, where an application traverses the screen looking for meaningful data in fixed positions.

When dealing with these LU protocols, your choice is often between utilizing an existing toolset and developing your own custom solution. This becomes a rather obvious decision given the complexities of the latter option. In either case, you would have to develop or use a low-level network handler and then move up the OSI stack to supply the additional logic needed to handle the specific requirements of the target system, including byte conversion using different code pages, packet assembly and disassembly, session connection, conversation management, and security. This is no small task.

BizTalk Adapters for Mainframes

BizTalk ® Adapters for Host Integration Server help you connect to mainframe and midrange systems and integrate your Windows systems with their host applications, data sources, messaging, and security. By connecting these technologies, the resources residing on a mainframe system become another BizTalk-managed endpoint that can be used as a send or receive location, for example.

There are four main adapters in Host Integration Server 2006:

• BizTalk Adapter for Host Applications
• BizTalk Adapter for WebSphere MQ
• BizTalk Adapter for Host Files
• BizTalk Adapter for DB2

In addition to the BizTalk adapters that Host Integration Server offers, network connectivity components for TCP/IP, SNA, and TN3270 are also provided to supply a transport layer for the data to reach the host.

Once installed, the adapters are registered so that BizTalk can locate them, and you can add them in the BizTalk administration console as shown in Figure 1.

Figure 1** BizTalk Adapters in the Administration Console **(Click the image for a larger view)

In addition to the adapters themselves, all the adapter features are also installed in order to provide direct integration with the mainframe from within Visual Studio®. For data access, the DB2 and host file Microsoft® .NET Framework 2.0 managed providers are installed and available to be used either programmatically or through integration with Server Explorer in Visual Studio. A Host File Designer and DB2 Data Access Wizard are also provided so that users have the ability to directly access and design host files.

Network connectivity tools, a transaction integrator, and Host Integration Server projects are also provided as part of the design-time experience, and there's sample code to address the vast majority of supported scenarios for connectivity to mainframes and AS/400 systems. There's even a Host Simulator for those who don't happen to have a mainframe at home!

The SimHost can be found under the System folder in the Installation Folder.

Making the Connection

In order to connect and choose the right protocol, you need to get the appropriate parameters and security credentials from the mainframe administrator. When connecting to a DB2 database, for example, you are presented with options for TCP/IP and SNA (LU 6.2). These settings apply across all layers of integration with the mainframe.

When connecting using TCP/IP, a port number is required as the mainframe will listen on that port for all incoming requests. SNA configuration requires more input from the mainframe administrator, and the connection has to be configured using the SNA Manager MMC snap-in, shown in Figure 2. For both configurations, the SDK provides worksheets to help you gather the data you need from the mainframe administrator.

Figure 2** Configuring an SNA Connection **(Click the image for a larger view)

Accessing DB2 databases is a breeze with the BizTalk Adapters for Host Data. By using the adapter for DB2, it is possible to connect, read or write data, and execute common SQL commands against DB2. Host Integration Server further simplifies this functionality by providing a Data Access Wizard for testing purposes.

Aside from the BizTalk Adapter functionality, the DB2 adapter also provides a .NET Framework data provider for DB2 that can be used to directly access information in DB2 database servers. The data provider for DB2 is based on the Microsoft network client for DB2, which provides pooling, schema generation, transactions, and static SQL.

After you connect to the database, the BizTalk Port Transport Properties contains the string needed to make the connection available to the BizTalk runtime.

When you have the host adapter for DB2 installed along with the .NET Framework managed provider, you can also access DB2 data on the host programmatically as the code in Figure 3 demonstrates. Debugging and tracing tools are provided for both TCP/IP and SNA protocols.

Figure 3 Accessing a DB2 Database

using Microsoft.HostIntegration.MsDb2Client;
[WebMethod(TransactionOption=TransactionOption.RequiresNew)]

public DataSet GetAccountsFromDB2(string connString)
{
    MsDb2Connection conn = new MsDb2Connection(connString);
    MsDb2DataAdapter da = new MsDb2DataAdapter();
    DataSet ds = new DataSet();

    try
    {
        conn.ConnectionPooling = false;
        conn.Open();
        da.SelectCommand = new MsDb2Command();
        da.SelectCommand.Connection = conn;
        da.SelectCommand.CommandText = "SELECT * FROM MSDB2BANK";
        da.Fill(ds);
        ContextUtil.SetComplete();
    }

    catch(Exception e)
    {
        ContextUtil.SetAbort();
        throw e;
    }

    finally
    {
        conn.Close();
    }

    return ds;
}

Exposing COBOL Apps as Endpoints

The BizTalk Adapter for Host Applications is based on the Transaction Integrator (TI), formerly known as COM Transaction Initiator (COMTI). The adapter extends TI technology to provide COBOL applications and transactions as endpoints to be managed and consumed by BizTalk and .NET clients.

Before digging deeper into the BizTalk Adapter for Host Applications, you need to understand how the programming model affects how you call any COBOL transaction on the host in the design and runtime environments. The importance of achieving connectivity as noted earlier is that it plays a major role in deciding which programming model you will use to call COBOL applications from BizTalk and any .NET application.

The purpose of choosing a programming model is to determine how to interface with the mainframe. The BizTalk Adapter for Host Applications does not install any components on the host, so you're on the hook for figuring out the host-side aspect of integrating with your mainframe applications.

If TCP/IP is the protocol chosen for connectivity, you have two options. The first is the MSLink model, where a listener application resides on the host. The listener intercepts inbound calls and dispatches them to the appropriate COBOL application, as depicted in Figure 4.

Although the diagram looks intimidating, the concepts it shows are not complicated. The best way to simplify it is to borrow an analogous process. If you consider IIS as an example, the listener block in the diagram represents the TCP/IP stack after it is loaded at boot time and bound to a network adapter. Once the IP stack is loaded, IIS is configured to communicate with it via a port list. In Figure 4, this would be the concurrent server. In IIS, the most common port obviously is port 80; in the mainframe world the most common ports are 3000 and higher.

Figure 4** Host App Integration via an MSLink Listener **

The server transaction program (TP) is slightly more complex, but to keep it simple, think of the ASP.NET HTTP ISAPI extension. When IIS receives a page request for a file of type ASPX or ASMX, it hands the request over to the IIS extension. The extension then uses the ASP.NET runtime to process the inbound request, handle the response to the client, and so on.

In the MSLink model, there are three essential components. First is the listener, which is supplied by IBM and used for TCP/IP port and socket management. Once the listener receives an incoming request from the .NET client, it accepts the connection, instructs the .NET client to send the data Transaction Request Message (TRM), and starts the concurrent server.

Second is the concurrent server, which handles incoming requests from the .NET clients, parses them, extracts the transaction the .NET client wants to call, and makes the actual call. It also keeps the connection open waiting on the server TP to respond. Once returned, the socket is closed.

Finally, the server TP executes the end transaction the client actually intended to call. The TP receives the data, invokes the COBOL logic, and returns the result to the concurrent server, which in turn sends it back down to the .NET caller.

The concurrent server contains all the TCP socket-specific code. In fact, Microsoft supplies a sample listener in the SDK called MSCMTICS.CBL for customers as guidance. Using an MSLink listener is the preferred method since it requires no additional changes to the COBOL applications you can call, assuming that they are callable (for example, they have no screen output).

The second approach to TCP/IP connectivity is the User Data model, where each callable COBOL application contains the TCP socket code and can be called directly from within your .NET code as depicted in Figure 5.

Figure 5** Host Integration via User Data and .NET **

Building on the MSLink example, and to maintain consistency in my analogies, I will stay with the IIS metaphor. Compared to the three-step MSLink process in Figure 4, the User Data model combines steps two and three. This would be similar to developing your own IIS Server, say, or Windows Communication Foundation (WCF) channel, to listen in on a port, pick up the inbound request, process it, and return the call back to the client, all in the same running process. In the User Data model, the concurrent server and the end transaction the user intended on calling all reside within one TP on the mainframe.

It is important to note the difference between the two here: the MSLink model is less intrusive since nothing has to be done to the called transaction and all the TCP socket code is handled in a generic listener. Contrast this to the User Data model, which requires the socket code to be present in every transaction the client wants to call. Another important feature to note is that, in the MSLink model, the end transaction is called in a non-blocking manner, whereas in the User Data model the entire process is executed in a single iteration.

If SNA or IP-DLC is the connectivity method in use, these programming models still apply, but there is no need for a listener application since the mainframe supplies a native transaction that acts as a listener called CSMI by default (but could have a different name). This transaction uses a Distributed Program Link (DPL) that acts as a gateway among calling clients and the end transaction the user wishes to invoke as shown in Figure 6. By using a DPL, there is no need to install any software on the mainframe, hence the advantage of this approach to a non-DPL, SNA-based transaction communication. With a DPL the .NET client is disguised as just another mainframe calling a CICS TP.

Figure 6** Host Integration without a Listener **

When the CICS link has been chosen, the .NET client sends the request; the mirror transaction or CSMI then takes control of the inbound request and passes the COMMAREA to the TP after calling it. Once the business logic is invoked and processed, CSMI returns the data back to the client.

Once the programming model has been identified, you can start exposing COBOL applications as endpoints and .NET assemblies. To do so you will need a COBOL copybook from the mainframe developer. The copybook is similar to a C++ structure definition in that it identifies the input and output data the COBOL application expects. The COBOL copybook is imported into Visual Studio using the Import COBOL wizard (see Figure 7).

Figure 7** Importing the COBOL Copybook **(Click the image for a larger view)

After a successful import, a .NET assembly is produced and you have the option of choosing the type restrictions as Web service, None, or BizTalk Adapter for Host Applications. Choosing the latter will generate all the artifacts needed to add this to a BizTalk project and enlist the COBOL transaction in any business process or orchestration managed by BizTalk.

When this step is complete, you'll need to deploy the .NET assembly and provide the adapter with the metadata needed to call the COBOL application. The metadata describes the Remote Environment (RE), also known as the mainframe. The RE is configured by using the BizTalk Adapters for Host Applications TI Manager MMC snap-in.

After the RE is configured, the .NET assembly can be associated with the target RE. You can also now define the RE in the port parameters.

So far you've seen how to expose a COBOL application or transaction from within .NET as a BizTalk endpoint. The imported COBOL copybook exposed the data, and the RE provided the interface to the mainframe where execution of the business logic takes place. To move that business logic to the Windows platform, vendors such as Micro Focus and Fujitsu provide CLR-compliant COBOL .NET compilers and Visual Studio plug-ins to import the COBOL code and run it natively on Windows. These tools can even go a step further and translate the COBOL code to C#, for example.

Integration Beyond Adapters

Two other important integration features are not provided as adapters, but they can be exposed to BizTalk as endpoints: Host Initiated Processing (HIP) and Session Integrator.

HIP allows the mainframe COBOL applications to call any .NET assembly residing on the Windows Server® platform to participate in any BizTalk business process. HIP has design time tools integrated with Visual Studio, and it uses the TI MMC snap-in for managing the metadata needed to host the HIP listener, in addition to the security policies needed to secure the endpoint. The communication transport follows the same guidelines highlighted earlier and Microsoft provides sample COBOL code to test HIP connectivity and functionality.

Session Integrator provides a set of .NET objects that enable you to do screen scraping of COBOL terminal applications. It's a server-based technology and can come in handy if the COBOL applications you are trying to integrate with are not callable and write directly to the screen with the UI not separated from the business logic. Figure 8 shows what a sample screen-scraping application looks like. The data is exposed via a .NET object, which turns the screen canvas into rows and columns that you can read data from or write data to.

Figure 8** Screen-Scraping Application Based on Session Integrator **(Click the image for a larger view)

Wrapping Up

The BizTalk Adapters for Host Integration Server are provided by Microsoft to achieve interoperability and integration with mainframes and AS/400 legacy systems. The adapters enable enlisting and exposing COBOL, DB2, and other mainframe resources as BizTalk endpoints. As you've seen here, using the adapters is certainly a lot less cumbersome than rolling your own integration components due to the complexity of the legacy mainframe environment. In addition, the adapters do not require any footprint on the host, which is appealing to IT departments faced with a continuing shortage of individuals with the skills to develop and manage host applications.

For more information about legacy host integration issues, see the TechNet Host Integration Server TechCenter at technet.microsoft.com/his. You'll find a variety of resources there, including the informative whitepaper "Introduction to BizTalk Adapters for Host Systems" (technet.microsoft.com/library/bb726925.aspx).

Bashar "Bash" Badawi has 20 years of IT experience in developing Microsoft-based solutions and consulting. Bash is currently a Senior Program Manager with the BizTalk/Host Integration Server group based in the field helping customers and partners understand and implement enterprise solutions.