Mainframe and midrange modernization with Azure Logic Apps
This guide describes how your organization can increase business value and agility by modernizing your mainframe and midrange environments using Azure Logic Apps. The current business world is experiencing an era of hyper-innovation and is on a permanent quest to obtain enterprise efficiencies, cost reduction, growth, and business alignment. Organizations are looking for ways to modernize, and one effective strategy is to augment the business value while using existing legacy assets.
For organizations with investments in mainframe and midrange systems, this means making the best use of platforms that helped send humans to the moon or helped build current financial markets and extend their value using the cloud and artificial intelligence (AI). This scenario is where Azure Logic Apps and its native capabilities for integrating with mainframe and midrange systems come into play, by opening the door to the AI world for legacy investments. Among other features, Azure Logic Apps incorporates the core capabilities of Host Integration Server (HIS), which has been used for mainframe and midrange integration at the core of Microsoft's most strategic customers over 20+ years. As a result, Azure Logic Apps has become an Integration Platform-as-a-Service (iPaaS) for mainframe and midrange systems.
When enterprise developers build integration workflows with Azure Logic Apps, they can more quickly deliver new applications using little to no code or less custom code. Developers who use Visual Studio Code and Visual Studio can be more productive than those who use IBM mainframe development tools and technologies because they don't require knowledge about mainframe systems and infrastructure. Azure Logic Apps empowers business analysts and decision makers to more quickly analyze and report vital legacy information. They can directly access data in mainframe data sources, which removes the need to have mainframe developers create programs that extract and convert complex mainframe structures.
Cloud native capabilities for mainframe and midrange system integration
Since 1990, Microsoft has provided integration with mainframe and midrange systems through Microsoft Communications Server. Further evolution of Microsoft Communications Server created Host Integration Server (HIS) in 2000. While HIS started as a System Network Architecture (SNA) Gateway, HIS expanded to include IBM data stores (DB2, VSAM, and Informix), IBM transaction systems (CICS, IMS, and IBM i), and IBM messaging (MQ Series). Microsoft's strategic customers have used these technologies for more than 20 years.
To empower customers that run applications and data on Azure to continue using these technologies, Azure Logic Apps and Visual Studio have gradually incorporated these capabilities. For example, the HIS Designer for Logic Apps that runs on Visual Studio, and the 3270 Design Tool, help you create metadata artifacts required by the built-in connectors that you use for mainframe and midrange integration in Azure Logic Apps. These built-in connectors run using the same compute resources as Standard logic app workflows. This design not only allows you to achieve low-latency scenarios, but also extends your reach to address more disaster recovery and high availability customer needs.
For more information about the Microsoft's capabilities for mainframe and midrange integration, continue to the following sections.
Microsoft HIS Designer for Logic Apps
This tool creates mainframe and midrange system metadata artifacts for Azure Logic Apps and works with Microsoft Visual Studio by providing a graphical designer so that you can create, view, edit, and map metadata objects to mainframe artifacts. Azure Logic Apps uses these maps to mirror the programs and data in mainframe and midrange systems. For more information, see HIS Designer for Logic Apps.
Microsoft 3270 Design Tool
This tool records screens, navigation paths, methods, and parameters for the tasks in your application so that you can add and run those tasks as 3270 connector actions. While the HIS Designer for Logic Apps targets transactional systems and data, the 3270 Design Tool targets 3270 applications. For more information, see 3270 Design Tool.
Azure Logic Apps connectors for IBM mainframe and midrange systems
The following sections describe the built-in, service provider-based connectors that you can use to access and interact with IBM mainframe and midrange systems when you create Standard workflows in Azure Logic Apps.
Note
Although some of the following connectors are available as "shared" connectors that run in global Azure, this guide is focused on the built-in, service provider-based connectors, which are available only when you create Standard workflows in Azure Logic Apps.
IBM 3270
This Azure Logic Apps connector for 3270 allows Standard workflows to access and run IBM mainframe applications that you usually drive by navigating through 3270 emulator screens. The connector uses the TN3270 stream. For more information, see Integrate 3270 screen-driven apps on IBM mainframes with Azure by using Azure Logic Apps and IBM 3270 connector.
IBM Customer Information Control System (CICS)
This Azure Logic Apps connector for CICS provides Standard workflows with the capability to interact and integrate with CICS programs using multiple protocols, such as TCP/IP and HTTP. If you need to access CICS environments using LU6.2, you need to use Host Integration Server (HIS). For more information, see Integrate CICS programs on IBM mainframes with Standard workflows in Azure Logic Apps using the IBM CICS connector.
IBM DB2
This Azure Logic Apps connector for DB2 enables connections between Standard workflows and DB2 databases that are either on premises or in Azure. The connector offers enterprise IT professionals and developers direct access to vital information stored in DB2 database management systems. For more information, see Access and manage IBM DB2 resources using Azure Logic Apps.
IBM Host Files
This Azure Logic Apps "connector" for Host Files provides a thin wrapper around the "Flat File Parser" feature in Host Integration Server. This offline "connector" provides operations that parse or generate binary data to and from host files. These operations require this data to come from any trigger or another action that produces binary data. For more information, see Parse and generate IBM host files using Azure Logic Apps.
IBM i
This Azure Logic Apps connector for IBM i lets Standard workflows interact and integrate with COBOL and RPG programs running on IBM i systems using TCP/IP. If you need to access IBM i environments using LU6.2, you need to use Host Integration Server (HIS). For more information, see Integrate COBOL and RPG programs on IBM midranges with Standard workflows in Azure Logic Apps using the IBM i connector.
IBM Information Management System (IMS)
This Azure Logic Apps connector for IMS uses the IBM IMS Connect component, which provides high performance access from Standard workflows to IMS transactions using TCP/IP. This model uses the IMS message queue for processing data. For more information, see Integrate IMS programs on IBM mainframes with Standard workflows in Azure Logic Apps using the IBM IMS connector.
IBM MQ
This Azure Logic Apps connector for MQ enables connections between Standard workflows and IBM MQ servers on premises or in Azure. Microsoft also provides IBM MQ integration capabilities with Host Integration Server and BizTalk Server. For more information, see Connect to an IBM MQ server from a workflow in Azure Logic Apps.
Challenges for mainframe and midrange systems modernization
Mainframe and midrange systems can host multiple environments that contain programs, data, files, and tools. Over the years, these environments might not have been refactored or were left to grow and reach their limits, despite hardware upgrades. These environments might also have been maintained by multiple developers and IT admins, who follow different programming patterns and techniques, or recruited other parties to help with tasks that require expertise scarce in the market. Along with a shrinking pool of experienced professionals, all these factors create a complex and challenging job of modernizing mainframe and midrange environments.
While the following list isn't comprehensive, defining a successful modernization strategy minimally includes ways to handle the following tasks:
- Maintain the current service level indicators and objectives for your environments.
- Manage coexistence between legacy data along with migrated data.
- Conduct DevOps across environments during coexistence.
- Manage application interdependencies.
- Define the future of the mainframe scheduler and jobs.
- Define a strategy for replacing commercial off-the-shelf (COTS) products.
- Conduct hybrid functional and nonfunctional testing activities.
- Maintain external dependencies or interfaces.
With these tasks in mind, customers typically choose any of the following paths to conduct mainframe and midrange systems modernization:
Big bang
This approach is largely based on the waterfall software delivery model but with iterations in phases. The big bang approach is adopted more by customers with small mainframe or midrange systems and low complexity environments due to a low number of lines of code, low application density, and well-known legacy systems or programming languages.
Agile waves
This approach follows the Agile principles of software engineering. The Agile waves approach is adopted more by customers with larger mainframe or midrange systems and high complexity environments due to a high number of lines of code, high application density, lesser-known systems or programming languages, and a high number of dependencies and interfaces.
The choice between these paths depends on your organization's needs and scenarios. Each path has benefits and drawbacks to consider. The following sections provide more information about these modernization approaches.
Big bang or waterfall
A big bang migration typically has the following phases:
Envisioning: Kickoff
Planning: Identify and prepare planning deliverables, such as scope, time, and resources.
Building: Begins after planning deliverables are approved
This phase also expects that all the work for dependencies has been identified, and then migration activities can begin. Multiple iterations occur to complete the migration work.
Stabilizing or testing: Begins when the migrated environment, dependencies, and applications are tested against the test regions in the mainframe environment.
Deploy: After everything is approved, the migration goes live into production.
Organizations that typically choose this approach focus on locking time, migration scope, and resources. This path sounds like a positive choice but includes the following risks:
Migrations can take months or even years.
Deployments to production are riskier.
The analysis that you perform at the start of the migration journey or during planning is no longer accurate because that information is usually outdated.
Organizations typically focus on having comprehensive documentation to reduce delivery risks for delivery.
However, the time spent on providing planning artifacts causes exactly the opposite effect. Focusing on planning more than executing tends to create execution delays, which cause increased costs in the long run.
Agile waves
An Agile approach is results oriented and focused on building software and not planning deliverables. The first stages of an Agile delivery might be chaotic and complex for the organizational barriers that need to break down and to align the migration team. However, after the migration team matures following several sprints of execution, the journey becomes smoother. The goal of this approach is to frequently release features to production and to provide business value sooner than with a big bang approach.
An Agile waves migration typically has the following sprints:
Sprint zero (0)
- Define the team, an initial work backlog, and the core dependencies.
- Identify the features and a Minimum Viable Product (MVP) to deliver.
- Kick off mainframe readiness with a selected set of work items or user stories to begin the work.
Sprint 1, 2, ..., N
Each sprint has a goal where the team maintains a shipping mindset, meaning that they focus on completing migration goals and releasing deliverables to production. The team can use a group of sprints to deliver a specific feature or a wave of features. Each feature includes slices of integration workloads.
Shared elements, such as jobs and interdependencies, exist and have impact across the entire environment. A successful strategy focuses on partially enabling jobs, redesigning applications for modernization, and leaving the systems with most interdependencies until the end to first reduce the amount of migration work and then complete the scope of the modernization effort.
Microsoft recommends modernizing mainframe and midrange system workloads by following an iterative, Agile waves-based model by focusing on investments in the new platform, while limiting the growth of legacy systems. This approach considerably reduces implementation risks by preserving the existing business value, while introducing the modernized environment. That way, your team can also leverage technology skills that help your business be more competitive. This scenario is where Azure Logic Apps can help you in your modernization journey.
Modernization patterns
Good design includes factors such as consistency and coherence in component design and deployment, maintainability to simplify administration and development, and reusability that allows other applications and scenarios to reuse components and subsystems. For cloud-hosted applications and services, decisions made during the design and implementation phase have a huge impact on quality and the total cost of ownership.
The Azure Architecture Center provides tested design and implementation patterns that describe the problem that they address, considerations for applying the pattern, and an example based on Microsoft Azure. While multiple design and implementation patterns exist, some of the most relevant patterns for mainframe modernization include the "Anti-corruption Layer", "Strangler Fig", "Saga", and "Choreography" patterns.
Anti-corruption Layer pattern
Regardless which modernization approach that you select, you need to implement an "anti-corruption layer" using Azure Logic Apps. This service becomes the façade or adapter layer between the mainframe legacy system and Azure. For an effective approach, identify the mainframe workloads to integrate or coexist as mainframe integration workloads. Create a strategy for each integration workload, which is the set of interfaces that you need to enable for migrating a mainframe application.
For more information, see Anti-corruption Layer.
Strangler Fig pattern
After you implement the anti-corruption layer, modernization progressively happens. For this phase, you need to use the "Strangler Fig" pattern where you identify mainframe workloads or features that you can incrementally modernize. For example, if you choose to modernize a CICS application, you have to modernize not only the CICS programs, but most likely the 3270 applications along with their corresponding external dependencies, data, and jobs.
Eventually, after you replace all the workloads or features in the mainframe system with your new system, you'll finish the migration process, which means that you can decommission your legacy system.
For more information, see Strangler Fig pattern.
Saga and Choreography patterns
Distributed transactions such as the two-phase commit (2PC) protocol require that all participants in a transaction to commit or roll back before the transaction can proceed. Cloud hybrid architectures work better following an eventual consistency paradigm rather than a distributed transaction model.
The "Saga" design pattern is a way to manage consistency across services in distributed transaction scenarios. A saga is a sequence of transactions that updates each service and publishes a message or event to trigger the next transaction step. If a step fails, the saga executes compensating transactions that counteract the preceding transactions. For more information, see Saga distributed transactions pattern.
In Azure Logic Apps, workflows can act as choreographers to coordinate sagas. Workflow actions are atomic, so you can rerun them individually. The Scope action type provides the capability to run a group of actions only after another group of actions succeed or fail. Azure Logic Apps conducts compensating transactions at the scope level, while Azure Event Grid and Azure Service Bus provide the event management required for specific domains. All these services, which make up Azure Integration Services, provide the support required by customers when they need a reliable integration platform for mission critical scenarios. For more information, see Choreography pattern.
While this article covers several modernization patterns, complex solutions require many more patterns and that you clearly understand your organization's modernization goals. Although the task to extend the value of legacy assets is challenging, this option is the best way to preserve investment in these assets and prolong their business value.