How Venerable Accelerated Growth by Modernizing its Mainframes on AWS

AWS Feed
How Venerable Accelerated Growth by Modernizing its Mainframes on AWS

By Krithika Palami Selvam, Sr. Modernization Architect – AWS
By Gary Crook, CEO – Heirloom Computing
By Niranjan Kulkarni, Project Manager – Cognizant
By Rohit Darji and Ward Britton, Sr. Modernization Consultants – AWS

Venerable needed to modernize a core business application for managing agent commissions. This mainframe application was known as the Agent Management System (AMS).

Venerable was formed as the result of a divestiture from a large U.S. insurance business in 2018. As a new company, Venerable adopted a cloud-first strategy, which meant zero investment in establishing a mainframe infrastructure.

Modernizing the application would allow the company to pursue its business objective of driving growth whilst divesting the legacy mainframe technology from the prior company.

Venerable’s technical goals were to improve performance (especially at peak periods) and provide a disaster recovery (RTO and RPO) capability. All of these goals had to be accomplished within a 24-month timeline.

To achieve this outcome, Venerable engaged AWS ISV Partner Heirloom Computing and AWS Premier Consulting Partner Cognizant. The AMS application was replatformed into a cloud-native Java application running successfully on Amazon Web Services (AWS), through a process known as automated refactoring using the Heirloom product suite.

The project was completed in 16 months—8 months ahead of schedule. The modernization drove cost savings of more than $1 million per month. In addition, Venerable was able to embark on its business acceleration strategy earlier and reduce OpEx costs by more than 80%.

According to Venerable CIO Tim Billow, “The system was custom built in the 1980s, and the design was extremely complex. Despite the complexity and high-transaction counts, the system was successfully replatformed to AWS, where it has been in production since June 2020. This has been a hugely impressive implementation that has enabled Venerable to adhere to its cloud-first strategy.”

In this post, we walk through Venerable’s modernization journey of replatforming its AMS application to the AWS Cloud through automated refactoring.

About Venerable’s AMS Mainframe Application

Venerable’s AMS application contained 3.8 million lines of COBOL and Assembler code on the z/OS-based mainframe. It was an online CICS mainframe application with a small percentage of the base system supported by batch processing. The application was written predominantly in COBOL and used BMS for its screen definitions.

Data for the online application was hosted in VSAM files, but the base system utilized a DB2 database. Users accessed the application through terminals via CICS command-level transactions. Background CICS transactions were also used for batch processing within the CICS environment. These transactions were initiated internally via the CICS START command.

Since AMS provided core business-critical functions for commission calculations, payment generation, license validation, supplemental compensation, accounting, and statements, it was critical the modernized application provide exactly the same business functionality when running on AWS as it did on the mainframe.

However, there were a few key risks during the project. Venerable had to tackle lack of a well-documented application inventory and with limited application knowledge and customer resources to support the modernization.

Solution Overview

To modernize the application, Venerable partnered with Heirloom Computing and Cognizant using the Heirloom product suite. Heirloom is a unique mainframe modernization software solution that uses compiler-based technology to automatically refactor online and batch mainframe workloads (COBOL, PL/I, JCL, etc.) as cloud-native Java applications for deployment to any industry-standard Java Application Server.

Heirloom’s bimodal development feature means the replatformed applications can continue to be developed with the Heirloom Eclipse IDE in the original language, or in Java, or any mix. This flexibility enables organizations to retain SME knowledge, utilize the best available resources, and ensures a well-managed low-risk transition of skills within teams.

The AMS application included 650 online screens and 250 batch jobs, which were successfully replatformed to Amazon Elastic Compute Cloud (Amazon EC2) using Heirloom product suite.

The application data was made up of 110 VSAM files and 25 DB2 tables, which Venerable migrated to relational tables in a Microsoft SQL Server on Amazon Relational Database (Amazon RDS) using the Heirloom Data Migration Toolkit, without any changes to the original application source.

Owing to the archaic software architecture of the AMS application, Venerable used Heirloom Computing’s Probe discovery tool to automatically discover all of the legacy application assets that made up the AMS application. Probe also rapidly identified the asset dependencies with any exceptions or missing items highlighted for remediation by the team.

Overall, using the Heirloom product suite and Probe was considered by the team to be a major factor in the early delivery of the modernization project.

The table below shows the mapping of the different services between legacy on-premises and the AWS Cloud:

Distributed Team Collaboration

The project team, made up of 6-12 people from Cognizant, three Sr. Solution Architects from Heirloom Computing, and the Venerable SME successfully modernized the legacy AMS application into a cloud-native Java application in just 16 months.

The exceptional teamwork between Heirloom Computing and Cognizant coupled with the strong and active support of Venerable executives made this project a huge success.

All of the migration work was done on Amazon WorkSpaces running Microsoft Windows 10. This provided the distributed multi-location team with a powerful, responsive, collaborative, and secure development environment.

Modernization Challenges

While the modernization was completed ahead of schedule, it still was not without challenges.

The application was architected to use multiple background transactions, including pseudo-online transactions initiated by users or generated internally. These complex Program List Table (PLT) initialization and shut down processes, written in CICS, presented technical challenges during migration.

The use of numerous GETMAINs for dynamic memory allocation and addressability, along with the extensive use of pointers, required simulation of z/OS memory addressing within the Heirloom framework. Finally, the application’s VSAM files had variable and multi-record formats, and sorting the mainframe files required collating sequence differences between ASCII and EBCDIC.

Additionally, batch processing consisted of 250 jobs using 2,000+ programs and was done using a CICS application. This impacted the online performance of the application, particularly when processing several thousands of CICS statements that were passing large COBOL records of 128K bytes between programs.

Because of this, the application needed performance architecture changes to meet the throughput requirements of month-end and year-end processing on the target platform. Venerable remediated the application code to replace some of the online background transactions with Java batch processes that were optimized with embedded SQL queries.

Solution Architecture

The following image shows the target deployment architecture of Venerable AMS application on the AWS Cloud.

Figure 1 – Venerable-AWS deployment architecture.

Venerable migrated 15GB of the application data from DB2 database and VSAM on the mainframe to Microsoft SQL Server on Amazon RDS. Venerable also chose to run the modernized application on a 16-core m4.4xlarge Amazon EC2 instance with 64GB RAM.

Once the application was replatformed to the Java platform, its performance exceeded its prior mainframe performance. In addition, Venerable was able to improve application reliability.

Let’s dive into some of the changes that Venerable made to address its application’s use cases.

Disaster Recovery

Venerable created a disaster recovery (DR) strategy based on its business objectives to meet a Recovery Time Objective (RTO) of 4-6 hours and Recovery Point Objective (RPO) of 15 minutes. It implemented a multi-AWS Availability Zone solution, in which an independent copy of the AMS application stack is available on another availability zone (Active/Passive strategy).

Venerable chose to use Elastic Load Balancing to achieve fault tolerance. Using Application Load Balancer, Venerable can check the health of the EC2 instance that its application is running on without needing manual intervention. If the load balancer detects the primary instance is unhealthy, it raises an alert via Amazon CloudWatch and immediately routes the traffic to the failover DR instance in the other availability zone.

Protecting Data in Transit

To encrypt data in transit, Venerable used X.509 SSL/TLS certificates when downstream payment files were sent to third parties for payment processing.

Secure Secrets Storage

To securely store and manage secrets like passwords, encryption keys, and database credentials, Venerable used AWS Secrets Manager. The company replaced hardcoded credentials in the application source code with an API call to Secrets Manager to programmatically retrieve secrets.

Solution Walkthrough

Step 1: AMS Application Discovery
(Duration – 1 month)

In the early stages of discovery, Venerable completed a detailed analysis of the AMS application with inventory baselining using Heirloom’s Probe software.

Probe is designed to accelerate delivery of mainframe modernization projects through automated application inventory analysis and collection and process orchestration. It was used in this project to produce an application map, collect application assets, identify complexity and dependencies, and extract metadata required for code refactoring and data migration.

Step 2: Conversion of Online Screens and Easytrieve Programs | Data Migration
(Duration – 3 months)

The CICS BMS screens were automatically refactored by Heirloom into modern HTML5/JavaScript web pages. The user interface retained the look and feel of the existing BMS online screens, so the end-user experience was not changed during the migration.

Figure 2 – Sample BMS screens converted to HTML5 / JavaScript web pages.

Venerable converted the Easytrieve programs to intermediate COBOL programs using Cognizant’s proprietary tools. Since the VSAM datasets consisted of non-sequential segments with variable record layouts, custom Python scripts developed by the Cognizant team were integrated with Heirloom to convert the VSAM data into the relational data format.

The VSAM-to-SQL data transparency was a key feature in the Heirloom framework that enabled the successful conversion of VSAM files to Microsoft SQL Server on Amazon RDS without needing to make any application code changes.

Step 3: Conversion of Assembler Programs | Data Migration Validation | Batch Jobs Migration
(Duration – 2 months)

To convert Assembler programs to COBOL, Venerable used a semi-automated approach using Cognizant’s proprietary pseudo-code generator. The converted COBOL programs were subsequently refactored to Java using Heirloom.

JCL jobs were executed by Heirloom’s batch environment, which provided a complete JES/JCL implementation. This included all of the IBM utilities within the application. In parallel, the data migration was validated to ensure an exact match between the mainframe datasets (DB2 and VSAM) and the relational database tables.

Step 4: Refactoring of COBOL Programs | Security Migration | Scheduler Migration
(Duration – 4 months)

Using Heirloom’s compiler, Venerable recompiled COBOL online transaction processing and batch processing workloads for execution on the Java platform. Because the Heirloom Framework had native CICS support for Temporary Storage Queues (TSQ) and Transient Data Queues (TDQ), code remediation was not required during refactoring. This accelerated the modernization.

For the RACF Security-to-Active Directory conversion, the access rules for mapping users, groups, and resources were unloaded from the mainframe RACF database and migrated to AWS Directory Service. The Heirloom framework ensured all of the application access rules are replicated in the new environment.

The Tivoli Workload Scheduler (TWS) handled scheduling batch processes on the mainframe. These processes were then migrated to a Linux version of TWS running on AWS.

Step 5: Testing
(Duration – 5 months)

Testing was critical to validate the functional equivalence between the mainframe application and the refactored Java application. Complete test coverage was essential to achieve functional equivalence.

Unit Testing: (Duration: 2 months)

• Unit testing took 2 months and ensured each class/method in the refactored Java programs was tested independently by mocking-up dependencies.

System Integration Testing, User Acceptance Testing (Duration: 3 Months)

• System integration testing was done to verify the behavior of the complete system, including interactions with other systems.
• In parallel, integration testing was done to verify communication and interactions between components to detect interface defects.
• User acceptance testing was done to validate that the application met the business objectives.

Step 6: Production Deployment
(Duration: 1 month)

A CI/CD pipeline was created to automate the building, testing, and deployment of applications on the AWS Cloud. Venerable used Jenkins on Amazon EC2 for build and deployment. The company also used AWS CloudFormation templates to provision the different environments in a fast and consistent manner.

After all the pre-defined launch criteria were met, the application was promoted to production and deployed.

Conclusion

Complex, poorly documented mainframe systems can be efficiently and effectively modernized to a cloud-native architecture. Venerable achieved this modernization by executing deep-dive discovery using automated tools and an agile approach to migration.

With active engagement from application SMEs, Venerable was able to use an automated refactoring pattern to migrate its mainframe application quickly.

Working together as one team, Venerable, Cognizant, Heirloom Computing, and AWS managed to modernize a complex legacy mainframe application onto a modern technology architecture. The team completed the migration within 16-months of the original 24-month timeline.

Modernizing to AWS allowed Venerable to build an acquisition engine to underpin business expansion and also divest its technology stack from its parent company. In addition, Venerable reduced OpEx costs by 80% by migrating its mainframe infrastructure from on premises to AWS.

AWS can help you assess how your company can get the most out of cloud. Join the millions of AWS customers that trust AWS to migrate and modernize their most important applications (including mainframe applications) in the cloud.

To learn more about mainframe migration, visit Mainframe Modernization with AWS and AWS Migration Acceleration Program (MAP) for Mainframe.

Contact AWS to start your migration journey today!