CONTINUOUS TESTINGMAINFRAME TESTINGThe Case for DevOps on the Mainframe in 2020

Despite very real market pressures, mainframe testing has been slow to evolve. To find out why, we recently surveyed 100 IT executives. Read the results here, to learn more about real devops advancements for the mainframe in 2020
Jeff Hughes Jeff HughesMay 6, 20205 min

The Death of the Mainframe has been Greatly Exaggerated! Pundits have been predicting its demise for many years. Nothing could be further from the truth. The mainframe is alive and well and thriving. These workhorses continue to be a strategic asset powering today’s application economy – acting as systems of record for vast numbers of new web-based and mobile apps. Once relegated to run only COBOL-based applications, the mainframe has adopted Linux OS and other programming languages including Python, Java and C++ to name a few. Mainframes have effectively shaken their dinosaur image and run many newer cutting edge applications. For example, did you know:

  • 90% of new customer-facing apps rely on mainframe connectivity.
  • 55% of enterprise applications touch the mainframe.
  • 80% of corporate data resides on mainframes.

Because so many applications run on mainframe and distributed environments, mobile-to-mainframe as they say, there is a growing need for implementing DevOps practices on these systems. Automated code reviews, automated unit testing, code coverage testing, continuous integration and automated deployment are all benefits of implementing a DevOps strategy on the mainframe. One important, but neglected, aspect of DevOps on the mainframe is testing. The benefits of testing early and often in a distributed environment can also be realized on the mainframe.


But, testing on the mainframe has been slow to evolve. The reasons are numerous, but this recent devops on the mainframe survey sheds new light on this challenge. To find out why, we recently surveyed 100 IT executives. The individuals in this survey represent a wide variety of industries, including business services, consumer services, financial services, healthcare, manufacturing, retail, telecommunications, transportation, utilities, and wholesale trade. More than half work in Fortune 1000 companies with revenues of over $1.5 billion.


The first reason cited in the survey for not doing testing on the mainframe is that more than half of the test teams are still relying on manual, waterfall approaches. This means that they write the code and only do unit or regression testing to determine suitability for a production release. Little to no pre-production testing is done. When companies adopt an agile approach to development on the mainframe they quickly see the need to adopt agile testing practices to keep pace with their accelerated application development. It appears from the survey that most companies want to move towards agile testing on the mainframe, but there are a number of barriers that will require new tools along with management support. The following table from the survey illustrates this.


Application Testing Challenges

Other Barriers to Slow Adoption


There are other barriers that are contributing to slow adoption of testing on the mainframe and they include:


  • Mainframe Access. Test teams have limited access to their mainframe. Often, they have limited hours in which they can do testing on the mainframe. There are costs associated with mainframe access that make it prohibitive to do frequent testing required in an agile environment. Tools such as Service Virtualization can mimic mainframe responses, addressing access and cost constraints associated with testing. 


  • Test data. Data is a big issue. Many teams lack sufficient data for testing, and what data they do have is often flawed. They also find it difficult to mask personally identifiable information as mandated by the European Union and other governing bodies around the globe.  Tools such as Test Data Manager also support mainframe application testing, delivering the right test data whenever needed.


Achieving DevOps on the Mainframe


Like any successful DevOps approach, changes in people, process and technology are required. The focus of this mainframe survey was primarily on technology needs and where respondents felt there was the greatest discrepancy. About two-thirds of the executives in the survey said that automated test execution topped their wish list of improvements. They were also eager to optimize test coverage, improve the quality of applications delivered, accelerate time to market, generate test data more quickly and lower overall test program costs. 


The survey goes on to say that more than six out of 10 are investing in test automation, while nearly half have adopted new tools or strategies for load and performance testing as shown in the following table. 


Desired test program improvements

The key to your DevOps success is to start by leveraging existing tools that already work in your mainframe environment. This is perhaps the easiest and least expensive approach. For example, if you are already using Service Virtualization in your distributed environment you should look for ways to utilize this technology in the mainframe environment. If you’re using a test data management solution you should explore if that solution will also work on the mainframe.


Don’t Fear the Mainframe


The mainframe should not be considered an afterthought in your testing plans. Your DevOps strategy should include cross platform testing. Your enterprise depends on the reliability and raw processing power of the mainframe and many of your applications that utilize or run on the mainframe need to be tested on the mainframe. With the right tools you can minimize testing’s impact on the mainframe while ensuring that your applications will perform the way they were designed.

The benefits are clear. Modernizing your mainframe testing can help you work more efficiently, reduce costs, improve time to market and drive higher levels of customer satisfaction. But for the best return on investment, you need the right testing tools. 

To learn how to jumpstart your continuous mainframe application testing program, visit the following links:

Broadcom Test Data Manager 



Jeff Hughes

Jeff Hughes

Jeff Hughes is a product marketing engineer for Broadcom. He has over 20 years experience in technology marketing with emphasis on testing, application development, security, Cloud and network technologies. Jeff joined CA/Broadcom in May, 2015 and provides marketing support for Test Data Manager and Service Virtualization. He is the author or 11 books on technology and marketing along with numerous ebooks and white papers. He has been a speaker at global conferences and trade shows and has been a regular presenter at CAWorld for the past three years.