Continuous Testing: Build The Thing Right & Build the Right Thing
In the world of DevOps and Continuous Delivery, testing has quickly emerged to be the bottleneck to the process that aims to constantly put new features into the hands of users. Not surprisingly, Continuous Testing has now become a “buzz-word worthy” concept. What is continuous testing and why should business owners care? What are its business benefits?
What is continuous testing? The Definitive Guide For Continuous Testingdescribes it as follows – “Continuous Testing is the practice of testing across every activity in the Software Development Life Cycle (SDLC) to uncover and fix unexpected behaviors as soon as they are injected.” Continuous Testing can be seen as a set of best practices and supporting tools linked together in the CICD tool chain that enable companies to constantly test incremental software code, with intelligence in selecting the tests for optimal test efficiency and effectiveness.
Continuous Testing is a key enabler of Continuous Delivery. Without CT, CD runs the risk of putting capabilities into production with hope and wish for quality and value. CT can be practiced for on-prem software or SaaS-based software. It can be applied to traditional architecture applications, but it is best applied to new micro-services and SaaS based applications.
For business owners or product/service managers, Continuous Testing ensures the team is building the thing right, as well as build the right thing.
Build the Thing Right
“Building the thing right” refers to building high quality applications. Broadly speaking, from users’ perspective, “quality” of an application encompasses several aspects. Here listed as increasingly higher “bar” of quality:
- Application stability and performance: at the very basic level, applications that often crash while users are interacting with them or are very slow in rendering are generally considered low in quality.
- Feature function that behave as advertised: applications that can not accomplish users’ expected task flows or perform expected feature functions are generally considered low in quality.
- Sophistication in its feature / functions: beyond performing basic functions, applications that offer unexpected capabilities, ones that give pleasant surprises are often considered ingenious or higher quality.
- Ease of use / intuitiveness of the application: high quality applications allow users to accomplish complex tasks, yet are easy to use, require minimal to no training or references to documentation.
Continuous Testing help accomplish all of these goals. Testing requires resources – compute cycles for test execution, manual test case creation and maintenance, generation of test data or test environment – all require time, effort, and money. Therefore, the ability to constantly update test coverage, identify best tests to include in nightly test suites, and pinpoint high risk areas of the codebase can ensure more rigorous testing.
With end-to-end traceability and model based testing, product owners and developers can create test scenarios from detailed user story requirements and task flows. A requirement can be traced from its definition, to code, to test. Continuous and intelligent testing, by automatically updating test cases when new features are developed, analyze code and service dependency maps, incorporating existing feature usage information, can optimize daily test regimes.
Continuous Testing can also constantly kick off load tests and code scans to ensure performance and security at every granularity of code. When working together, a continuous testing system can ensure stability and performance of the application. It can also ensure that the feature is behaving exactly as the business owner designed and intended.
Build the Right Thing
Perhaps even more important than ensuring an application is built right is that we are building the right application, one whose features will solve users’ problems, bring value to the customers, and are pleasing to experience. Continuous Testing help with this. By breaking down the unit of features to be tested and make them rapidly available to users, Continuous Testing shortens the time gap between design and usage, accelerate the user valuation cycle, and pin point the area of testing. When used in conjunction with feature flags and canary deployments, continuous testing presents end-users constantly with slightly updated capability and user experience. Based on customer feedback, the new capability can be rolled out more broadly or rolled back. The agility from this continuous delivery model ensure that over time we are building the right set of features and giving customers the most intuitive experience.
Continuous Testing Resources
To help you on your Continuous Testing journey, we have recently published “The Kitty Hawk Venture,” a novel about Continuous Testing in DevOps to support Continuous Delivery and business success.
As a companion piece to “The Kitty Hawk Venture,” you can also download the “Definitive Guide to Continuous Testing” so you can learn the strategies, technologies and techniques that go into a successful CT program.