This month Broadcom released a new version of Automic Continuous Delivery Director that empowers intelligent continuous testing to drive high-quality releases against increasingly shorter timelines.
Authors Jez Humble and David Farley stated, “The most common failure mode, by far, is that there is too little automated testing, not too much.” The challenge with test automation is how to execute thousands of tests against frequent releases without extending agile release cycles or degrading the quality of each release.
The new version of Automic Continuous Delivery Director (CDD) is designed to address these challenges and more. CDD is already used by companies around the globe for complete continuous delivery orchestration. Available as code or with an intuitive UI, CDD is designed to empower agile teams to collaborate and deliver new capabilities with higher efficiency and lower risk.
Now, we have enhanced CDD to further address the number one bottleneck in continuous delivery pipelines, testing. The practice of continuous testing is designed to execute automated tests as part of software delivery pipelines. Embedding automated tests within continuous delivery pipelines drives feedback to developers and the line of business.
How does the new version of Automic Continuous Delivery Director improve the quality and velocity of high-frequency releases?
Reducing Test Cycle Times
Automic Continuous Delivery Director reduces test cycle times by intelligently selecting and executing the most relevant tests associated with each build. Why run an entire bank of test suites against the small changeset that exists within an agile sprint? For example, if you’ve added the ability to log in with Facebook, you don’t need to test the checkout process.
How does this continuous delivery insight functionality reduce test cycle times? Automic Continuous Delivery Director uses heuristics and machine learning against four data points.
- Code to test suite mapping.
- Weighing the stability of tests.
- Noticing new or updated tests.
- Manually marked mandatory tests.
Code to Test Mapping
Mapping code files to test executions allows Automic Continuous Delivery Director to select the test suites that are most relevant to a particular build. Since agile methodologies mandate frequent small changes, it is a waste of time to run every test when a small subset may be implicated by the code that changed. Intelligently selecting the tests that correlate to the code changed in a build reduces test cycle time while maintaining the quality of the release.
Impact of Test Stability
We all know that many tests are flaky. Tests can be flaky or unstable for any number of reasons, and these unstable tests can be a sinkhole of wasted time. Flaky tests can skew your test score results or introduce false results that have to be looked into and diagnosed. Automic Continuous Delivery Director intelligently weighs the stability of tests and will automatically prioritize flaky tests to run first in a test cycle. This prioritization will provide you with the fail-fast capability prescribed by DevOps and allow you to dismiss failed tests quickly.
Testing the Tests
Any time you create a new test or update an existing test, you have inserted change into your continuous delivery pipeline. Change equates to uncertainty and risk. We usually don’t think of needing to follow agile practices for test development, but the need exists. Automic Continuous Delivery Director prioritized new or updated tests to run early in the test cycle. This way, you will quickly know if these changes have negatively impacted your continuous testing cycle. If the updated tests fail, you can make the necessary changes and immediately rerun the test cycle. Remember, the entire bank of tests are not rerun; just the most important or relevant subset will execute.
The Manual Override
Yes, machine learning and heuristics are helpful, but they are just tools. No algorithm should be a complete replacement for human judgment. Automic Continuous Delivery Director allows authorized users to mark tests as mandatory. Mandatory tests will always run for every build. But we didn’t stop there. We, humans, develop confidence in ML algorithms if we can understand why the algorithm made the decision it did. Automic Continuous Delivery Director provides intuitive graphs and details so you will understand the logic behind its decisions.
To learn more about Automic Continuous Delivery Director and try the free starter edition, visit: CDDirector.io.