Most organizations still use manual testing for majority of their quality assurance plan, despite it offering no sensible way to measure coverage or to manage changes. Studies show 70% of all testing is still manual, with only 69% of functionality delivered by the code developed. In addition, manual testing is riddled with ambiguity, with 56% of defects stemming from unclear requirements. Even when test requirements are captured in natural language, it can be hard to ensure you are on the same page with other testing and development team members, much less your company’s business process owners. It is no wonder that over 60% of IT projects fail.
But the inability of manual testing to keep pace with Agile development is only part of the problem. In today’s Internet of Things (IoT) economy, with the proliferation and complexity that microservices development brings, development teams are struggling to embrace API tests as a critical component of effective QA testing in-sprint. With 20 billion devices expected to be connected to the Internet by 2020, API testing is used far less frequently and effectively than it should be.
3 Challenges to Consider when Using API Testing In-Sprint
For the most part, development teams are focused only on testing for a ‘happy path’ and ensuring that APIs work as designed under optimal circumstances. Seldom do in-sprint reviews account for unlikely paths (test scenarios). It is simply too difficult to validate the many ways a user might manipulate the application and related sub-applications, much less automatically generate test designs optimized to provide complete test coverage.
API testers on QA teams are faced with very real world challenges when modeling for tests within a single API (or multiple APIs involved in a chain of end-to-end requests). It is difficult to address the different permutations of test scenarios they need to validate. It is worthwhile to note that any API testing solution needs to first resolve three important challenges:
- Keeping Pace with Agile Development
API tests often can be run in milliseconds. But with manual API test design, we aren’t realizing the full advantage of the speed that API tests can offer. If you design and code every test by hand and can only complete 20 tests a day, you’ve clearly lost any speed advantage. Those losses mount exponentially as manual modifications are made in response to ongoing changes. Clearly, it is daunting to maintain test validity in the face of the proliferation and complexity of APIs driving an IoT economy. API test designs need to keep pace with Agile development and accelerate for test automation in-sprint.
2. Visualizing Complex Dependencies
APIs don’t stand alone. They support varying environments and depend on other APIs and services – creating a tangled web of dependencies that you do not control. API testers undertaking functional unit testing have to understand all the nested connections that need to be addressed. But with limited visibility and traceability of these dependencies, most organizations are shackled by slow and incomplete test designs that address a very low percentage of the different logical combinations of components involved in the function of any API. Teams need to be able to fail faster in-sprint for more immediate feedback. API tests that are light on visibility and traceability fails to prioritize what needs to be tested. This creates resource waste from inefficient test coverage, to increasing technical debt and costly rework.
3. Orchestrating for Completeness of Testing
Most API testing depends on the state of the production environment and the database operating in the background. For example, if you have an API that controls the types of information various users will be able to access, you need user data to test how your API functions as a gateway. That means you must prepare the relevant test data ahead of time and match it to your tests to ensure it is stable and repeatable. That includes making sure that your test data has been desensitized to remove personally identifiable information and that it is data regulation compliant. Teams need to be able to eliminate test delays by optimizing test designs to be configured to requirements direct from their local machines. Orchestrating test services for completeness in testing, to be shared and reused across the business to create fuller tests.
A New Model-Based Approach for API Testing
It’s time for a better, faster way to design API tests and maintain them over time. One highly effective solution is to use model-based testing tools that automatically generate tests based on models of the system under test. With model-based testing, you don’t worry about test design or priorities. You simply build a flow chart of the logical function of your API. You capture things that could happen or conditions that might be encountered and how those impact outcomes. You end up with a visual model that provides a clear, unambiguous roadmap of how you expect your API to behave.
One bonus with model-based testing is that no special skills are required. Modelling can be readily understood by both experienced testers and novice team members from the business side of the house. That means everyone stays on the same page. Once built, the model does the heavy lifting by automatically generating the test cases needed to verify that your API functions as it should along all the pathways in your model. And when something in your API environment changes, you simply change the model, not the tests. Your modeling tool generates new tests that you can immediately execute, without having to intervene manually.
Test cases can be executed by your test technology of choice to deliver the precise level of test coverage you need. That means no over testing or under testing. You can run an exhaustive set of all possible test cases, or tell your requirements modeling tool to optimize instead – building the minimum number of test cases for a specific coverage requirement.
Taking the Next Step
Are you ready to make your API testing more effective? It’s time to adopt a model-based testing program that has the power and flexibility you need – allowing you to drag and drop blocks that represent each aspect of the functionality you want to test.
Watch our on-demand webinar on Model-Based API Testing, and you will see how you can capture all the dependencies and possible outcomes from start to finish – from the big picture down to each detailed step. Learn how adopting an automated approach to requirements modeling and test design lets you automatically generate an optimal set of test cases for each of the pathways through your model – and do so in the blink of an eye.
When you’re ready, sign up for a free trial of Agile Requirements Designer and start building your model for maximum API test coverage. You will be able to map all the pathways that determine which assertions will be true as you establish the logical flow, and then automatically generate the API tests that can be used by anyone on your team. Changes become highly efficient as you simply update your model – with Agile Requirements Designer determining which of your many API tests will be affected and automatically updating them. It couldn’t be simpler.
- Acceptance Testing
- Agile Development
- Agile Testing
- Application Testing
- Automated Testing
- Behavior Driven Development
- Continuous Development
- Continuous Integration
- Functional Testing
- Model Based Testing
- Open Source
- Performance Engineering
- Requirements Testing
- Test Case Optimization
- Test Center of Excellence
- Test CoE
- Test Collaboration
Collin enjoys helping traditional Central IT and Agile development teams Plan-Build-Test-Run for speed in quality application release. Taking on grassroots engagements with practitioners in application test automation for the Agile software development lifecycle and DevOps delivery, Collin has broken new grounds in next generation programs like predictive insights for Dev-QA teams, and hybrid cloud management solutions for Ops teams in the software defined datacenter.