TEST ENVIRONMENTSShift Happens: API Testing Reduces the Pile

Avatar Steve FeloneyAugust 2, 20185 min

A Developers Life: Coding Eight Days A Week? Not Really.

If you are coding only 1 day a week (2 days out of a sprint) you are not alone.  Not even “Eight days a week” would help that much (Yes, I’m a Beatles fan).    8 hours a week of coding is all developers, on average, are able to achieve. Why is that? Because of all the shift that is happening.  “A day in the life” of a developer is coding, testing, documenting, validating, releasing etc….  Developers now must “Carry that weight” of tasks that used to be spread across multiple groups, such as API Testing.

Not only do developers have more work to do, they are supposed to get it all done in sprint and have the code be part of a continuous delivery pipeline.  This is a great goal to have, but to achieve this is difficult. Something generally must give. And that something is testing. Services/applications are now being released with more risk than needs to be exist because the level of testing is significantly lower than what used to occur.

So what is the answer to this?  Of course, it’s automation.  Just automate the tests and it’s “Good Day Sunshine”.  Ok, that isn’t true.  But we hear that a lot.  Automation is the key to agile and continuous testing.  So it is.  But just automating your test scripts isn’t enough.  Code and UIs change. Scripts break. Scripts must but be rewritten. Maintaining scripts is difficult and time consuming.  So developers and testers resort back to manual testing and the bottleneck remains.  Well if testing is your bottleneck just automate your scripts J. Doing the same thing over and over again and expecting a different result makes you “The Fool on the hill”.

In my last “Shift Happens” article, I talked about shifting (yes pun intended) from UI automation to API Testing automation.  Automated API Testing has the advantage that APIs are more reliable and less fragile that UI tests.  They also give you the ability to test at the building blocks of applications and services rather than doing “black box” testing through a UI.

Automating at the API level is better and will alleviate work for developers; however, the developer still needs to create the API tests. They still must think about what they want to test and how to test it.  This is work most developers don’t want to do or still do not have time to do.  When they create the tests they generally create 1 positive test and 1 negative test and say “Good Night”.  That clearly isn’t thoroughly testing the API.  What can be done to test better while reducing the developer workload, or at the very least not increasing the workload?

What Is Property-based API Testing?

It’s called Property Testing. What is it? “Tell me why” it will help.  Property testing tests the properties of the code. You can think of properties as rules, invariants or requirements.  For APIs you can think of the properties as the API specification.  The specification tells you a lot about the API.  You know the number and the data types of the parameters, the “verbs” of the API, the constraints of the parameters, etc…  From this knowledge, API tests can automatically be generated.  These tests are not example based tests where there are pre-defined input values with expected results, like we do with most testing today.  Property tests test the API specification to validate the API is functioning as expected, by generating values for parameters based on the specification and constraints.  With property testing we now have a way of not just creating 1 positive and 1 negative test, we can automatically create hundreds of tests that will test positive, negative, edge and even SQL injection cases for an API.  All without the developer thinking about it.  The developer just needs to provide the API specification. Developers have these specifications today in the form of Swagger, RAML, WADL, WSDL, etc… documents.  To get the value of Property based API testing, the developer doesn’t have to do much work at all.  They can just sit make and let the auto creation and execution of automated API tests do their work.  “It won’t be long” to get hundreds, if not thousands of tests, created and executed because these are small API tests that in many cases can be run in parallel.

Now before you say “You can’t do that”, I’m not saying that example based testing isn’t important, it is.  But if you want to comprehensively test an API, relying solely on the limited example based tests that developers come up with in a hurry, will not get it done. With almost no effort involved you can start to holistically test the API.  Why wouldn’t you?

Have you ever been interested in Test Driven Development (TDD)?  Found it too daunting to do correctly?  2 week sprints made it impossible to do?  With property testing of APIs, the comprehensive set of tests can be generated before one line of code is written.  Why?  As long at the API specification exists, the tests can exist.  If the tests exist, then developers can develop against the suit of tests and not just develop against 1 or 2 tests.

But wait there’s more!  As mentioned earlier, automation is the key to continuous testing.  But that wasn’t really true.  The key to continuous testing is automating the automation. Every time a script breaks and a developer or tester must fix it, rewrite it, or rerecord it, continuous testing breaks.  But if the automated scripts can automatically be created, “Ob-La-Di, Ob-La-Da”, now you can have real continuous testing.  This also goes for monitor in production.  How often do those scripts break?  As often as the scripts in testing break.  But if you can pass the scripts generated in testing over to production, you just extended your continuous testing to really be truly continuous.

In summary the Beatles were the best band ever.  Shift happens, but we can alleviate some of the work while improving the quality of a release by utilizing property testing as part of API testing.  “The End”

Learn More About API Testing

To help you understand how API Testing fits into the Continuous Testing ecosystem, download the “Definitive Guide to Continuous Testing” so you can learn the strategies, technologies and techniques that go into a successful Continuous Testing program.

Related Posts