Lean testing or dying in beauty? Concentrate on the essential or cover all? Is lean testing just the dernier cri, for those, including management, who think testing is too costly?
There is an evolution in testing as well. Not from ape to human, but from “Happy without testing” via “test it baby” and “no risk no test” to “no future” for testing… At least according to Nils Langner from LeanKoala this is the way it often evolves. After a phase of no testing, you hire/train/develop testers, who test everything, full coverage in contents and phases and tools. Then you realize this is too much and concentrate your effort on those areas with highest risks or highest use level. Last but not least there will be a phase where even that is thrown away and instead “lean testing” is implemented.
Our June StugHH meetup had Nils over as special guest, together with Torsten Franz from Finanzcheck they gave an insight on the idea of lean testing, and illustrated it with the LeanKoala approach. So what is “lean testing” all about? The main idea is, that at least for websites, you would be able to tell from a few symptoms, if your website is still well behaving correctly after a new deployment. For example, you will get HTTPStatus !=200 for many reasons, for example for syntax errors, controller errors, and other integration issues. So if you have a tool quickly checking if all relevant websites do not throw http errors, then you are already pretty safe. This idea, together with a very extended monitoring of errors thrown by the application in the log, will give most major issues. These two then would be extended by some crown testing for the visuals and functionality.
Sounds easy? Is easy. Especially when and if you use the “LeanKoala as a service” as provided by Nils on www.leankoala.com, where he combined quite a few tools that can be used to identify the majority of the most relevant symptoms of failure on web sites. Try it. As Nils says “Risks usually are smaller as you would expect” and “good enough coverage often is good enough”.
Feeling uneasy now? Is this too lean – even anorexic?
I did feel very uneasy. Where did the test of functionality go? What about dependencies between several pages, like in wizards? What about unit tests? All the other testing stuff I love? Am I just not evolved far enough, or is the lean testing approach too risky?
To be fair, Nils does not say, that this would be suitable for all situations. His Lean Koala Service had been developed for publishing companies to begin with. Also, Torsten added that he would be using LeanKoala not instead of functional testing but (also) to get rid of all those annoying automated Selenium tests, that exactly do those kinds of checks. So quite some increase in efficiency just by using the tools for situations they are meant for.
So basically, there is no need to be scared of this special testing diet, as with all diets you just have to do it properly.
In the end, the whole Idea reminded me a bit of tools like “Checkstyle”, just way more advanced, where with some configuration, you can identify many things really quickly that otherwise would not have been discovered or only in a very time-consuming matter. However, for the execution of dynamic tests you would still use different tools.
The overall concept seems to me just a combination of a risk based approach moved to the extreme, together with a “shift-right” approach. This seems to me very suitable for websites, where the risk is not based on the dynamic interaction of different functionalities. For those websites, still other risk based techniques shall be employed. For some risk categories, it seems a good approach to base selection of tests to be executed on typical symptoms for errors. But for some risk categories, the correct positive result will be more relevant. The right mix of techniques is the best – as always. But for websites I can well imagine to employ the Lean Koala approach, at least to do the housekeeping – and leave time for the “beautiful curves of non-anorexic” tests.