Test Automation
A great place to start when you are thinking about test automation whether you aren’t doing any automated testing right now or whether you are but just not super comfortable with it and not sure if you are doing it the right way is to kind of go back to the beginning.
The very first thing that you need to understand is the automation test pyramid. If you are not familiar with it, it’s a pyramid developed by Mike Cohn a while back, its essential point is that you should have many more low-level unit tests than high-level end-to-end tests running through a GUI. The automation test pyramid is a graphical representation to help you develop a better-automated testing strategy.
Why should I care about which tests I automate? #
You should know that as we move up in the pyramid three things start to increase:
- Cost to develop and maintain the tests
- Execution time
- The possibility of false negatives
Your goal while adding tests to your process is to make sure that those three things are well balanced, you don’t want to waste time automating tests so that at the end you don’t use them because they take too long, or even worst having to spend more time maintaining the tests than writing application code.
However coverage does also increase as you go up the pyramid, a unit test is going to cover the smallest part of your codebase so you will need many unit tests to cover all your code but an end-to-end test is probably going to cover much more than just one method. Plus those tests at the top are more business-relevant, so you will need to have some of those as well in your feedback loop.
I would consider that an application feature is properly tested when it has both unit tests and acceptance tests. I generally start by making sure I have good unit test coverage. The unit tests will catch edge cases and confirm correct object behavior. And gradually introduce acceptance tests that exercise the application like an end-user. But always keep an eye on those acceptance tests, as you add more, your feedback loop is going to get longer, so you would always want to keep their execution time under a certain limit.
Why do we test #
Generally, there are two main reasons why people end up testing things:
1.Regression prevention
This is where you write a test to prevent a bug from recurring later on so you know that when something breaks.
2.Behavior Driven Design
You write tests so that you can see how the code want’s to be used so that you know what code to write, which is kind of a weird way of looking at it, but this falls under the camp of BDD (Behavior-driven design).
Either way, you will at some point find yourself asking: What should I be testing?
I love what Uncle Bob Martin gave as an answer to this 😅:
“You only test code that you want to work”
In practice and since I can’t just go ahead and start testing every single bit of code, I generally go through a simple flowchart in my head :
1. Is the code private?
Is it part of my public API, because if I have a private method in my code then you can’t under normal circumstances call it. I don’t write a test for those kinds of methods, because for private methods it’s either never going to be used (Dead Code), or it’s going to be used by one of my public-facing methods that is part of the public API, and I will run tests for the public API.
2. Is the code part of a library What I mean by this is to check whether or not I am testing some framework or external library code that is probably already tested. You can still do it, but for me, that’s not testing that’s overtesting. Because what you should do if ever you find a bug in the library is to report that to the maintainers and even better; propose a fix if the project is open-sourced.
The main idea of this post is that test automation helps improve quality with speed. But with so many types of tests, you must get the right balance. The test pyramid gives you a framework to help get this right: Most tests should be unit tests, then as you go up you should have fewer tests.