Respect Your Unit Tests

Our team just completed a major release of our software. It’s been three months since the last release. This is a very long time based on our usual pace of 2 to 3 weeks, so we could expect to have some issues after deployment. Had we shown some respect for our unit tests, we would have fewer issues.

One of the problems we had was related to some external libraries we were using, which required our models to use conform to the JavaBeans spec ( having both a getX() and a setX(X x) method). In some cases, the getX() was the only meaningful method, and the setX(X x) was only put there as a placeholder to conform to the API. If the setX(X x) had a nice comment explaining it’s purpose in life, we were safe. But sometimes, we had the stub setX(X x) method, without an explanation of why it was there. And upon doing some reference checks, we would quickly determine that it was not used by any code, and could be removed.

That’s just what happened in this case. A developer noticed the dead code, asked around if it was used, got concensus that it wasn’t and proceeded to remove it. He ran the unit tests, fixed a couple of broken tests, and checked in his changes.

He removed un-used code, and then had to fix some unit tests.

And when we released the code to production, we had problems.

The fact that the unit tests failed and needed fixing should have been the indication that the code was used, even if not explicitly referenced by any of our code. The fact that we had production problems confirmed it. The unit tests were in place to prevent this error. But the developer bypassed that warning.

The unfortunate fact is that it is very easy to learn from a mistake. But it’s these mistakes that we need to file away as experience, so that we get better at what we do. Take care in writing unit tests, so that they are meaningful and accurate. And then take greater care when modifying them. When in doubt, trust that the unit tests are right. Grab a pair; get a second opinion; get a third opinion. Let the unit tests serve as one of your safety nets when making changes to code. And respect your unit tests. They don’t lie.