Chris Holmes Online

Brad Wilson and James Newkirk posted a few thoughts today, and a question, concerning code coverage. Brad specifically asks:

Are you on an agile team? Do you do code coverage? What does it mean to your team?

We use code coverage, but not at a regular interval like I’d like us to. Part of the problem is that we still can’t get CruiseControl.Net to work. After a couple of weeks fussing around we finally managed to get it to work with Vault, but now that it correctly gets the latest source code from Vault we can’t get it to fire off the build. Go figure.

That said, we do try and make ourselves aware of our code coverage.

I like James’s analogy to a barometer. He says:

The answer lies in a metric that can be used to predict the weather, barometric pressure. Today, weather.com says the barometric pressure in Seattle is 29.66”Hg and falling. More important than the absolute measurement is the trend: it tells you that the overcast and drizzling rain outside is going to get worse, not better. The two pieces of information – value and trend – are used to predict what will happen next. The same can be said for code coverage. You can determine the relative health of your tests by using the value and the trend to determine the appropriate action. That’s the important bit: code coverage gives you relative measurements against itself, not an absolute measurement against a target value.

I agree. Somewhat.

While people may look at code coverage as a metric, I look at it more as a tool. I like to use code coverage as a way to help me ensure that I’m writing proper unit tests. When I look at the analysis for a class under test and find a method uncovered, or an else statement uncovered, that prods me to think of a unit test to exercise that code. And that prodding – that process of thinking about how to test something – is the real value of code coverage (at least to me). Because nine times out of ten, that process causes me to rethink that particular line of code, or that method. It makes me question. Do I really need that else statement? Is there a refactoring I can do to get rid of it? Is there a better way to write that method? Is this a method that doesn’t belong in this class to begin with?

I like tools, techniques and coworkers that make me question my code writing abilities. I like it when a tool makes me question my design. Code coverage is a great way to initiate that process, and to me that’s its greatest benefit.

When it comes to actual coverage numbers, I prefer to shoot for 100%. But I have to qualify that: 100% for classes that matter. James brings up the same point when he writes:

It is inevitable that code is written which isn’t covered by a unit test. A few examples of acceptable code without tests might include: web service wrappers generated by Visual Studio, views in a Model-View-Presenter system, and code for which failure is only possible because the underlying platform fails (like helper methods that pass default values into more complex .NET CLR methods).

I don’t really care about a number like 75% code coverage for the entire solution file. That doesn’t tell me anything useful. I care about 100% for a class that matters. I expect presenters and use case classes (workitems, etc.) – classes that do business logic work – to be tested thoroughly and have 100% coverage. If they don’t, then there has to be an explanation, and usually it’s because someone forgot to write a test, or because the design is bad and needs refactoring.

So to answer Brad’s question: What does it mean to our team? It is another “entity” watching our backs, ensuring that we’re doing all we can to write the best code we can. It’s like calling pair programming “real time code review”. Code coverage for us is a way to guide us, to let us know that we’re really thoroughly testing the important classes in our architecture, and we feel secure in knowing we’ve covered all our bases.

It’s another pair of eyes, in a way. And I like that.