Overly Mocked

by Charles Miller on December 16, 2003

As far as Unit Testing goes, mock objects are a pretty useful addition to the toolkit. In a componentized system, by mocking out all the other components, you can isolate the one you're testing that much more effectively. Mock object frameworks exist to make the job a little easier: providing simple ways to provide substitute objects, and record and predict method calls.

Unfortunately, once you get in the mock object habit, it becomes too tempting to test things that you really shouldn't be testing. Because you can test to a granularity of which methods get called on which components, because it's easy, you find that you do it as a matter of course.

Often, you shouldn't be. Let's get down to what a unit test really should be. A unit test should predict some observeable effect, and test that it can make that effect occur. How the object makes the test pass is irrelevant, so long as the test passes. Mock object testing looks a little too closely at what lines of code are actually going to be written to make the test pass. Testing lines of code directly, rather than the effects of those lines of code, can lead to irrelevant and hard-to-maintain tests.

If you start writing mocks to predict (and test for) every call to the objects that are being mocked out, you end up with fragile tests that break trivially as the result of refactorings that change how the object being tested does something, without changing what the object does.

But that's not all.

Most of the systems I've seen over the last few years have been written in layers. Down the bottom, you have components that do the important stuff, but don't talk to other components much. The layer above coordinates between the lower-level components, but doesn't do much interesting itself apart from that delegation.

So what happens if you test those higher-level components with mock objects? Well, you sort of end up with tests that do absolutely nothing but test a series of method calls on mock objects. This is, I would suggest, worse than useless. What are you testing, really? You're not putting forward any kind of verifiable hypothesis about what the method you're testing should be producing: you're just writing it twice, once in the test and once in the object being tested.

If you get the series of calls wrong in the test, you'll get it wrong in the object too. You have exactly the same margin for error with and without the test, you've just moved the risk around a bit.

It pays with mocks, therefore, to always keep in mind the question: "what exactly am I testing?"

Previously: Sick

Next: Lots of Little Languages