The mess that mocks can make
Aside: I guess this post is really about mock frameworks rather than mocks, but I didn't want to break the partial alliteration I had going in the title :)
A question was posed recently on one of our discussion lists about whether Rhino Mocks was a good framework to use for unit tests. As a TDD EDD advocate, conversations like that fire off warning bells as for the most part, I don't believe that they are suited to writing unit tests in the context of test-first development. I'd go so far as to say that they can even be detrimental.
As an example, here are a couple tests from the Humble Dialog Box post which is part of Jeremy Miller's series on 'Building your own CAB'.
[Test]
public void CloseTheScreenWhenTheScreenIsNotDirty()
{
MockRepository mocks = new MockRepository();
IHumbleView view = mocks.CreateMock();
Expect.Call(View.IsDirty()).Return(false);
view.Close();
mocks.ReplayAll();
OverseerPresenter presenter = new OverseerPresenter(view);
presenter.Close();
mocks.VerifyAll();
}
[Test]
public void CloseTheScreenWhenTheScreenIsDirtyAndTheUserDecidesTo
DiscardTheChanges()
{
MockRepository mocks = new MockRepository();
IHumbleView view = mocks.CreateMock();
Expect.Call(view.IsDirty()).Return(true);
Expect.Call(view.AskUserToDiscardChanges()).Return(true);
view.Close();
mocks.ReplayAll();
OverseerPresenter presenter = new OverseerPresenter(view);
presenter.Close();
mocks.VerifyAll();
}
A few questions spring to mind:
What story does each test tell? The method names are very descriptive, but the code doesn't provide clear support for the behaviour.
Where have the three As disappeared to? What happened to the expected trilogy of sections: Arrange, Act, and Assert?
and perhaps most importantly,
What happens when I want to refactor?
If you look carefully at how the mock object is created, you're really just setting up (a) expected behaviours and possibly (b) an expected sequence of calls. When code is refactored, unless you're accessing a public API, chances are that you will need to change both (a) and (b). That means going back through each unit test and making the changes by hand. Not very agile.
Furthermore, if you're using NMock and you want to change the interface (IHumbleView in the example), you'll also need to go back and change all the method names by hand since that framework uses strings. Also not very agile. In both cases, this leads people to abstain from refactoring, which is one of the major benefits of practicing test-first development.
There are two other points as well:
- Giving a brand-new test-first developer a mock framework is probably hazardous to their health. It can easily lead to poor habits such as writing tests that are highly coupled to the implementation of the classes being tested. See (b) above.
- Except for very small fixtures, there is actually less code involved in writing a real mock object. Why would anyone want to make extra work for themselves?
Now compare the above tests to these ones that were written without a mock framework.
[Test]
public void ViewClosesWhenTheScreenIsNotDirty()
{
MockHumbleView view = new MockHumbleView();
Presenter presenter = new Presenter(view);
presenter.Close();
Assert.IsTrue(view.CloseWasCalled);
}
[Test]
public void ViewClosesWhenScreenIsDirtyAndUserDiscardsChanges()
{
MockHumbleView view = new MockHumbleView();
view.DiscardChanges = true;
Presenter presenter = new Presenter(view);
presenter.Close();
Assert.IsTrue(view.CloseWasCalled);
}
and the matching code ...
public interface IHumbleView
{
bool IsDirty { get; }
bool DiscardChanges { get; }
void Close();
}
public class MockHumbleView : IHumbleView
{
public bool IsDirty
{
get { return isDirty; }
set { isDirty = value; }
}
public bool DiscardChanges
{
get { return dischardChanges; }
set { dischardChanges = value; }
}
public bool CloseWasCalled
{
get { return closeWasCalled; }
}
public void Close()
{
closeWasCalled = true;
}
private bool isDirty;
private bool dischardChanges;
private bool closeWasCalled;
}
To be honest I'd take it one step further and actually test everything directly through the view but that's not what this post is about.
I think everyone can agree that the second group of tests is much easier to read and definitely convey what they're trying to prove. If I was introducing someone to the concept of test-first development, I'd want them to start writing tests like that for a while before worrying about mock object frameworks.
Mock frameworks are probably useful in some situations, but think carefully if you really require them in a before diving in.
Update: Steve Otteson pointed out (rather correctly) that I'm really talking about 'stubs' when I refer to 'real mock objects'. The difference between the two is explained here.
Comments
- Anonymous
July 13, 2007
I'm actually new to agile development, and I was using Rhino Mock to test a controller by mocking a passive view when I realized I was not gaining anything. It took about five minutes to write a mock that wouldn't break when the controller was refactored.It seems that mock frameworks are something that I will know when I need, but until then they should be avoided. - Anonymous
July 13, 2007
Yes, it's too bad these frameworks don't come with a warning sticker on them :) I wouldn't say it's necessarily something you'll ever need either. You may, and it's good to know they exist, but so far I've been quite happy without them. - Anonymous
August 12, 2007
The comment has been removed - Anonymous
August 18, 2007
The comment has been removed - Anonymous
August 19, 2007
Looking forward to reading about your discoveries Brian :) I did read through your Deep Dive series, as well as the 2005 refactoring of the Video Store example.I hate to be an echo chamber, but I did learn a few things from those posts and so I'll be linking to them soon :) - Anonymous
August 19, 2007
Nikola: Sorry for taking so long to reply.a) I'm not claiming that mocks don't test the contract or behaviour of a method. My point is that you often write several tests for a single method, and if you are using a mock framework, it's not easy to refactor that method if/when it changes. Your tests will still fail if you create a real class for the mock.b) I don't think I've claimed that order of the tests is important anywhere .. maybe you could explain this point a bit more?c) I'll have to disagree that having a 'big pile of type stubs' at the end of a project is a bad thing. They should be located in your test assembly and not have any effect on the rest of the system. I'm also not sure why you would want to completely remove them. - Anonymous
November 08, 2007
I was commenting your next sentences:========If you look carefully at how the mock object is created, you're really just setting up (a) expected behaviours and possibly (b) an expected sequence of calls. When code is refactored, unless you're accessing a public API, chances are that you will need to change both (a) and (b). That means going back through each unit test and making the changes by hand. Not very agile===============re: a)The difference you are describing exists only in NMock where method names are strings. My point was that with the Rhino Mocks we have strongly typed mock members accessors so there is no difference at all (in that sense) in refactoring benefits of using stubs vs mocks re: b)I just said that if you don't explicitly contract the "expected sequence of calls" (that stands for ordered set of action, right?) Rhino mocks won't break a test.By default, Rhino mock don't care about the order of expectations.re: c)I don't see any advantage in using stubs in tests (two points above are some of examples), so beeing forced to create a bunch of "paralel" types somewhere just to run tests sounds as an extra work to me. Another thing, in case of stubs when "original" type is been updated, the "stubbed" one has to be updated manualy too. In case of mocks (due to their dynamic nature) no extra "parallel" work needed - Anonymous
November 08, 2007
Just a side note (delete it if you like): I don't know why I've now different nick on the site but malovicn and Nikola Malovic are both me :)