JPragma Blog

Pragmatic Developers, Elegant Software

Don’t over spec your tests

with 3 comments

Yesterday I had a discussion with my colleagues about the properties of good tests. I think in general tests have 4 purposes in the following increasing order of importance:
  1. Validate correctness of the system under test
  2. Document usage and expectations of the tested module
  3. Help designing component’s API and interactions (when practicing TDD)
  4. Provide a safety net that enables fearless refactoring
The last point is the most important in my opinion. To provide such safety net, tests must be, as stated by Kent Beck, “sensitive to changes in system behavior, but insensitive to changes in code structure”.
How to achieve this?
Perhaps we should prefer higher-level component/module tests. Such tests are quite more stable and insensitive to structural changes. We should limit the usage of mocks in such tests, probably only mocking collaborators that live outside of the component boundaries.
We should only verify interactions with collaborators that are essential to the business logic of our component.
What do I mean by that? Often I see unit tests where developers stub responses of component collaborators and then verify ALL these interactions. With Mockito they sometimes utilize “lenient” matchers like any() or isA() while stubbing; and “strict” matches like eq() while verifying. This technique is OK, but in my opinion, it should only be applied to true mocks – calls that are essential to the behavior of the system.
Calls to simple data providers (stubs) shouldn’t be verified at all, otherwise, it delivers wrong intentions of the code author as well as makes tests quite fragile.
The difference between stubs and mocks is greatly explained in this article by Martin Fowler.
What do you think? How do you make your tests insensitive to structural changes?

Written by isaaclevin

October 19, 2019 at 11:10 am

Posted in Java, Testing

3 Responses

Subscribe to comments with RSS.

  1. Valid concern. Perhaps, the answer is to write unit tests for lower-level compoents and to have only integration tests for higher-level?


    October 20, 2019 at 6:06 pm

    • Probably. Such unit tests of lower-level components/classes should be extremely easy to write. Their goal is simply to “lead” creation of the class. We shouldn’t hesitate throwing them away if necessary and re-writing completely. Unfortunately quite often we try to “quickly fix” failing unit test greatly damaging its value and readability.


      October 20, 2019 at 7:27 pm

  2. […] properties are very important for integration/end-to-end tests. I discussed this in my previous post. When writing “stable” e2e tests, I use real components/collaborators for the entire […]

Leave a Reply to isaaclevin Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: