Martin Fowler: Specification By Example | Javalobby

Martin Fowler: Specification By Example | Javalobby


... I do think that having a comprehensive automated test suite is more valuable than the term 'side-effect' implies. Rather like if someone offered me a million dollars to hike up a mountain. I may say that the main purpose of the hike is the enjoyment of nature, but the side-effect to my wallet is hardly trivial...

• Specifications are supposed to be general, to cover all cases. 
• Examples only highlight a few points, you have to infer the generalizations yourself. 
• This does mean that Specification By Example can't be the only requirements technique you use, but it doesn't mean that it can't take a leading role.

I like this idea, actually. It's just my style of thinking requirements, and the way of designing computer software. It doesn't mean we should not abstract, or generalize the requirements - That's another thing I like. I like Specification by Examples, because no matter how hard we are going to work on the specifications at the first place, we could not get a completely correct one. What we could do would be:

• Some incomplete requirements, which mean that we have something leave behind.
• Some incorrect requirements, which mean that something we want to be implemented could be something we don't really need, or something we should not implement, for it is too complex, or whatever reasons.

So, we should be more pragmatic. Now, let see what Fowler said.

So far the dominant idea with rigorous specifications, that is those that can be clearly judged to be passed or failed, is to use pre and post conditions. These techniques dominate in formal methods, and also underpin Design by Contract. These techniques have their place, but they aren't ideal. The pre-post conditions can be very easy to write in some situations, but others can be very tricky. I've tried to use them in a number enterprise application settings, and I've found that in many situations it's as hard to write the pre and post conditions as it is to write the solution. One of the great things about specification by example is that examples are usually much easier to come up with, particularly for the non-nerds who we write the software for. (Timothy Budd pointed out that while you can show a lot of stack behavior with pre and post conditions, it's very tricky to write pre and post conditions that show the LIFO property.)

I think Fowler and I have some common ideas:

• It is as hard to write the pre and post conditions as it is to write the solution.
• Writing examples to specify the requirements is much easier.

Why? That's what software development is supposed to work:

• Every piece of computer software under development is supposed to be something new, or most of it is new. Otherwise, we just need to copy the old one and it will just work. It is because of the replication characteristics of computer software.
• Even worse, there would be some unknown techniques involved.
• When people try to come up with some pre/post conditions in the specifications, they need to think about what the software would work carefully, to make sure they are feasible, or even, could be finished within a certain development period.
• However, since every piece of software are new, or mostly new, or involve some new techniques, people could not simply guess what would be the pre/post conditions without truly development experience - however, it's new, nobody work it out before. So people need to think over every detail of development to work the specification out. Or, they just turn out to be something wrong. Maybe not completely wrong, but at least, there would be a lot of risks to have those specifications to work on.

Ok, let's see what Fowler said.

An important property of TDD tests is that they involve a double-check. In fact this highlights an amusing little lie of the XP community. They make a very strong point of saying things Once and Only Once, but in fact they always say things twice: once in the code and once in the tests. Kent has pointed out that this double-check principle is a vital principle, and it's certainly one that humans use all the time. The value of the double check is very much tied into using different methods for each side of the double check. Combining Specification By Example with production code means that you have both things said quite differently, which increases their ability to find errors.

It's interesting. Everything has its pros and cons. We advocate single source to avoid inconsistency. However, applying double source could increase the ability to find errors, which is double-check here.

Another thing about this double checking will add some extra code maintenance work. We always need to keep in mind this.

The formal specification community have constantly had trouble verifying that a design satisfies a specification, particularly in doing this without error prone humans. For Specification By Example, this is easy. Another case of Specification By Example being less valuable in theory but more valuable in practice.

One Design by Contract fan pointed out that if you write a specification in terms of tests, then the supplier could satisfy the specification by just returning hard-coded responses to the specific test inputs. My flippant answer to this is that if you are concerned about this then the issue of tests versus Design by Contract is the least of your worries. But there is a serious point here. Tests are always going to be incomplete, so they always have to be backed up with other mechanisms. Being the twisted mind that I am, I actually see this as a plus. Since it's clear that Specification By Example isn't enough, it's clear that you need to do more to ensure that everything is properly communicated. One of the most dangerous things about a traditional requirements specification is when people think that once they've written it they are done communicating.

I like this statement very much!!


Specification By Example only works in the context of a working relationship where both sides are collaborating and not fighting. The examples trigger abstractions in the design team while keeping the abstractions grounded. You do need more - things like regular conversation, techniques like Domain Driven Design, indeed even doses of Design by Contract. While I've never had the opportunity to use Design by Contract fully (i.e. with Eiffel) I regularly think about interfaces in contractual terms. Specification By Example is a powerful tool, perhaps my most used tool, but never my only tool.


(For more thinking on Specification By Example, if not by that name, take a look at Brian Marick's writings. Somewhere on his site there probably is one super page that sums up his view on it. Of course finding it is less valuable than reading all the stuff there while you're trying)