AcceptanceTestOnlyDevelopment

My colleague Johan Andersson and I presented this approach in our paper at XP2003. However, we called it AcceptanceTestDrivenDevelopment, only to discover that more famous people (like Joshua Kerievsky) had simultaneously adopted the term to describe something else which very definitely didn't imply not also driving development with unit testing. Joshua describes it as adding a big, more occasional "beat" to the frequent beats of test-code-refactor. We put AcceptanceTests where most people put their unit tests - in those frequent beats.

This can only be a short summary - this case is more fully argued in our paper. If you find these ideas intiguing, I'd encourage you to read that paper.

A summary of the thesis:

Orthodox XP's Unit Testing achieves five different things, as I see it. These are all coupled together in one practice. We came to XP with automated acceptance testing already in place, with the approach that over time has produced TextTest. It seemed odd to start by introducing another sort of testing, and we have gradually adapted and improved our process, to cover these five things. And in some cases to improve on them.

Here's the Big Five, others may think of more:

1. Verify correctness of the code you write
2. Drive the design (TestFirstDesign)
3. Provide detailed information to allow rapid fixing when tests fail
4. Predicting the behaviour of the code before it is written, and hence independently of the design
5. Documenting the design

A lot of responsibility for one little practice! Our "decoupled" process shoots at

1. Verification by automated AcceptanceTests (in TextTest)
2. Driving design by UsageFirstDesign
3. Having detailed failure information on hand in DiagnosticsWithTheCode
4. Selective prediction when thought useful, and frequent general predictions, by using TextTest's capabilities in this area.
5. Selective design documentation with Doxygen (Javadoc for those blessed to be clear of C++)

That makes a few more practices, but we believe they're all individually easier to do well than Unit Testing. A practice that aims at 5 benefits is somewhat susceptible to having only some of them in mind when trying to do the practice. Most Unit Testers I meet feel they struggle with at least some of the Big Five, with the last one being a particularly common cause of scepticism.

In a head to head battle I believe we win clearly on point (1), which is to me the most important. After all, we are trying to do Testing. For anyone who believes a large unit test suite constitutes readable documentation of a design, we also lose clearly on point (5). The rest can be argued, I currently believe we win point (2) narrowly, though the theory of TestFirstDesign is not developed enough to really interact with it (it's all examples and "try-it-you'll-love-it" in my view). I've had a go on the UsageFirstDesign page though. (3) is probably a draw, while (4) may be a narrow defeat, depending on your view of the usefulness of universal behaviour prediction (which probably depends on your customer's ability to know exactly what he wants before it exists).

Do we believe orthodox XP is pants? Hardly. We have adopted most of the rest of it as suggested and we swear by PairProgramming and the PlanningGame. For most practitioners, an easy way of automating testing was a revelation, and is clearly an order of magnitude better than testing at the end or not at all.

But I do believe that even the best things are there to be improved. And I believe we have, at least in our domain. I'd welcome criticism of the type "this will only work in your domain because...". But don't believe it won't work at all, because it does, at least in our case.

Want to try the theory for yourself and see it in action? Read the proposals on XpExperienceExchange.

-- GeoffBache