XPinSymantecCaseStudy

From Software Development magazine, January 2002 issue:

"Adopting Extreme Programming practices, Symantec developers, testers, technical writers and managers took a calculated leap into the agile unknown ... and even the executives are impressed"

http://www.sdmagazine.com/documents/s=2279/sdm0201a/0201a.htm

We've been reading this article with interest. This section particularly struck us:

Cortez admits that it's also been tricky learning how to measure and report an XP project, especially in accordance with the existing Solution Centered Process (SCP). "The Symantec process folks are trying to understand estimation, but the problem's not with estimation. They're treating software development as a deterministic set of events, when in fact it's a stochastic process."

To say that development is stochastic is more than a little peculiar--we concluded that what Cortex means is that development is "difficult to predict exactly given the starting conditions", which is true. If Symantec's process people are typical of the breed, what they are likely looking for is an effective procedure for calculating values of the function estimate:requirement->dollars, to be applied once at the beginning of the project. And they can't find one, of course.

This indeed is the problem we've been having in bidding for the next piece of work from our major client. They want a prediction of how much it will cost for us to deliver a certain feature set. We don't really have any way to that information, since we deal in estimates, not predicitons. So what we did was to estimate the new work in pseudo-stories (that we wrote ourselves based on their spec), and reverse engineer our historical project velocity to turn those effort estimates into estimated devloper-weeks, and so pounds. This has the benefit of giving us a starting point for negotiations, but I worry that somehow the estimation-ness of it will get lost in the translation, and the numbers will be deemed a prediction.

What's interesting is that, all through the current project, the prime contractor's project office has been less than thrilled with the awkward, for them, way we track and report and commit to things, unlike the other sub-contractors who are reporting like tradtional development organisations do. However, another way that the other sub's are like traditional development organisations is that they still aren't delivering code that works (or, in extreme cases, builds) even though the project as a whole is well past the notional end of "development" and into "QA". Meanwhile, we've not delivered all the functionality that the customer would have liked, but what we have deliverd builds without warnings and works, modulo the small trickle of non-critical defects we're finding in system testing. There's a clear inference to draw here (and hypothesis to test, naturally), but our project office haven't done so, it seems. It will be interesting to hear what conclusions Symantec's process folk draw form their experience. --KeithB

---

This discontinuity between 'providing estimates' and 'meeting contractual obligations' is definitely a problem when it comes to doing work outside of your own organisation. And once you start playing the contract game - do what the contract says not what the customer wants - you have lost that win-win perspective that makes XP so powerful -- TimBacon (As an aside, there is an ACM published paper at http://www.idiom.com/~zilla/Work/kcsest.pdf that apparently proves mathematically that accurate estimation of software schedules is impossible, even to within an arbitrary level of accuracy.)