Sunday 13 May 2007

Project management

(Part of an occasional series on matters technological. It beats doing real work.)

We've been having some internal wrangling here at Matchpeg Towers about project management/software development methodologies (not much wrangling - we're all fugitives from organisations which like signatures in triplicate on everything).

Matchpeg is essentially in the retail software business. We sell tools at low cost, in large quantities, to happy customers who we rarely have any un-automated contact with. We do very little bespoke work. We don't really have "clients".

There's a polarised debate all over the web about the relative merits of waterfall (relatively linear, relatively traditional) software development processes versus more agile methodologies.

Waterfall doesn't work for us. It's not that we're a bunch of laid-back hippies - agile doesn't work for us either.

The problem with both is most concisely represented by the Agile Manifesto. Notice anything about this? (Leaving aside the fact that it reads like a Miss World contestant's pledge to seek out world peace and a cure for cancer.)

The references to customers are both in the singular, not to "customers" plural. We're in the retail software business. We have lots of customers. And they're so diverse that you can't boil them down into a single representative entity called "the customer" (or even "The Customer").

(The same failings can be found in things like the official PRINCE2 material: same assumption of a single customer; same assumption of a single entity paying for the project; same assumption that there's a contract.)

In short, our experience is that no well-known software development process translates at all well to the retail world. They all seem to stem from a founder's experience of in-house development for a large company, or bespoke development for a single (usually large) client. Using a popular term in IT, these methodologies don't "scale" well. They make all sorts of assumptions which stop working when you're dealing with a number of diverse customers, each of whom isn't guaranteeing to pay you anything.

These methodologies are created by consultants in order to perpetrate consultancy. They're for use with "clients", not "customers".


On a related note, we're bemused by the way the debate between waterfall and agile is usually presented: an either/or choice, generally accompanied by comments such as "waterfall is so lame".

Instead, it seems to be about risk/reward profiles. A question such as "Which is better, waterfall or agile?" looks as meaningless as "Which is better, stocks or bonds?" The answer depends on your circumstances and what you're trying to achieve.

As a very general rule - to which there are, yes, countless exceptions - the anecdotal reports on the web tend to indicate that agile methodologies offer the prospect of better returns (faster development) at the risk of greater loss (complete anarchic breakdown of a project into marauding bands of developers).

It's not a coincidence that waterfall methods tend to be preferred by larger, older firms, and agile tends to be preferred by smaller, newer firms.

If you're a small company just starting up, you have to take risks. You gravitate towards agile because you're not going to get anywhere by being dull-but-steady. Older companies - particularly public companies - don't and can't have the same appetite for risk. Everything militates against it: the shareholders want predictable returns, the employees want to keep their well-paid jobs. The potential additional reward doesn't justify the additional risk; the reward is too incremental, and the risk is too absolute.

The really nasty fights start when the incentives and risk-tolerance of the business owners are different to the incentives and risk-tolerance of the people running projects for them. This works both ways: an impetuous CEO who finds that he's hired a pen-pushing dullard of a project manager, or a cautious board which has landed itself with a bunch of mavericks who won't be significantly affected if they cause their project (or even the entire firm) to implode.