Friday, December 30, 2005

The Deforestation of Requirements

Requirements documents are a unnecessary waste of time and trees.

Let's look at the typical process of developing a requirements document. This differs a bit between custom development and product development efforts, but the end result of each is typically a formal requirements document. (For my reader's digest version of the typicaly requirements document process, see the bottom of this post).

The classic requirements document consists of words. Lots and lots of words. They may be organized functionally, by feature, by category (system, business, market...). But it's all words.

But words don't compile into working software. And here is our first huge impedance mismatch. Words can have multiple meanings. Put them into sentences, and the multiple meanings multiply. Put them into paragraphs, and usually instead of helping to clarify, it just adds more room for interpretation.

And because software developers need to somehow translate these words into bytes, there is a lot of further discussion. Many of these discussions are identical to the discussions that went into making the Requirements document in the first place. "What does it mean when it says the system shall present the list of search results. What data from the results to you want displayed? Do you want it in a tabular list, or some other presentation? And how about 'must respond in a timely manner'? Is that 1 second? Or 10 seconds? Is that everytime? Or just most of the time? How much is most of the time?"

Most of the Agile methodologies replace the requirements document with something a little more useful. Called use cases, story cards, UI wireframes, they at least try to look at things from an end user perspective. (Which is great for business software, but a bit harder for embedded or infrastructure software).

But the biggest stride forward in these agile methodologies wasn't the introduction of yet another way to capture words. It was the introduction of the concept of iteration between the developers and the users. It was the concept that it is almost impossible to get requirements right the first time, and that you'll need a few passes to refine some ideas, to recognized dead ends, and to find areas no one thought to include the first time (or two) around.

It was the recognition that no one is smart enough to see it all and get it right just by thinking about hit and writing words in a document. It bears repeating - No One Is That Smart. Not users, business analysts, product managers, or software developers. No one. So don't start a process that assumes this is even possible.

(Here is where the advocates for requirements documents shout "it's an evergreen document - it evolves with the project!" Oh great - you recognize that iteration and refinement will be necessary, yet you introduce a piece of paper that must be continuously updated and reconciled with the software, and that at the end of the project is tossed out. I realize that the additional overhead implies a lot of extra dollars for the consultant, or a much bigger budget for the product development team, but let's pretend that's not our true goal here.)

Should a team make some sort of stab at figuring out what they want to build (and just as importantly, why) before starting on these iterations? Absolutely. It's a great idea to have an idea of what you want to build before spending more money to build it.

My point is that the initial stab should be just that - a stab, an outline, a set of notecards. Having the associated business reasons for each idea, and force ranking these ideas have lots of value too. But to spend a lot of time trying to refine these and getting "buy-in" across an organization - this has been a waste of time in pretty much every project I've been involved with, because we just don't know enough yet - not until we start to implement do we have the knowledge to refine.

And with today's IDEs providing ease of rapid iteration, this is the quicker, better path to good working sofware.

Side Notes:

(For arguments on the other side, you can read a couple of good blogs from friends of mine here and here that make the case for formal requirements docs. My opinion is that their arguments are logical, but not practical in the real world of real corporations, real developers, and real training. I've tried both, and know which I've found has a better outcome.)

Readers Digest version of the Requirements doc process.

For custom software (a "one off" project for a single paying customer):
The customer has someone in charge of the project from their end. Let's call this person the Customer Project Lead (CPL). The CPL is on the hook to help the project team identify and reach all the "stakeholders" in the project (business people, line people, managers, IT, and possibly other departments). A series of meetings is held where the goals of the project are discussed (or at least, the reasons why the customer is paying this vendor and what they hope to get out of it). The stakeholders start tossing off things the system has to do (usually, these are very much in the weeds, and are things about the current system/process that they either love and want to keep, or hate and want to change).

For product (software product designed to be sold and used by many customers):
Product managers take the role of a proxy CPL. They are supposed to talk to real, paying customers to find out what they want, like, and don't like. They are supposed to be familiar with competitor products and what they do and how they do it. And they are supposed to distill all this down into a requirements document. (Some of the better ones iterate a bit with engineering - "can you make it do this? How about this?" - but in the end, all the knowledge is supposed to end up in a requirements document.


At Wednesday, January 04, 2006, Blogger Jeffrey Davidson said...

I'm on the other side of the fence from you on this one, but I look forward to continuing the discussion. If only because I'm not convinced Agile can work on complex project. And the alternatives after that scare me.

And please join the discussion on the message board too. It will be good to hear from "the other side of the fence."

At Wednesday, January 04, 2006, Blogger A Muser said...

I hear what you're saying - I used to build satellites for a living, and iterative development isn't realy an option for a platform 22,000 miles away...

But for platforms within reach, I think it's the better way to go. Large, complex projects must be broken into smaller, manageable subprojects anyway, so why not treat each subproject using Agile approaches? (with one more "architecture" project added where the subsystem interfaces and common object model is managed).

At Saturday, February 03, 2007, Blogger Michael Craig said...

Many business writers/business analysts try to incorporate heaps of incidental details into their requirements documents because they are too scared of getting into trouble with stakeholders for missing things or are poorly trained.

When they do this they end up in a vicious cycle and one you give an inch to the business, they take a mile. IT needs to take a more-pro active stance and control their own documents and contents of the documents.

Here is a good post.


Post a Comment

Links to this post:

Create a Link

<< Home