The basic unit of work in eXtreme Programming is the User Story. User stories describe some functionality that is to be added to the system under development. From the story, it should be possible to create a set of automated acceptance tests that, by succeeding, show that the story has been correctly implemented. Ideally (but sometimes not in practice) User Stories give the developers complete freedom to decide what the system should do internally in order to satisfy the user's needs.
The idea is, in a well-factored (OO?) system, you can treat bits of functionality independently, estimate them independently, and shuffle them around in the development schedule at the whim of the customer. Most of the time, in my experience, this works pretty well in practice as well as in theory. Often you discover there are far fewer inter-story dependencies than you originally thought.
Time after time on projects, though, a set of requirements end up written on cards that are the round peg to the User Story's square hole. They can't be treated as stories, they are something else. These requirements are: security, usability, performance, consistent architecture, and logging (especially audit logs). These are generally shoved in the catch-all category of Non-Functional Requirements. They get written on a card, pinned up onto the board with the rest of the cards, and worried about for the entire project because we're just not sure what to do with them.
One problem is that they violate the idea that each User Story is an independent piece of work that can be estimated in isolation. Each of these requirements places a burden on every story that gets developed, and often can only be estimated with all the other stories in mind. The classic XP response to this is that these are obviously not general requirements, but should be cut up and subsumed into each story that gets developed. I'll get back to this approach later, it's an important issue.
The other big problem is that worst way to develop some of these requirements is the piecemeal, story-by-story approach. The worst kind of UI is the one that accretes over time, with functions added here and there as they get developed. The worst kind of security is the one that treats each of the components of an application in isolation, and doesn't pay enough attention to how they fit together as a whole.
Security is particularly tough, because it goes against the grain of engineering practices. As Bruce Schneier wrote to comp.risks in November 1999:
Security engineering is different from any other type of engineering. Most products, such as word processors or cellular phones, are useful for what they do. Security products, or security features within products, are useful precisely because of what they don't allow to be done. Most engineering involves making things work. Think of the original definition of a hacker: someone who figured things out and made something cool happen. Security engineering involves making things not happen. It involves figuring out how things fail, and then preventing those failures.
Breaking these requirements up across stories also means that there's no single person with whom the buck stops. When you have a requirement, but no individual responsible for seeing it happen, you'll either end up with a piecemeal, incomplete solution, or one heroic programmer will unofficially take it upon themselves to make sure it happens (I'm not theorising here, but speaking from experience on a number of projects with varying degrees of XP-ness and agility). Trusting in the appearance of a hero is depressingly common in IT projects, but it's always bad planning, and it's one of the things that XP is supposed to protect us from.
I was looking into Aspect-Oriented Programming a few weeks ago, when it grabbed me. These aren't non-functional requirements at all. They're the requirements equivalent of cross-cuts. I internally labeled them in my head as cross-cutting concerns, and the term fit so well that my brain's been using it ever since.
Aspect Oriented programming is the natural extension to Object Oriented programming because it addresses something that was not handled very well by objects: concerns that are not easy to modularise because they cut across multiple modules or class heirarchies. Cross-cuts.
In OO, the solution to cross-cutting inevitably involves duplicating code across objects, refactoring mercilessly to minimise the extent of that duplication. The responsibility for each cross-cut gets placed in every object it effects. In XP, the solution to cross-cutting concerns inevitably involves duplicating these concerns across stories, refactoring so that stories can share code and techniques to minimise the duplicated effort. The responsibility for each cross-cut gets placed in every story it effects.
Just like AOP improved OO by recognising the existance of these cross-cuts and creating a new facility to deal with them explicitly and efficiently, XP could be improved by recognising the existance of cross-cutting concerns, and devoting explicit resources to them across User Stories. This means giving somebody responsibility for overseeing the system-wide architecture, someone to do the overall UI design, someone responsibile for overall security, even someone responsible for audit trails and logging. Who those someones are could change from iteration to iteration, to keep viewpoints fresh and open up the exchange of ideas, but they should be assigned.
This is destined to be an unpopular notion, because assigning someone this role explicitly will increase the load factor, and reduce the team's throughput of user stories. If someone spends the iteration thinking of, and testing the application's security, that's time they're not spending adding noticeable features to the program. That said, it will increase the quality of the finished product, and quality is that one lever in the XP planning game that must always stay at 100%, right?
(note: This story will probably be edited a few times over the next day or so. It kinda escaped before it was ready. Since the changes may be extensive, I'm not going to mark the edits as I normally would.)
