User stories seem to be a constant source of confusion

One team has user stories that cannot be deployed to production unless a number of other user stories are completed as well.

Another team has user stories that can each go to production independently, but each user story has so much information in it that it takes 30 mins to read, and longer to understand.

Still another team always writes 3 user stories each iteration, one for all the development work, one for all the testing, and one to deploy everything to production. Any other user stories are production support.

It seems a tremendous number of people do not understand the way of working that user stories represent and how to leverage that value.

How did we get here, and what is the point of user stories anyway? To answer that, we will start with looking at what came before user stories, the problems that existed, and where user stories fit into the solutions for those problems.

What was there before user stories and the problems it caused

A common way for large software companies to operate at one time was to spend months writing up the design before starting work. This would result in a rather large Product Requirements Document, which would then be passed to the developers to follow the detailed instructions of how to build the system. This came to be known as the “waterfall” approach, and there were several problems with it:

• Delays: The design phase took so long that when it was handed to the developers, it was already behind schedule.
• Missed Requirements: Questions by the developers were discouraged, leading to more time spent trying to understand intent and missed requirements.
• Poor Code Quality: There was pressure to rush the developers, leading to poor quality code and lots of errors.
• Missed Assumptions: There were always one or two wrong assumptions made that led to problematic solutions that did not realize their potential.
• Missing the Right Market: When conditions in the market changed, they were not able to adapt quickly, thus producing a solution that missed the opportunity or looked like imitation.
• Inflexibility: So much effort was put into the writing of the design specifications/requirements, that changes proposed by the developers to improve the design were not considered.
• Wasted Time: It led to weeks of time spent discussing a feature that ended up being cut.
• Increased Queues and Batch sizes: It resulted in work queues where work would sit and wait to be implemented, and large batches where a mistake could take weeks or months to identify.

How user stories fit into a solution for the problems of Waterfall

In the early 90’s there was a software project called the Chrysler C3 project, where they wanted to try a new approach. They asked some good questions about how to improve.

• How to reduce time to market?
• How to adjust to market conditions faster in an uncertain environment?
• How to reduce the cost of mistakes or wrong assumptions?
• How to incorporate new learnings into an existing design?
• How to improve code quality?

They came up with a new approach that came to be known as XP Agile. One of the central ideas was taking smaller steps through a process called Evolutionary Design.

The fundamental difference was moving from taking months to fully plan out a design, to having the design process be as short as possible, even just a few hours, identifying the smallest thing they could start working on right away. Their intention was to get quick feedback in order to prove assumptions, or get something small to market sooner and then expand it. They found the smaller you were able to slice work that could go to production as a unit, the faster they were able to achieve a return on investment.

The big problem was getting a skeletal understanding of the requirements so you could start building a first iteration. It was a common struggle to be afraid to give any answer until the perfect answer could be compiled into a massive doc.

User stories were an attempt to replace the large design specification documents while creating a simple structure to clarify the user facing context. They were tiny, only a few sentences or even just a few words. They weren’t meant to be comprehensive, or even a set of requirements, but to be placeholders for conversations for when the developers were ready to work on it.

User stories without understanding small batch sizes

To take months designing a solution before it is given to developers to write a number of user stories for it is like taking an engine out of a Ford GT and putting it into a tractor. Sure, you might get it to move, but it won’t win the Le Mans.

Let go of the belief that we should accurately capture all the requirements before starting work.

This creates the same problems as creating a requirements document, even if you write user stories afterward. It leads to creating bloated applications to handle all scenarios that rarely or never happen, but missing good solutions for the most common ones.

When we give up on believing that we can perfectly predict and plan for the future, we can start asking “How can I adapt my strategy to incorporate the fact that I can never perfectly predict the future?”. The answer is to deliver smaller slices.

User stories are not requirements. They are tools of planning used to reduce handoff and should be discarded after implementation. When we work incrementally, the first user story might not be in the final requirements, but be a rough draft that proves or disproves our assumptions.

User stories are to be placeholders for conversations. Not “Please read the stories and ask questions if you have them” (that is a handoff), but “Here is the problem we are wanting to solve, here is the current flow of the system, here are the proposed changes to the flow, any questions or suggestions?”

The true value of user stories

There are different mechanisms that people have come up with to help answer the question “How do I write a user story?” such as INVEST, the 3 C’s, RIVET and the Connextra method. However, the driving force behind user stories is working in small slices that go to production independently.

One of the biggest benefits of user stories is that it allows you to defer detailed analysis.

This prevents you from wasting time in detailed analysis of a feature that then does not get implemented at all, or then having to rewrite it because the design is improved.

User stories can answer questions like “What is the most difficult part of making this feature work?”. They can also be refactoring to prepare for new functionality.

So, keep in mind that user stories are only a part of a larger approach to product development that revolves around deploying smaller slices to validate assumptions early and often. For more understanding of how to do that, check out Splitting Stories - A Resource List.

Related Discussion:

We had an open discussion through our Industrial Logic TwitterSpace. Here is the recording of that conversation.