Welcome to the first in a continuing series of blog posts about the ongoing Agile transformation happening at CenturyLink Cloud. We are a development organization spanning several sites in the US and Canada, all responsible for creating either our underlying cloud platform or one of the products that runs on top of our platform.
As an organization, we are committed to building the most effective development teams that we can. For us, that means adopting a process closely related to Extreme Programming, Lean/Kanban, and DevOps. We have small teams, each of which is as cross-functional as we can make them, and they are responsible for the product management, creation, and support of their individual products.
The level of Agile knowledge varies from team to team, but they're all working towards raising that level. Our goal is to continuously learn and improve, day to day, week to week, and release to release. We're not perfect, but our improvements always keep perfection in mind as a target.
In this initial post, I'd like to talk about how many of our Product Owners are evolving in the craft of story writing and how this evolution benefits both them and their teams.
Fishy User Stories are Good User Stories
Calling a user story a fishy thing may seem a little derogatory. So why is being fishy a good thing for a story? There's actually a reason for this that will become clear shortly; but first I'd like to introduce the topic a bit and give some background about the current situation on some teams.
Stories versus Tasks
There are a couple of terms I want to define and make clear before we get started. The first is user story, which is a small piece of functionality described by the Product Owner and implemented by the team that provides value to one of the end users of a product. The two key characteristics of a user story are that they are independent, meaning that you can implement all of a single story and provide some value to a user, and that they do in fact provide some value to an end user. The second term is task, which represents a development activity that, when taken with other tasks, adds up to a user story. By their nature tasks are not independent, nor do they provide end user value by themselves.
An example of a story is "Allow users to retrieve cycle time information for any story within the current iteration", which is made up of many tasks, such as "Query the database for start and end dates of given story". The value in the story is obvious, while the task is a piece of what it takes to produce that value.
I like to summarize this by saying that stories represent value while tasks represent effort. We use effort to create value.
This distinction is important to the rest of this post.
User stories are generally not atomic, meaning that they can almost always be broken down into smaller pieces. Large user stories can be broken into smaller stories, each of which still provides a piece of end-user value. This process can repeat many times, with stories being split into smaller and smaller stories, as long as the result still provides end-user value. You can end up with two or three or a hundred user stories that each add some value towards the larger feature.
At some point, however, you get to the end of that line. Some stories are so small that any further decomposition leaves the realm of story and enters the purview of tasks. These subsequent breakdowns become specific implementation steps that represent the effort expended to develop the encompassing story.
Our point of discussion for this article comes in when stories are almost small enough to implement, but still a bit too large. At this point, there is still another level of breakdown that is needed, but that breakdown can go one of two ways. And the direction it goes, in large part, is dependent on who does the breakdown. A Product Owner, whose main focus is building the right product for the end-user, will break this down into smaller stories. A technical team, whose main focus is building the product right, will tend to break this almost-small-enough-story into tasks.
And herein lies our topic.
Iteration backlogs filled with tasks
Several of our teams begin their iteration planning meetings with stories that are in that "just a bit too large" state. The technical team digs into these stories to really understand them, and in the process starts to architecturally decompose the work into tasks. That leads to the technical stories in the iteration backlog.
Example Let's look at an application I'm building in my spare time. I'm building a metrics application that draws information about user stories and the data associated with them from Trello, stores them into Orchestrate, our newly acquired DB-as-a-Service, and provides web applications to view the metrics. All of this is housed on AppFog, our Platform-as-a-Service.
The first feature I am trying to implement is to create a chart of cycle times for stories for any team over a specified time period. This story is pretty big for a single developer like me to implement, especially because I'm actually the Director of Software Engineering, and they don't let me write real code anymore! Here's that first story as a Trello card:
The first thing I did was to play around with the Trello API a bit, to learn how data is extracted from it. Then I dove into Orchestrate, to learn how to get data both in and out. After learning what I needed, I drew up a candidate architecture consisting of several microservices to be run inside AppFog.
At this point, I have several ways I can go. I already know what I want to build and all the features I want; so I could just start building the feature. Let's assume I didn't go down that path because it would be really uninteresting for this article! The two real options I have are to break the large story into tasks or technical stories, or to decompose this into smaller user stories.
The breakdown into technical stories might look something like this:
Create script to retrieve data from Trello and store into Orchestrate.
Create query script to get data out of Orchestrate.
Build RESTFUL endpoint to allow UI to provide date range and receive cycle time data.
I could start implementing each of these tasks one at a time and, at the end, I would have created the entire feature I wanted. However, there are some issues with this from several points of view:
To start, teams that break down stories like this are unintentionally cutting their Product Owner out of the discussion loop. Since each of these tasks represents technical activities and doesn't easily tie back to something the user might care about, it's hard for the Product Owner to engage until they can see something.
Another closely related issue is that the Product Owner can't offer any advice or steer the implementation of the larger story until all of the tasks are finished and the final implementation can be demonstrated. Since none of these tasks are independent, it takes all of them to produce something visible to an end-user. That delays Product Owner feedback about the feature being developed until development is finished. This adds risk to every story since feedback is delayed, which can lead to unnecessary rework.
The final problem, or at least the last one that I'll mention, is that there is an unspoken integration task that happens when all of the tasks are finished. When developers work on independent tasks, they have to fit them together like puzzle pieces when they're all finished. Unless they were able to perfectly predict how the pieces would fit together, there is always some work to integrate the tasks into the whole. This adds additional risk and time to the implementation of every feature.
On to the Fishy Part
There is a different way to break down user stories that some of our teams use, one that mitigates the issues described above. Instead of breaking down the user stories into tasks, they continue the decomposition of the user stories into ever slimmer pieces. When you do this, the Product Owner can remain involved throughout the entire development cycle. The stories each represent something they understand, which allows the Product Owner to see incremental progress through implementation and react to and mitigate architectural and integrations risks as they arise.
It takes a bit of coaching and practice to get to this kind of story breakdown. The key to it is understanding what the essential bit of value in the user story is and implementing that first. As a metaphor, imagine the user story as a big block of rock or ice. Climb on top of it with a spike and take a core sample of that block, all the way through it. That thin, narrow, vertical slice represents that story essence that we'll implement first. We'll get that working. Then, we'll keep adding onto it, incrementally expanding the initial core until it's as big as we need it to be.
As before, let's take a look at an example of this.
Example In my mind, the essence and value of the first story from above (cycle time report) is to provide the user with the ability to get the cycle time of any individual story. Assuming we can get that, everything else can be added later, including the date ranges, the UI, the ability to see multiple stories, and the additional information about each story. So, let's build that first.
Here is that first story, along with its acceptance criteria:
Something important to note is that the acceptance criteria are much more targeted to the specifics of this story, defining exactly what cycle time is, than in the above example. Since this smaller story was broken out of that larger one, we're better able to specify our acceptance criteria more narrowly. This lets the Product Owner describe more specifically what it means for a story to be finished, and allows the development team to understand a story better, which leads to better estimates, implementation, and testing of the final product.
To implement this, the developers still have to pull data out of Trello and store it into Orchestrate, query Orchestrate for a single card, grab the data from that card, calculate the cycle time, and then return it from a RESTful endpoint. In effect, they're having to create the first implementation of a simple system that cross-cuts all of the architectural layers. From experience, none of this is very hard and should be able to be done in about a day, assuming the team previously learned how to do each of these steps individually.
Once the story is finished we have the ability to get the cycle time for a single story, show this to a user, and validate that this is the information they need. It is small, and perhaps not incredibly useful as a standalone capability of our system, but it is enough to gather some feedback and to act as a core for the rest of the feature!
Moving along to the next story, let's expand our core a little bit. The question to ask yourself when deciding what to build next is to consider what's the next most risky or valuable piece of this story? In my mind, I'm not worried about building the UI, as I'm planning for a Single Page App driven off the RESTful endpoint we're already building, and that's not very risky.
I found out more about Trello's API while implementing the last story which caused me to adjust my architecture. The previous example used a static load of data from Trello that didn't reflect any changes to cards made after the data was loaded. Trello, however, updates cards in real time as they're moved across a board. What we need to do is define a webhook that is deployed to AppFog, listens to updates from Trello, and stores them as events into Orchestrate, which can then be queried for the information we need. That's all pretty technical, but I can summarize it into a story that is entirely user and business facing:
I happen to know that this work involves creating a Trello webhook and an endpoint to receive the webhook call, though none of this is called out in the story. It's entirely up to the development team to figure out how to implement this.
Once this was finished we encountered another problem. In the process of testing it, we discovered that Trello tells us about updates to any card that is moved along our board, even cards that were created after our initial data load. This caused an error when the webhook tried to store the data, since we didn't have any knowledge of the newly created card. So, we needed another story that added information about the new card into the static data loaded previously. I'll show this in a bit less detail as, by this point, I think you get the idea about the content of these more focused stories:
Allow for reporting cycle time on newly created cards.
Once these few stories are implemented, we should be able to create a story in Trello, move it across our board until it is finally complete, and then retrieve the cycle time for that story. This should work regardless of whether the card existed prior to us initially loading data or was added afterwards. That works, can be demonstrated to anyone, and has some value, albeit limited.
Now that this piece of the overall larger feature is implemented, we can move on to the rest, which might start off like this:
Return cycle times for all stories in a date range.
Add additional information for all stories returned.
Create basic UI without distinguishing types of stories.
Distinguish different story types by color.
Allow date range to be chosen by user.
...and so on.
The different directions taken in story breakdown illustrated above is common on Agile teams. There are pretty clear advantages to working with user stories rather than technical stories, and we are trying to help teams make this leap. By focusing on slicing user stories as thinly as possible, as a Product Owner you can:
Get feedback early and often.
- Act on that feedback and adjust as you go.
- Stop adding to the feature when it is complete enough.
- Better define what done means for each story.
Understand and prioritize your backlog.
I do have to admit. This different way of thinking can be hard to adjust to. As a Product Owner, you have to believe that the incremental progress and frequent feedback is worth the price of writing smaller user stories. And, as a developer, you have to believe that it's OK to build something simple at first and let it grow and evolve as more stories are implemented.
The conversation is ongoing and will take a while longer to spread across the entire organization. In the end though, I hope to reach a place where all stories are business and user focused, leaving teams free to implement them in any way necessary to accomplish the goal. The result is we're able to constantly deliver value every day and always building towards a larger goal.
Oh yeah, about the fish... The idea of slicing user stories into very thin slices has been called User Story Sashimi, given its goal of taking the whole and cutting off very thin slices of it. If you're interested in more information about slicing and dicing your user stories, you can google User Story Sashimi to read more about it.
Good things to come Please check back over the coming weeks. We'll be adding more posts about our journey further into the Agile development process. Upcoming topics include favorites like the value of tracking and managing cycle time, learning how we use cumulative flow diagrams to understand teams,Agile project management,and other thrilling subjects.