How Reference Stories Solve the Challenge of Story Estimation across Teams

Are you trying to calculate the sizes of complex features? A feature is composed of many stories that may have been estimated by different teams. But are those estimates consistent across teams and over time? Probably not. Stories may be impacted by the teams’ differing skill sets, changes in environment and assumptions, and other factors.

Story estimation is a challenge for all Scrum teams. Teams estimate their stories during backlog refinement and planning sessions. At these events, they assign estimates to stories using story points, often using a Fibonacci series. When deciding how many points to associate with a story, they consider factors like risk, complexity and effort. These estimates tend to become more accurate and consistent over time. But what happens when multiple scrum teams work together to create a single feature/epic? How do you get teams to estimate stories consistently across teams?
One team may size a story as a 3 (story points) but another team sizes the same story as an 8. To understand their true velocity, it’s useful for teams to provide consistent estimates. This is where the concept of reference stories is useful.

A reference story serves as a guide to assist teams in estimating by providing a benchmark story to base new estimates on. The teams create reference stories with specific story point values. Then, when estimating a new story, they use the reference story to determine its size in story points. Later, the total size of these estimated stories can be used to determine the size of their associated reference features.

So How Do You Create Reference Stories?

By conducting a reference story workshop. Who participates in the workshop? All teams do. If you’re using the Scaled Agile Framework (SAFe), this would include every team in the release train. Everyone works together in the workshop to create the reference stories. The purpose of this event is to identify and size the stories that will define the guideline/example stories that are then used to define the estimate for future stories. Once these reference stories are defined, the teams will compare an un-estimated story to these reference stories to determine its estimated story point value. This will make all future story sizes consistent!

NOTE: All teams participate. It is critical that everyone agrees that these reference stories will be used for all future estimations.

Workshop Steps:

Preparation

• Gather a list of 10 to 12 stories, preferably ones the team has already worked on.
• Print out each of these stories with full description and acceptance criteria.
• Create a spreadsheet for the stories, as shown below:

• Invite the full scrum team: Product Owner (PO), Scrum Master (SM) and team members. Only team members will determine the estimates. POs and SMs can provide information about the stories as needed.
• It’s best to have the team on site and around a table large enough to layout the printed stories.
• Depending on the size of the group, allow an hour to ninety minutes for the workshop.

Ranking the Stories

• Start by reviewing the purpose and value of the workshop.
• The first exercise is to have the group rank the stories three times, according to each of the following criteria.
  • Complexity: this includes time to research, analyze, plan, and communicate. Story complexity should be based on an average for your team and not for any specific team member.
  • Risk (Unknowns): these include dependencies, rework (fixing defects, changes based on stakeholder feedback, retesting, etc.), and time spent waiting to be able to do work. If you know your stakeholders are likely to request changes, factor that into your estimates. Backlog refinement identifies risks and reduces unknowns, but these always exist in new development.
  • Effort: this is the time required to code, test, and validate the code. In other words, the time not spent on complexity and risks. For simple stories with few risks/unknowns, most of the time will be spent on “Effort.” For complicated stories with many unknowns, “Effort” may make up a small percentage of the total time needed
• Begin the process by comparing stories, based on Complexity. Ask the teams to compare the first two stories and have them pick which story is larger, based on Complexity.
• Next, bring in a third story and ask how it compares to the first two stories. This continues until all stories are ranked based on complexity.
• Once this first round of ranking is completed, add a numerical ranking to each story. The least complex story has a rank of 1, the next-least complex has a value of 2, and so on, until all stories are numbered. For example, if there are 10 stories, the most complex story is ranked 10 and least complex is ranked as 1. Record these rankings in the spreadsheet.
• Next, repeat the process two more times to rank the stories by Risk and Effort.

Before ranking the stories, reorganize them to avoid having the Complexity ranking impact the Risk ranking.

• When the ranking of the three areas is completed, sum the Complexity, Risk, and Effort values for each story.
• Next, sort the stories in the spreadsheet based on the sum of the values, with the largest stories first. The spreadsheet should now look like this:

Estimating Stories Using the Combined Rankings

• Now, pick a story in the middle and assign it five-story points. Next, the team will estimate the remaining stories using a Fibonacci series.
• Select the next larger story and ask the group how many story points this story has, in comparison to the first 5-point story. This could be the same figure (or larger).
• Next, ask the team to estimate a story that is one smaller than the original 5-point story.
• Continue this estimation process, alternating between larger and smaller stories, until all stories have story points assigned.
• Now the table is complete and will look as follows:

What’s Next?

Now that this process is completed, the teams will use these reference stories in the refinement and planning process. Work with the scrum masters to ensure the teams are following through. The temptation to revert back to gut-feel or bottoms-up estimating will be strong at first, but it’s important not to slip back into old ways. Using these reference stories across all teams should make your estimations more consistent. As your organization takes on larger efforts, this consistent estimating will allow for the creation of reference features. A reference feature size is determined by its associated stories and provides a great tool for estimating large-scale efforts.

While this guide to creating reference stories should help your teams immensely, if your organization still needs support navigating the ins and outs of Agile processes, please don’t hesitate to reach out to us. Our experts at Anexinet would love to help you get started.

*Thanks to Victor Penman for his assistance with this article.