Most refinement sessions feel like going to the dentist: nobody wants to lie in the chair.
I dislike most refinement meetings for two reasons:
1. Massive time sink that wastes everyone time. We’re arguing about completely different levels of abstraction. Some debate what color wallpaper the house should have, while others argue about the best way of tackling the plumbing. As a result, reaching agreement on anything becomes extremely difficult and people don’t really listen to each other.
2. We don’t talk and think about the things that really matter. We get stuck discussing irrelevant implementation details and steamroll into execution mode. We obsess over moving quickly, instead of slowing down and taking some time to really think things through so that we can actually move faster.
Why do we usually end up in these kind of nightmare refinement sessions? What can we do to fix them?
The typical solution is to create an even better, more exhaustive template and waste even more time preparing refinement - only to make things even worse than before.
The amount of preparation is rarely the problem for your refinement, it’s the level of collaboration, exploration and discussion.
If anything, refinement usually sucks because we’re over-preparing and silencing the necessary conversations and collaboration that are crucial for building great products.
You don’t need a better template, you need better collaboration and facilitation.
Achieving this is impossible unless Product Managers adequately manage the four different levels of abstraction that are at play.
Your Typical Refinement Session: Ignoring the Four Levels of Abstraction
What usually happens during refinement that someone presents a set of (nearly finished) User Stories that attempts to tackle all four different layers of abstraction at the same time.
We talk about (1) the problem we’re trying to solve and why it matters, plus (2) how we intend to solve from a functional perspective and (3) all the technical steps that are necessary to bring it to fruition - and we’re discussing all these steps simultaneously while we’ve already decided how to (4) slice the work because all those User Stories have already been created.
The problem with this approach that we’re trying to have four entirely different conversations at the same time that all depend on each other.
We can visualize the four abstraction levels of refinement as follows:
User Stories usually serve to make this concurrent abstraction noise even worse, because User Stories can belong to one abstraction level or cover all four of them at the same time. This is what makes using User Stories potentially extremely treacherous.
Some User Stories are actually real-world stories of Users that contain the problem we’re trying to solve while other User Stories are reverse engineered from functional or technical implementation details. That means they’re actually not User Stories but User Fairytales. But we call them User Stories all the same and don’t distinguish them from each other.
The big problem is by expressing everything as a User Story, that nobody can tell anymore which abstraction level we’re talking about. Are we talking about what the user needs? Are we talking about how it’s going to work? Or are we talking about both?
This makes refinement ineffective and confusing, we can visualize it as follows:
Talking about the four abstraction levels simultaneously makes everyone dizzy and confused, because we’re going through a set of User Stories that contain a fragmented and duplicated version of the whole picture. We will be trying to answer far too many different questions at the same time, and causing the team to experience super high cognitive load.
In short, when you use User Stories across all four abstraction levels, it becomes extremely difficult to discern which abstraction level we’re talking about because they can be used at any abstraction level, or even cover all of them at once.
The end result is that everyone zones out, because we’re trying to have four different conversations at the same time where everybody will care about different things.
How do we fix this insanity where everybody is talking about about different abstraction levels at the same time?
Mastering the Four Abstraction Levels of Refinement
The first pass of refinement should be to go through all the abstraction levels in a linear and stage-gated fashion. You start at the top (1) and you go to the bottom (4).
Once there’s clarity about each of the different abstraction levels after the first pass, there’s freedom to traverse them as necessary to figure out the best solution.
You begin with the problem to solve. When that’s clear to everyone, you talk about the functional solution and how it relates to the problem to solve. When that’s clear to everyone, then you talk about the technical steps to realize the solution. Then you talk about the simplest first humble slice of work that you can use to discover the best solution possible.
The reason for this simple: the higher abstraction levels are anchored in the lower levels. Everything is connected to each other. If you discuss all of them at once, then the picture becomes murky and causes a complexity explosion in our heads.
After you did a first pass, it’s really important to slow down and iterate over different ways of solving the problem, or maybe even we can reframe the problem to solve it differently. You have to take your time and come up with different solutions, so you can go for the simplest one where you maybe even have some doubts that it will work.
It’s crucial spend a lot of time talking about how we slice the work intelligently so that we delay decisions as much as possible to a later point in time when we have better information and richer understanding.
Remember: the best line of code is the one we don’t write. We must create humble slices of work where we can work with what we do know to discover what we don’t know.
By working this way we can respect Gall’s law, which is one of the most important laws in software development:
“A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”
If your first milestone isn’t a simple system, you’ve already screwed up and set yourself up for failure from the start.
You should think in terms of weeks for the first version, because otherwise you’re probably going down the rabbit hole of building a complex system from scratch that will slow everything down you’re going to add to it.
Stop Ignoring the Four Abstraction Levels of Refinement
The paradox of effective refinement:
The more and better you prepare for refinement as Product Manager by having a set of finished work items, the worse your refinement session will usually be.
The more all the work has already been crystallized before refinement, the more likely it is to be suboptimal, disconnected from reality, completely wrong and cause cognitive overload for everyone involved.
If you’re in such a situation, here are four steps to follow:
1. Don’t start refinement with a finished set of User Stories. It’s a waste of everyone’s time, because User Stories frequently cover all different layers of abstraction. If you do want to start with a User Story, keep it purely rooted in the problem we’re trying to solve. And when you break it down in functional or technical chunks, refer to the original User Story and stop using User Stories, as otherwise everyone will be confused what abstraction level we’re talking about.
2. Start by going through each of these layers of abstraction in a linear fashion. You don’t need any User Stories for this. Don’t discuss the solution unless the problem we’re trying to solve is clear to everyone. Manage the cognitive load, so that everyone has a clear understanding of the whole picture before we discuss how everything relates to each other.
3. Iterate across all layers of abstraction to get the simplest solution and first milestone possible that will potentially solve the problem.
When there is sufficient understanding of all four levels of abstraction, that’s the moment you should go through the different layers in a nonlinear fashion and slow down to iterate over different potential first versions together with your team.
Remember: the first version is rarely the best. Slicing the work all about creating a humble first milestone that acknowledges we’re not as smart as we think we are and allows us to take into account Gall’s law: complex systems have evolved from simple systems that worked
The solution we’re building is complex and iterative, where anything we learn on one abstraction level can affect all the other levels, but the initial narrative we present to the team should be linear, simple and incomplete to manage the cognitive load.
Because that is the only way we can prevent cognitive overload and create the necessary conditions for collaboration, conversation and friction to produce the best solutions.
During refinement facilitate the first pass of the four abstraction levels linearly, so you can ultimately discover the best solution that will work in a messy and nonlinear fashion.
"The problem is that we’re trying to have four entirely different conversations at the same time" is indeed what I face most.
I have no problem that all 4 levels are discussed in the same workshop, but it has to go through the different levels step by step, and - in my experience - first and foremost the problem space needs to be clarified way better.
Teams are so rushing into solutioning without even understanding the problem to solve, if it is even a problem to solve...
Strong article.
As a facilitator, I would regularly ask:
"What level of abstraction are we discussing right now?"
"Are we talking about the problem, the solution, or the implementation?"
In my experience, many refinement discussions become difficult simply because people are unknowingly operating on different abstraction levels.