Cost of Delay or Crystal Ball 🔮 of Delay?
Are Prioritization Frameworks Fool's Gold?
I recently stumbled on an insightful post by Sam McAfee on prioritization frameworks which resonated strongly with me.
Sam has coached hundreds of teams on prioritization frameworks since falling in love with Weighted Shorted Job First in 2009. He writes the following about prioritization frameworks:
"Let’s just say I’ve explored this issue pretty thoroughly. And I’ve never really found any of them to be as useful as I really wanted. Each of them tries to apply math to something that can’t be quantified: How is our customer going to value this thing we are making? And how many of them will? And at what price? Now let me be clear: you CAN use data to estimate delivery times probabilisticly. And that is something worth doing. But the “value” column in your prioritization table? It’s nonsense."
My experience mirrors that of Sam, though I don't share the same extensive background in coaching teams or guiding others in prioritization frameworks.
Let me tell you a story from my personal experience to show how destructive the wrongful application of prioritization frameworks can be.
The Fool's Gold of Prioritization Frameworks
Once per quarter, all Product Owners would be summoned to sit in the same meeting room together and discuss all roadmaps. Every item on the roadmap would have a business case attached to it that was painstakingly created based on a pre-defined template.
The business cases served three purposes
To decide what not to do
To prioritize what we would be doing
To boss other teams around with big numbers so they would be forced to help you.
Nobody would dare to admit it, but that third point was actually the main purpose of the exercise.
The business cases on the roadmap determined the worth of what your team was doing. Teams that had business cases low in terms of expected money, were considered less valuable. The leadership team loved the teams that had big business cases.
As a result, everybody would inflate their self-invented business cases in an attempt seize even more power for their department.
The more of these roadmap sessions I attended, the more I witnessed the rubble left by prioritizing based on such business cases. I've seen features get delayed that were guaranteed to deliver money (e.g. introducing the most popular payment method of one of the largest countries the e-commerce store was operating in), because they got thrashed by hypothetical business cases that turned out to deliver zero value in the end.
Deciding what not to do, and prioritizing based on numbers invented on an Excel sheet, is often a terrible idea. Let me tell you why.
Business Cases Often Bias Prioritization Towards Imaginary Money
Prioritization should start with customer value. Things that are valuable to the customer, can be difficult to express in money. Because you can't immediately put a monetary number on it, does not mean those things are not worth doing.
Let me give a concrete example. Let's imagine you work for Atlassian and you decide to add a feature to support keyboard shortcuts in Jira to make it easier to add issues. How much money would it bring in? How sure would you be of that?
Now imagine Atlassian is missing the most popular payment method in the Netherlands (iDeal) for their subscription page. With a little bit of research, you probably could come up with a pretty good guess how much money it would bring in when you would add it.
However, the important thing to keep in mind, nobody will care about that additional payment method being there, if they're not getting sufficient customer value to want to upgrade their subscription.
As a result, business cases that spit out lovely (often imaginary) financial numbers will get prioritized, but that isn’t necessarily what will lead to the most business value long-term.
Look at Facebook as a telling example. When they started out, they were delivering customer value but didn’t have a strong business model to capture that value. Facebook now has too much focus on business value and too little focus on customer value. Their stock price is plummeting, and they are making a last stand with the Metaverse.
What makes something valuable is difficult to predict. I once gave a workshop for a group of Scrum practitioners, and when discussing what value meant one of the participants noted:
"My wife once bought a cargo bike. I was strongly against and didn't see the benefit. Then, the first time I drove with the bike it opened my eyes and I immediately realized the freedom and flexibility it brought to the table. Since then, I'm a big fan and use the cargo bike all the time."
Even the customer sometimes doesn't know what will make their life better, until they use the product for the first time and experience it themselves. The value of something is expected to provide is often riddled with noise and assumptions, that are difficult to unentangle without proper experimentation and research.
In Dutch we have a saying:
"Jezelf rijk rekenen."
The saying means you're counting yourself rich. You don't get rich by counting, you get rich by delivering actual value.
Focus More on Discovery, Less on Prioritization
Marty Cagan refers to discovery as how we figure out what to build.
We shouldn't start with building, we should start with learning.
On the Y-axis you have the Believability of Information, and on the X-axis the Level of Effort. Here's how I express believability in simple terms everyone can understand:
"Every feature is innocent of delivering value unless you have evidence that proves otherwise."
When you start, you have the least information, understanding, knowledge and evidence of what you're trying to do. Hence, you should try to do simple things that require little effort and produce quick feedback loops that reduce the value uncertainty (or increase value confidence). Then, as you gain more information and evidence, you can decide whether the opportunity is worth pursuing (or not).
The curve beautifully shows that building is an expensive and slow way to find out you're wrong. You want to find out you're wrong in less expensive ways, and then as you have more confidence, invest in more expensive and believable methods for gaining more information.
The fundamental problem is that we often believe we know more than we do. And to make matters even worse, as Sam McAfee beautifully expresses it:
"We’re so afraid of being wrong that we’ll spend weeks gathering data to try to make the formula accurate enough that we don’t have to make a decision, because the numbers will do it for us. It’s just risk aversion. And risk aversion kills innovation and creativity. You want risk aversion, go work in insurance."
If you want to deliver value, it's time to roll up your sleeves and get your hands dirty. You can't discover the reality of delivering value if you're stuck in your head shuffling imaginary numbers on a spreadsheet and not talking to customers.
No prioritization framework can remove noise or conjure information you don't have out of thin air. Focus on removing the noise first and discovering the signal, before you worry about prioritization.