11 Laws of Software Estimation for Complex Work
Wrong estimates aren’t your fault, but they are definitely your problem
I was sitting alone in a meeting room and anxiously waiting for my manager to enter.
I had been summoned for a sudden meeting by our Chief Product Officer (CPO). He told me the meeting was urgent and important, and I should drop whatever I was doing to make it.
I was five months into my first job as a product owner. My seven-month contract was nearly expired, so I was sitting there worried and contemplating my future at the company.
Finally, the door opened, and he arrived with a beaming smile. Immediately my nerves disappeared. I still remember his words: “We just landed one of the biggest deals in our company’s history. I need you to work on an exciting new project.”
When I heard the words ‘exciting’ and ‘new,’ I instantly became suspicious. I began wondering, “What’s the catch?”.
He said, “To close the deal, we promised the customer some custom development in our SaaS product. I place my trust in you to deliver this crucial piece of functionality on time so we can finalize the deal.”
“Ah… there’s the catch,” I thought.
The conversation ended with the famous last words: “Don’t you worry, what they want is really basic.”
What had happened was that our CTO and CPO had decided together that what the customer was asking for was simple and easy to build. They also concluded it would be beneficial for many other customers.
I received a list of seven bullet points precisely as it was present in the contract. I had to deliver this functionality in twelve weeks. I had not been involved in the estimation, nor had any of the teams. I also didn’t understand what we were trying to achieve apart from building what’s on this list.
I immediately got to work. After chatting a few minutes with a senior developer, I felt a sense of impending doom: there was no way in hell we would make the promised timeline. I was surprised we’d sign a deal promising something we’d never be able to deliver. If someone had talked to a single developer, they would have arrived at the same conclusion.
C-level people had made the estimates without involving any teams. Those at the top rubbed their crystal ball, invented a date, and threw it over the fence to the teams.
Now it was our problem to meet that crazy timeline. And when I confronted my manager with the sheer lunacy of what he was asking of me, he’d said, “You’re overcomplicating the matter. Just keep it simple.”
Insane Estimates Doomed Us From The Start
We’d never deliver on time because the functionality in those seven bullet points required building new features in our whole product across ten different teams.
We had concluded the first bullet point would at least take two months. After that, we would have a foundation, and we would be able to predict with greater certainty how much the remainder would take. But I could already guarantee that it would never take twelve weeks in total. There was too much work and too many different teams involved.
What’s even more interesting is that even though the wrong estimate wasn’t my fault, it definitely was my problem now.
Those who created this mess told me to “Figure it out” and “Make it work.” I had many questions about the seven bullet points in the contract, but those were answered with “I don’t have time for that. Do whatever you think would be best.”
Great, as if this wasn’t difficult enough already!
In the first conversation with the customer, I immediately broke the bad news. I made a counter-offer to deliver the first bullet point of those seven in the contract earlier than expected. I also promised to work actively with them to include their feedback in developing it further. I also let them know the rest would likely take longer, and after delivering this initial piece, I would have a better idea of how much longer.
Luckily the customer agreed, even though in no way they would have been contractually obliged to accept.
In the end, the whole project took more than a year to deliver. Luckily, we were able to keep the customer happy because we worked closely together with them and managed their expectations.
What can we do to prevent getting in painful and challenging situations like this? Why are our estimates often wrong? What can we do to make them as accurate as possible? How can we make the best of a necessary evil?
After building software products for more than ten years, I’ve tried to capture all I’ve learned in these eleven laws of software estimation listed below; they can help you better navigate complex work and the complex reality of producing software estimates.
1. The work still takes the same amount of time regardless of the accuracy of your estimate.
Let me ask you the following question: “How many pennies are in this jar?”.
Most likely, your estimate would be wrong. You can’t even conclude how tall the jar is from the picture. However, your estimate does not influence how many pennies are present. Reality is unaffected by your guess.
The same logic applies to estimating or forecasting timelines. Delivering a feature still takes the same amount of time, regardless of your estimate. It is our expectations on timelines that create a perception of late delivery.
2. No matter what you do, estimates can never be fully trusted.
An estimate is a guess. It’s a prediction we make with too little information.
When doing complex work, we know we lack information. Our information and understanding are limited to the extent that getting the estimate right means we got lucky, just like when guessing the number of pennies in the jar right based on that picture.
Expecting your estimates to be accurate is like expecting to predict the future by reading patterns in tea leaves. When you lack any signal, you can’t draw conclusions about the future based on noise.
That’s why you should never trust your estimates. You make estimates with the best information you had at the time, which is inadequate to draw any definitive conclusions when you do complex work.
Don’t trust your estimates, and be aware that they reflect your current — usually awful — understanding. However, it’s a start and better than nothing.
3. Imposing estimates on others is a recipe for disaster.
Nothing is more frustrating than forcing estimates the team needs to execute on. Because those estimates often will turn out to be incorrect, the team that tries to deliver on them will develop feelings of resentment toward those that produced the original estimates.
If you want buy-in, and better estimates, let the people that execute the work come up with the estimates. Then if the estimates are wrong, which they invariably will be, the team can only point their fingers at themselves.
Throwing estimates over the fence is a recipe for disaster and a sure-fire way to start a blame game where everybody blames each other.
4. Estimates become more reliable closer to the completion of the project. This is also when they are the least useful.
Early on in a project, you know the least, so your estimates will be the least accurate. This is also the moment when you’d like to know most about the required effort of your project, as you can pull the plug on the project without having to exert any effort.
As the project progresses, you will gain a better understanding and have more information about what needs to happen. Your estimates will become more accurate. However, as more and more of the project is completed, the accuracy of your estimates becomes less important. Also, because of the time you’ve already invested in the project, companies become less likely to quit development.
But don’t be fooled. Even when you’re nearly done, you can still discover something that blows up the whole project. In software development, seemingly small issues can have massive consequences.
5. The more you worry about your estimates, the more certain you can be that you have bigger things you should be worrying about instead.
Estimates are a red herring. It’s easy to get distracted and believe accurate estimates and meeting timelines are what matters most. However, as said before, it takes as long as it takes, and your estimate plays no part in it.
When teams try to meet the imaginary timeline produced by the crystal ball, tunnel vision creeps in. Like a Fata Morgana that appears when you’re stuck in the desert, the only thing everyone sees is the looming deadline. The whole discussion reverts to ‘How can we meet that deadline?’ while ‘Is it good enough?’ moves to the backseat, and ‘How can we move faster?’ climbs into the driver’s seat.
Obsessive focus on ‘When will it be done?’ guarantees you will move the delivery of value to the backseat.
6. When you suck at building software, your estimates will suck. When you’re great at building software, your estimates will be mediocre.
No matter what you do, your estimates will never be accurate. The ability to estimate accurately has little bearing on your ability to build great software. Better teams will usually produce better estimates, but those better estimates still aren’t good enough to deliver accurately.
Like the weatherman who cannot predict the weather accurately more than a week in advance, software development teams quickly reach a ceiling in how accurately they can estimate.
No matter what you do, you can’t guarantee your estimates will be accurate or that you will be able to deliver on time.
7. The biggest value in estimating isn’t the estimate but checking if there is a common understanding.
When estimating, a different estimate usually means there is a conflicting understanding of what needs to happen. By talking about the work to be done, differences in opinion of what needs to happen surface.
Creating a common understanding is the biggest value in estimation, not the actual estimate itself, which will still often turn out to be wrong.
8. Keeping things simple is the best way to increase the accuracy of estimates.
Removing complexity and uncertainty, either by doing the work or before you start doing the work, is the best way to increase accuracy of estimates. Keeping things simple is the best way to prevent your estimates from ballooning and blowing up in your face.
However, keep in mind, if you make things too simple, then you’ll end up with the same kind of problems as when you’re overcomplicating things.
9. Building something increases the accuracy of estimates more than talking about building it.
It’s appealing to talk, analyze, design, debate, and do infinite research before starting the work to abolish all uncertainty and risk. However, no matter what you do, you’re still limited to the knowledge of beforehand: what you can know before starting the work. You quickly experience diminishing returns from trying to draw conclusions based on inadequate information.
By doing the work and starting, you discover the real problems that matter sooner.
Work with what you do know to discover what you don’t know.
Being book smart won’t cut it, you need to be street smart too.
10. Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
Our usual response to produce a better estimate is to sit endlessly in meeting rooms and break everything down to the smallest detail. By doing this, we lose valuable time we could spend actually doing the work. As stated before, this is where you discover the problems that matter. On top of this, all those elaborate plans are detached from reality and will ultimately slow us down more and more as the project goes on.
Sticking to the plan as it deviates more and more is a dangerous thing. It is better to start with simpler plans that are easier to adjust. When you break down the whole project to the smallest details, at least now you can fool yourself that you have a better idea of how much later the project will be delivered.
11. Punishing wrong estimates is often like punishing a kid for something they don’t and can’t know yet.
Some companies believe that it’s possible to produce perfect estimates as teams mature and become stronger. Therefore, any team that is not able to produce accurate estimates is incompetent.
When you punish teams for making inaccurate estimates, you are penalizing them for things that are out of their control.
It’s like punishing a kid for something they can’t and don’t understand yet. It’s a form of bullying that leads to panic and anxiety without producing any tangible benefits.
Stop wasting time on the delusion you will ever be able to produce accurate estimates
In summary, these are the 11 Laws of Software Estimation for Complex Work:
The work still takes the same amount of time regardless of the accuracy of your estimate.
No matter what you do, estimates can never be fully trusted.
Imposing estimates on others is a recipe for disaster.
Estimates become more reliable closer to the completion of the project. This is also when they are the least useful.
The more you worry about your estimates, the more certain you can be you have bigger things you should be worrying about instead.
When you suck at building software, your estimates will suck. When you’re great at building software, your estimates will be mediocre.
The biggest value in estimating isn’t the estimate but to check if there is common understanding.
Keeping things simple is the best way to increase the accuracy of estimates.
Building something increases the accuracy of estimates more than talking about building it.
Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
Punishing wrong estimates is often like punishing a kid for something they don’t and can’t know yet.
Past performance is not a guarantee of future results, especially when it comes to software estimation. In 1943, at a casino in the USA, the color red won 32 times. It’s easy to fool yourself that there’s some kind of pattern, but this is a classic example of the Gambler’s fallacy:
“A failure to recognize the independence of chance events, leading to the mistaken belief that one can predict the outcome of a chance event on the basis of the outcomes of past chance events.” — APA Dictionary definition
Many software teams suffer from a software development counterpart I have coined the Complex Work Estimation Fallacy:
“The misguided belief that by estimating more often and working together for a longer period at some point in time, you will be able to produce accurate estimates.” — Complex Work Estimation Fallacy by Maarten Dalmijn
No matter what you do, you will never be able to make your estimates accurate. Don’t fool yourself by acting under the delusion you can.
Ask yourself the following question: does it matter if you’re able to predict exactly when something is finished? If you’re confident you’re making the best progress you can towards delivering something valuable, does it make a difference if it’s slower than you expected if that expectation was always flawed and imprecise to begin with?
Adjust your sails, deal with the unexpected and uncertain, instead of being angry at the winds you will never be able to predict and control.