Why Better Planning Won't Fix Your Scaling Problems
Better Coordination Does Not Solve Poor Collaboration
You’re doing Scrum with a few teams. Life is great. Things are going swimmingly and the Scrum Teams are happy. Then, as you start adding teams and scaling up, the first cracks begin to appear.
The talk at the water cooler is that we’re experiencing scaling issues. Management begins asking questions about how we intend to fix these pesky scaling problems that are slowing us down.
The obvious solution is to introduce a scaling framework because they are designed to fix scaling issues. But is this really the kind of solution we should be going for?
Scaling Complicated vs. Complex
Imagine you believe you're doing complicated work. Armed with the right expertise, you can do a pretty decent job at prediction and planning. Up-front coordination works well, and there will be few surprises you have to deal with.
When you have a few Scrum Teams, you can get by with approaching complex problems as complicated. Sure, you will experience some hiccups and hurdles, but nothing you can’t overcome by sticking your heads together and grinding it through.
Those same hiccups and hurdles become exponentially worse as you add more teams working together on the same product. Complex work means many surprises even experts can’t see coming. A planning problem for a single team suddenly affects multiple teams. And because you have multiple teams, the likelihood that another team discovers problems becomes greater too.
Suddenly our trusted and familiar approaches stop working. Why does this happen?
Tackling complicated problems comes much more naturally to us than complex problems.
Complicated – The Realm of Command and Control Where Planning by Experts Rules
In the realm of complicated problems, when you possess the right expertise, there will be few surprises:
Your ability to plan and predict largely depends on your expertise.
When your plans fail, you need to do a better job at planning.
When people make mistakes, you must give them better instructions.
When your actions don’t produce the expected results, you must add more controls and checks to your process.
Move the best information to those who possess the most expertise. It’s not important for information to flow fast, as there are few surprises and things you can’t see coming. You produce the best results when you minimize deviations from the plan made by the experts.
Complex – The Realm of Collaboration and Adaptation Where Flexible Teams Rule
In the realm of complex problems, expertise still matters, but no matter your expertise, there will be surprises that can turn your whole world upside-down.
Your ability to plan and predict is limited. Expertise helps, but it can also work against you when you believe you know more than you actually do.
Your plans will fail, no matter your expertise or how good you are at planning. Your plans must change based on what you discover and learn.
Mistakes are inevitable. It’s how you respond and learn from these mistakes.
Your actions won’t produce the expected results because of your inability to predict. There needs to be a tight feedback loop to produce the desired results.
Move decision-making to those with the best information and ensure the right expertise is present to make the best decisions.
The problem is that we often try to apply our complicated approaches to the complex domain. We experience scaling issues that aren’t actually scaling issues. These problems are caused by the fact we’re doing complex work — we have to deal with many surprises.
We don’t like these surprises and believe we could have prevented them from happening. We mistakenly believe spending more time in meeting rooms, talking, analyzing, and doing up-front design will improve our ability to plan and predict.
Alas, this never happens. When you do complex work, you will have surprises. Period. That’s the nature of the work, and nothing we can do can prevent that. As a consequence of applying our complicated mindset to complex problems, we end up actually making matters worse. We try to solve for poor planning, through better coordination, while our plans weren’t the problem that needs fixing.
Instead of pushing for better planning and predictability, we should focus on improving flexibility and the ability to swiftly respond to changes.
You will keep on bumping into the limits of planning and predictability when you keep focusing on improving:
Sprint Planning
Big Room Planning
Dependency Management
Agile Release Train Coordination
Product Increment Planning
In fact, by trying to improve your ability to plan and predict, you won’t only bump into limits. You will inject speculation into your plans and make things worse. You will become locked into plans that limit the collaboration necessary to deal with surprises and the unexpected.
What works is the following:
Reducing dependencies between teams. The fewer surprises that need to be coordinated across teams, the better. We are usually better at intra-team collaboration than inter-team collaboration.
Eliminating silos that increase feedback loops and reduce the ability to make decisions. Teams encountering surprises means they need to act in the moment. Acting in the moment means there is no time to ask for permission, gain a blessing from superiors or wait for perfect information. Teams need to have sufficient freedom and understanding to respond to what unfolds.
Less time spent in meeting rooms and planning, more time spent discovering what you don’t know before starting. Build small things and get feedback, instead of just talking about building big things. Do experiments and learn what’s really going on.
Agile means working with what you do know to discover what you don’t know. It’s about gradually reducing the fog of beforehand and minimizing the fog of speculation. Every step you take helps shape the way. You don’t reduce the fog of beforehand, what you can know before starting the work, by wandering the caverns of your mind.
Instead of worrying about glorious plans, wonder whether your teams have what it takes to deal with surprises and the unexpected.
The ability to deal with surprises and the unexpected is the biggest obstacle to predictability, not your ability to predict.
"Less time spent in meeting rooms and planning, more time spent discovering what you don’t know before starting. Build small things and get feedback, instead of just talking about building big things. Do experiments and learn what’s really going on."
This is such an important thing. Yet, for some reason, so many teams just keep going through the motions and doing the planning (and failing) dance as you outlined.