I've been through a ton of teams, companies, and projects over the last decade. I've yet to see a team effectively size their work and create in a way that was predictable unless it was super super small bodies of work.
I've only known the teams I've worked with to adapt to the chaos of the day-to-day. I still find it fascinating that there are teams that have their shit together somewhere in the world enough to use predictability.
A thought that occurred to me while reading your insight... Indeed it's not about predictability strictly speaking. But could it be said that in some broader way there's something to it, in this more precise sense: being more predictably successful (as in increasing the chances) in achieving the outcomes we need to achieve by being adaptable and advancing incrementally?!
I disagree to this view. For many reasons. In any serious, sustainable business an answer to the question 'When will it be done?' is crucial. And while, due to the inherent uncertainty of the work, a certain answer is not possible, a probabilistic one often is. And this in turn allows one to make the right decisions in a competitive market, and it informs one about the risks taken. So, while being agile may be primarily about being able to adapt to changing conditions, being also able to forecast with reasonable certainty - for example - by when those adaptations lead to a valuable result is not just valuable in itself, it's a necessity. Perhaps unless you're going for a high risk, one-time shot or feel comfortable to build a business on luck. Perhaps. The work Dan Vacanti and others have done in this area is invaluable.
I am familiar with their work, and what I'm writing still applies. Your predictions will be wrong. I've worked on many products and then you suddenly discover something that changes everything. The work isn't even part of your forecasts, until it is discovered.
I'm not saying you should not do this forecasting or there is no value in them. There absolutely is. But just like the weather you will still be wrong, especially the further in the future.
Only if whatever you discovered makes the near future vastly different from the recent past in terms of the flow of work. And this is then a true black swan event, extremely rare, even in new product development, not to mention the vast majority of work in SW-Dev, which is even less plagued with uncertainty.
It's not a black swan event at all. It depends on what you're doing. When rebuilding whole products, or making computer games, it's common. If you read 'Blood, Sweat and Pixels', every single game in there is delayed at least once.
Most games from Valve Software are delayed at least once too. Google the concept of 'Valve Time'.
I agree with you, when you are polishing a product or making small improvements, it's much more predictable. Then something like that popping up is far more rare, and black swan like.
But that's not all development work. Rebuilds are especially treacherous, in my experience.
Oh, interesting. If there's a whole book with loads of projects that are called unpredictable, I'd say this is evidence for a rather 'predictable unpredictability'. Whatever causes 'Valve time' to happen, causes may range from a immature understanding of preserving options, to a lack of late commitment, and some dozens of other possible shortcomings or dysfunctions. Even a inability to create a workflow that is within reasonable bounds and not behaving chaotic. We'll never know. To attribute observed delays to a inevitable and inherent unpredictability of a flow of valuable work, is quite unlikely. Besides that, dropping some work items from the backlog and replacing them with others doesn't change anything. Oh, they had to drop already delivered work? Well, what happened to fast feedback? I see a lot of possible causes for the delays, none of which has to do with inherent unpredictability. Of course, when you start a multi-year project, a forecast of the latest delivery date with 90% certainty in the first week of the project would most likely make every investor pull the money out and flee. Just like noone can forecast with reasonable certainty (hm, say 85%?) whether we'll have white Christmas in January. If that is the predictability, you refer to, we agree.
You can predict the flow of product backlog items, that doesn't mean you can predict when the puzzle is solved.
When is a game fun or good enough?
When is a product good enough?
We focus on completion rate of features, but that's like the completion rate of words. When all the letters are on paper it does not mean the book is done.
That's why we have to watch out when we say it's predictable IMO, because it isn't. From the putting letters on paper it's pretty predictable, but from writing a book that achieves certain goals when a reader reads it that adds a whole another level of unpredictability.
When Toy Story story board was completely finished, and on time, Disney pulled the plug and told them to rewrite the whole damn thing.
You could learn something new when you deliver something, and that may have a huge impact on the remainder of the work.
That's what I'm talking about and why it is unpredictable. We cannot guarantee up front that what we do is exactly what will create and/or capture value, and as we go on that journey it may have a big impact on timelines because what's done isn't done.
And that's asides from technical surprises that frequently happen when working with legacy systems or doing rebuilds.
If you can plan and predict, there is no need to adapt.
If you can adapt, it does not mean you should not plan and predict but you have to realize you are placing your trust in an unreliable narrator.
How come Microsoft, Amazon and the like deliver relyably and predictably? How come whole companies like Ultimate Software manage 50 Teams using continuous forecasting, and deliver on time without any estimation? In the light of evidence that probabilistic forecasting works, it's hard to convince me, that it doesn't. Especially not with examples where some sponsor simply pulls the plug for whatever reason - a rare and impactful and single event. Of course one can leave the house and be struck by a lightning bolt. That doesn't mean you're facing something inherently unpredictable.
I've been through a ton of teams, companies, and projects over the last decade. I've yet to see a team effectively size their work and create in a way that was predictable unless it was super super small bodies of work.
I've only known the teams I've worked with to adapt to the chaos of the day-to-day. I still find it fascinating that there are teams that have their shit together somewhere in the world enough to use predictability.
A thought that occurred to me while reading your insight... Indeed it's not about predictability strictly speaking. But could it be said that in some broader way there's something to it, in this more precise sense: being more predictably successful (as in increasing the chances) in achieving the outcomes we need to achieve by being adaptable and advancing incrementally?!
I disagree to this view. For many reasons. In any serious, sustainable business an answer to the question 'When will it be done?' is crucial. And while, due to the inherent uncertainty of the work, a certain answer is not possible, a probabilistic one often is. And this in turn allows one to make the right decisions in a competitive market, and it informs one about the risks taken. So, while being agile may be primarily about being able to adapt to changing conditions, being also able to forecast with reasonable certainty - for example - by when those adaptations lead to a valuable result is not just valuable in itself, it's a necessity. Perhaps unless you're going for a high risk, one-time shot or feel comfortable to build a business on luck. Perhaps. The work Dan Vacanti and others have done in this area is invaluable.
I am familiar with their work, and what I'm writing still applies. Your predictions will be wrong. I've worked on many products and then you suddenly discover something that changes everything. The work isn't even part of your forecasts, until it is discovered.
I'm not saying you should not do this forecasting or there is no value in them. There absolutely is. But just like the weather you will still be wrong, especially the further in the future.
Only if whatever you discovered makes the near future vastly different from the recent past in terms of the flow of work. And this is then a true black swan event, extremely rare, even in new product development, not to mention the vast majority of work in SW-Dev, which is even less plagued with uncertainty.
It's not a black swan event at all. It depends on what you're doing. When rebuilding whole products, or making computer games, it's common. If you read 'Blood, Sweat and Pixels', every single game in there is delayed at least once.
Most games from Valve Software are delayed at least once too. Google the concept of 'Valve Time'.
I agree with you, when you are polishing a product or making small improvements, it's much more predictable. Then something like that popping up is far more rare, and black swan like.
But that's not all development work. Rebuilds are especially treacherous, in my experience.
Oh, interesting. If there's a whole book with loads of projects that are called unpredictable, I'd say this is evidence for a rather 'predictable unpredictability'. Whatever causes 'Valve time' to happen, causes may range from a immature understanding of preserving options, to a lack of late commitment, and some dozens of other possible shortcomings or dysfunctions. Even a inability to create a workflow that is within reasonable bounds and not behaving chaotic. We'll never know. To attribute observed delays to a inevitable and inherent unpredictability of a flow of valuable work, is quite unlikely. Besides that, dropping some work items from the backlog and replacing them with others doesn't change anything. Oh, they had to drop already delivered work? Well, what happened to fast feedback? I see a lot of possible causes for the delays, none of which has to do with inherent unpredictability. Of course, when you start a multi-year project, a forecast of the latest delivery date with 90% certainty in the first week of the project would most likely make every investor pull the money out and flee. Just like noone can forecast with reasonable certainty (hm, say 85%?) whether we'll have white Christmas in January. If that is the predictability, you refer to, we agree.
You can predict the flow of product backlog items, that doesn't mean you can predict when the puzzle is solved.
When is a game fun or good enough?
When is a product good enough?
We focus on completion rate of features, but that's like the completion rate of words. When all the letters are on paper it does not mean the book is done.
That's why we have to watch out when we say it's predictable IMO, because it isn't. From the putting letters on paper it's pretty predictable, but from writing a book that achieves certain goals when a reader reads it that adds a whole another level of unpredictability.
When Toy Story story board was completely finished, and on time, Disney pulled the plug and told them to rewrite the whole damn thing.
You could learn something new when you deliver something, and that may have a huge impact on the remainder of the work.
That's what I'm talking about and why it is unpredictable. We cannot guarantee up front that what we do is exactly what will create and/or capture value, and as we go on that journey it may have a big impact on timelines because what's done isn't done.
And that's asides from technical surprises that frequently happen when working with legacy systems or doing rebuilds.
If you can plan and predict, there is no need to adapt.
If you can adapt, it does not mean you should not plan and predict but you have to realize you are placing your trust in an unreliable narrator.
How come Microsoft, Amazon and the like deliver relyably and predictably? How come whole companies like Ultimate Software manage 50 Teams using continuous forecasting, and deliver on time without any estimation? In the light of evidence that probabilistic forecasting works, it's hard to convince me, that it doesn't. Especially not with examples where some sponsor simply pulls the plug for whatever reason - a rare and impactful and single event. Of course one can leave the house and be struck by a lightning bolt. That doesn't mean you're facing something inherently unpredictable.