Maarten's Agile From First Principles Guide V0.8

How to Avoid the Agile Manifesto Confusion by Explaining Agile from First Principles

Jul 17, 2023

How do you explain Agile without referencing the Agile manifesto?

What is the essence of what Agile is about?

How do you talk about Agile without using the word Agile?

It’s easy to answer all of these questions when you’re capable of talking about Agile from first principles, without referencing the usual suspects like the Agile Manifesto.

Before we answer all of these questions at the end of this guide, let’s explore why answering them is necessary in the first place.

My book ‘Driving Value with Sprint Goals - Humble Plans, Exceptional Results’ is now available for pre-order at Amazon and Pearson. It’s the world’s first and only book on Sprint Goals. It covers topics mentioned in this guide like friction, humble planning and intent even more extensively.

1. Agile Manifesto: From Beacon of Clarity To Source of Confusion

The Agile Manifesto back in 2001 was a beacon of clarity and inspiration. We’ve come a long way since then. Due to the popularity of the Agile Manifesto, it didn’t take long for the consultants to latch on. The resulting Agile Industrial Complex flooded the market with certifications and waves of confusion.

Agile became a victim of its own success. Agile isn’t dead, but it did transform into a hydra-headed monstrosity. Ask ten Agile Coaches what ‘Agile’ means, and you will get ten different answers. Once a beacon of clarity and inspiration, it now has become a significant source of confusion.

Let me provide three concrete examples of how you can witness all of this confusion in the Agile community:

The echo chamber of silly discussions taking place in the Agile community, such as:
- ‘Being Agile’ vs. ‘Doing Agile’
- ‘Agile’ vs. ‘agile’ (yes, the difference is the capitalization)
- ‘Agility’ vs. ‘Agile’
- ‘Agility’ vs. ‘Business Agility’
- Is Scrum Agile or not?
The Agile Industrial Complex is humming at full speed to produce new certifications every month, with many of them contradicting each other. When you are asked the same question from the perspective of Scrum.org, Scaled Agile Framework, or SCRUMstudy, you will have to provide different answers that are often wholly incompatible with each other. Your allegiance is what determines the right answer, and there seems to be little common ground between them.
A giant Agile bubble is going on with so-called experts continuously being spawned. AI is lowering the barrier to entry to produce Agile articles, and poor quality content appears left, right, front, and center. The influx of fake experts results in even more confusion being spread by people who lack a basic understanding of Agile and Scrum.

Because of all of the confusion Agile is surrounded with, I try to avoid using the Agile Manifesto or the Agile label as a beachhead for establishing common understanding.

When you drop the Agile label in conversation, you have no clue what strings you’re pulling. You don’t know what the label evokes in the minds of your listeners. Those strings can pull the conversation in an entirely different direction than you had anticipated.

Many people share this sentiment. Here is a post by Peter Jetter with a comment from Allen Holub who more or less has given up on the word “Agile.”

I don’t want to sound pessimistic by pointing out these things. Let’s shift our attention toward solutions. How do we prevent our conversations from turning into one big Agile mush of noise and confusion?

I believe we can do that by trying to talk about Agile from first principles and removing the Agile manifesto entirely from the equation. It’s unnecessary and risky to reference the Agile Manifesto or Agile to make others understand what Agile is about.

Let’s start by exploring how we ended up in the Agile mush of noise and confusion.

1.1 The Birth of the Agile Manifesto

In 2001, 17 trailblazers in lightweight software development methods went skiing together in Snowbird, Utah. Together they created a manifesto describing the commonalities between their light approaches and dubbed the term ‘Agile’.

It’s crucial to understand the history of the Agile manifesto, because when you realize that the main purpose of the gathering was to provide a summary of what their lightweight approaches had in common, it becomes entirely clear what kind of questions the Agile manifesto was never intended to answer:

Why do we need Agile approaches?
In what kind of situations do we need Agile, and what kind of situations do we not need it?
If we conclude Agile is a good fit for our situation, how do we devise a way of working that is Agile?

Let’s focus on the positives. We stand on the shoulders of giants. The Agile Manifesto is awesome at capturing the spirit of what it means to be Agile. When you read it, you can grasp what Agile means and is supposed to look like in your teams. That doesn’t mean the Agile Manifesto tells you how you can achieve that.

The Agile Manifesto doesn’t answer when or why we need Agile. And if we realize we do need an Agile way of working, it doesn’t tell us how to realize it. The Agile manifesto provides a spirit of what an Agile way of working would look like.

The Agile Manifesto provides little guidance if you want to work in an Agile manner. And that’s problematic because many people try to reverse engineer what it means to be Agile from the Agile Manifesto, or they try to follow it as holy scripture to become Agile.

1.2 Reverse Engineering Agile From the Manifesto

Remember, since the Agile Manifesto provides a summary of what the different Agile approaches that existed back then had in common, it is focused on what the solution looks like and not why we need the solution.

Agile practitioners have supplemented the Agile Manifesto by trying to explain in more detail what’s the problem we’re trying to solve with it by invoking concepts like:

Cynefin framework
PDCA cycle
OODA loop
Stacey Matrix
VUCA

… and much more.

We’re trying to reverse engineer why Agile is necessary using complicated frameworks and explanations. The intent is good, because we do have to start somewhere. I believe we can start from even much older and more humble beginnings to explain why we need Agile approaches.

I believe the simplest explanation of Agile has its roots in military warfare. Understanding Agile begins with understanding friction and how it affects our plans, actions and results.

I want to stress, I don’t want to glorify war in any way. But there are important lessons we can learn from studying military warfare, as it is one of the ultimate exercises in dealing with friction.

Let’s examine why friction is crucial for understanding Agile.

2. Understanding Agile Begins With Friction

You have to start somewhere when you explain Agile from first principles, and the simplest starting point I know is the concept of friction.

Almost 200 years ago, General Carl von Clausewitz coined the concept of friction in his seminal book On War (1832) to explain the difficulties you face in battle. Let’s explore some quotes that illustrate the importance of friction and how it affects your plans, actions, and results.

Vom Kriege (1832) by General Carl von Clausewitz

“Friction is the concept that differentiates actual war from war on paper. Those surprising things that happen during wartime that make even the simplest thing difficult."
- General Carl von Clausewitz

Von Clausewitz posits that without friction, there would be no difference between war on paper and war in reality. This probably still sounds pretty abstract, let’s make it concrete:

“This enormous friction, which is not concentrated, as in mechanics, at a few points, is therefore everywhere brought into contact with chance, and thus facts take place upon which it was impossible to calculate, their chief origin being chance, As an instance of one such chance, take the weather. Here, the fog prevents the enemy from being discovered in time, a battery from firing at the right moment, a report from reaching the general; there, the rain prevents a battalion from arriving, another from reaching in right time, because, instead of three, it had to march perhaps eight hours; the cavalry from charging effectively because it is stuck fast in heavy ground.”
- General Carl von Clausewitz

The more friction we experience, the more surprises we will encounter. The enemy will be discovered too late due to the fog, the artillery will fire too late, or a report won’t reach the general changing the course of the battle. The more surprises we encounter, the more our plans must change to achieve the desired results.

Let’s explore the role friction plays on our plans, actions and results further by turning to the magnificent work of Stephen Bungay.

2.1 How Friction Affects Our Plans, Actions and Results

In 2010, Stephen Bungay published ‘The Art of Action’. The book examined the history of military warfare to extract valuable lessons we could apply to the world of business. Stephen Bungay identified the three gaps to show how friction affects our plans, actions and results.

The three gaps are:

The Knowledge Gap. The difference between what we’d like to know and what we actually know.
The Alignment Gap. The difference between what we want people to do and what they actually do.
The Effects Gap. The difference between what we expect our actions to achieve and what they actually achieve.

The more friction, the more surprises. The more surprises, the bigger the three gaps will be. The bigger the three gaps, the more our initial plans and actions must change to produce the desired results.

Let’s plot the amount of friction and together with our ability to make plans and execute actions to produce the desired results using the Cynefin framework:

When we experience no friction, what to do is Clear and obvious for anyone, no matter their expertise. There are no gaps to take into account. Anybody can create crystal-clear plans and actions to produce the desired results. We can even hand over plans to someone else to achieve the results we want.

As friction increases, what to do is no longer clear and obvious for anyone regardless of their expertise. Friction increases the gaps and what we’re doing becomes Complicated. We must enlist the help of experts to overcome these gaps. Without the right expertise, our plans and actions will encounter many surprises to produce the desired results.

When we put our plans and predictions in the hands of the right experts, we can make excellent plans with instructions to achieve the desired results. And we can even hand over plans to another group of experts so they execute and deliver the desired results.

As friction increases further, even experts begin to struggle. The three gaps increase to the extent that having expertise is no longer enough to make adequate plans and predictions. We’re entering the Complex domain.

The number of surprises increases, and predictability and our ability to make plans decreases even further. We still need expertise, but it’s no longer enough. Our success depends on our ability to quickly adjust our plans based on the surprises we encounter together with what we discover and learn.

As friction increases further, it becomes overbearing and we’re entering the realm of Chaos. Our focus shifts from making plans towards immediately taking action to try to get into a situation with less friction where we have a chance of being more in control.

2.2 Plan-driven Vs. Goal-driven Approaches

Why does this matter? Well, the level of friction we face determines the amount of surprises and the extent to which we can rely on planning and prediction. When what we’re doing is Clear or Complicated, plan-driven approaches work well. Just trust and follow the plan we’ve made, and life will be great.

As friction increases, we can no longer rely on our initial plans and predictions. When we enter the realm of Complex or Chaos, we have to rely on goal-driven approaches and make our goals explicit. We can no longer unconditionally trust our plans, and can only trust the results from our plans.

When plan-driven approaches work, where we can plan and predict while leaving the goal implicit and embedded in our plans, then we don’t need Agile approaches. The moment we can no longer trust our plans and have to rely on the difference between our observed and expected results to determine how much we we can trust our plans, that’s the moment we’re entering the realm where Agile shines and becomes necessary.

What should we do when we can no longer trust our plans and actions to produce the desired results?

2.3 As Friction Increases Trusted Best-Practices Turn Into Anti-Patterns

When we’re facing no or limited friction, given the right level of expertise, we can make accurate plans and predictions. Viewed through the lens of the three gaps, the best approach to use looks as follows:

When we know less than we’d like, we should spend more time analyzing and preparing. When people don’t execute the plan as expected, we should give more and better instructions. When our plans and actions don’t produce the desired results, we should add controls and checks so that the best course of action can be immediately executed.

Now, imagine as friction increases, and we enter the realm of Complex and we try to follow these same best-practices. What do you think will happen?

The best-practices for work that we can reasonably plan and predict, actually become anti-patterns as friction increases. Because we can’t plan and predict, these best-practices introduce a fog of speculation that clouds our ability to execute on our plans and actions to produce the desired results.

Visualized in another way:

In short, our usual and familiar approaches stop working. What works for Complicated doesn’t work for Complex and only makes matters worse. When companies don’t notice this is happening, they often end up in the Planning Cycle of Madness.

2.4 The Planning Cycle of Madness

When we don’t notice our trusted approaches are failing, then we get stuck in the Planning Cycle of Madness that many companies suffer from.

Let me walk you through the Planning Cycle of Madness. At some point, we fail to meet our plans, roadmaps, and timelines. The leadership team is angry and disappointed. We must do better the next time! We have to find a way to make our predictions and plans better.

Our default response is to spend more time planning, analyzing, designing, mapping dependencies, and coordinating the work up-front. Our plans look magnificent and astonish everyone, but upon closer inspection, we signed our own planning death sentence.

In our overzealous effort to blow everyone away with our plans, we injected noise and speculation. Our plans look great, but they are now disconnected from reality and mostly rooted in our imagination.

Then, when reality turns out to be different than what we’ve planned, we become paralyzed. Our plans act as an anchor that stifles the ability to collaborate and adapt. We’re so locked into our plans that they drag us down and prevent the collaboration, learning, and discovery that’s necessary to achieve our objectives.

As a result, we fail to meet our plans once again, and we try to do a better job at planning and predicting the next time - and we fail once again. And so the Planning Cycle of Madness repeats, often ad infinitum.

The Planning Cycle of Madness is a clear sign you’re applying plan-driven approaches in a context where you need goal-driven approaches.

To hammer the point home, let’s relate friction back to the Agile manifesto.

3. How Does Friction Relate to the Agile Manifesto?

We talked about the best-practices and anti-patterns for dealing with friction in the Complicated realm. We also talked about the anti-patterns for dealing with friction in the Complex realm.

What we did not talk about are the best-practices for dealing with friction when you’re doing Complex work.

The best-practice when we know less than we’d like to know is to clearly communicate intent: what are we trying to achieve, and why does it matter? When we want to people to do what is expected of them, let them define the plans and actions, and report them back to the higher levels. When our actions don’t pan out as expected, let the people that do the work adjust plans and actions as necessary in line with the intent of the original plan.

Intent is what makes goal-driven approaches possible. When people understand what we’re trying to achieve, and why it matters, they can adjust plans and actions to achieve the desired results in line with the original intent.

Now let’s relate all of this back to the Agile manifesto:

All anti-patterns for dealing with friction can be directly related to the Agile manifesto. The left-hand side contains best-practices for dealing with friction.

Now let’s go back to the initial questions this article set out tot answer.

3.1 How Do You Explain Agile Without Referencing the Agile Manifesto?

By talking about the best-practices and anti-patterns for dealing with friction, we’re talking about the same things as in the Agile manifesto. Except now we have a much deeper and richer understanding why the statements of the Agile Manifesto work and are necessary.

Friction, the resulting number of surprises and our inability to plan and predict to achieve the desired results are what drives the need for an Agile approach. If we can plan and predict, and there are few surprises, then we don’t need an Agile approach. A plan-driven approach is just fine.

3.2 What Is the Essence of What Agile Is About?

Agile is about dealing with friction.

It's about learning and discovering surprises- working with what we do know to discover what we don't know.

Our plans will be imperfect, execution flawed and the results unpredictable. Nothing we will do will prevent we will have to deal with surprises and encounter unexpected things.

3.3 How do you talk about Agile without using the word Agile?

When we can’t adequately plan and predict to achieve the desired results, we should switch towards goal-driven approaches. We should start with humble plans that are adjusted as we learn and discover what’s necessary while we do the work.

When you work with Scrum, it all starts by working with Sprint Goals. Without Sprint Goals, it’s not possible to follow goal-driven approaches that supply intent where the plans are adjusted as necessary to achieve the desired results.

“There are no steps for you to follow.

Every step you take helps shape the way.

You must go where there is no path and leave a trail.”

- Introduction of Driving Value with Sprint Goals by Maarten Dalmijn

4. Summary

When you can plan and predict, with or without requiring special expertise, you don’t need Agile.
Agile from first principles starts with understanding friction. As friction increases, the three gaps become bigger and at some point expertise is no longer sufficient to be able to plan and predict adequately. That’s where an Agile way of working enters the picture.
More friction, means bigger gaps and more surprises. More surprises mean our initial plans, actions and results will have to change as we do the work to achieve the desired objectives.
As friction increases and experts can no longer plan and predict accurately, then we should switch to goal-driven approaches. We should leverage humble planning together with intent to allow teams to adjust plans as they discover what’s necessary while they do the work.