“Why are we still releasing new features so slowly?” I was thinking about this a year after joining a fast-paced start-up. Every new feature was taking painstakingly long.
This was not what I expected. Start-ups should move fast. Especially if you are before product/market fit and still bootstrapping.
When I joined we had 3 development teams. One year and a multi-million seed funding round later, we had 9 development teams at our disposal. We tripled our development capacity, but new features were being released at the same pace. What was going on?
I hypothesized we were suffering from Brooks’ law. Brooks’ law states that adding more software developers to a late project makes it even later. We hired a lot of amazing developers who still needed to ramp up. As long as they are still getting up to speed they would be slowing other developers down.
Half a year later, we still were not delivering new features any faster. So Brooks’ law no longer was a probable explanation. There had to be something else going on, but what could it be?
Then we suffered a production issue that would open my eyes to the likely culprit.
Lessons learned from troubleshooting a production issue
We had just released a new feature for filtering on tags. Support received complaints the new feature was not working.
First thing I did was to upload a new picture myself and tag it with ‘Not Hotdog’. I tried to filter on ‘Not Hotdog’ and I confirmed it was not working.
I then made a list of all the teams involved in building the feature. Each team was responsible for a specific component:
- Uploads. Responsible for all back-end functionality related to uploading and processing images.
- Elastic search. Responsible for all functionality related to elastic search.
- Photo album front-end. Responsible for front-end of displaying photo albums and images.
- DevOps team. Responsible for the infrastructure and deploying new features in the cloud.
I decided to talk to the Uploads team first. I asked the lead developer why their feature was not working. He ran some checks and concluded the tags were saved on upload. He suspects the tags are not indexed and directed me to the Elastic Search team.
The developer from the Elastic Search team peeks in Elastic Search. The ‘Not Hotdog’ tag I added was indexed. Everything in Elastic Search looks great. So he directed me to the Photo album display team. It must be something in the front-end and it is not our problem. I began to get annoyed as this is the second time I was being redirected.
The front-end team discovers a front-end library has been deployed with an older version. The new filtering functionality depends on this library so this probably is the origin of the production issue. The front-end team advises me to talk to the DevOps team to get the right front-end library deployed. Tired of being redirected, I tell the front-end team they should talk to DevOps team instead and get it fixed. Finally, the issue gets resolved.
After these inefficient interactions, I realized the teams lacked ownership over what they were building. Our teams were organized around components. No team owned features, they just owned small parts of the whole puzzle.
The component team structure made it hard for the teams to deliver new features fast. All their efforts were tightly coupled. If one part of the four teams would be not working or delayed, then this would affect the delivery of the whole feature.
So why do companies start using component teams?
When product development starts, development teams often self-organize in component teams. When a product is vague, at least the technical components making it work are clear. It is easy to create your teams based on clear components.
The idea behind component teams is simple. Assign system components to teams. Each team is responsible for just one or more components.
Component teams help maximize the output of your developers by having them only work on specific parts of the system. Building something new in Amazon RedShift? Only a single team needs to worry about gaining experience with RedShift.
The downside of component teams is dependency management needs to be handled on the feature level. Features spanning component team boundaries immediately generate dependencies requiring active management. This is only manageable when you have few components or your features do not span a lot of component team boundaries.
Imagine you are a Product Owner and want to pick up ‘Feature 2’ in the picture below. Feature 2 depends on two components assigned to different teams. Feature 2 can only go live when two different teams have completed the necessary work on their components to make the feature work.
To go back to the filtering on tags example. Just as it was hard to troubleshoot the feature, imagine how ineffective it was to have all these different teams coordinate to deliver just such a small feature!
The uploads team made it possible to add tags when uploading. The saved tags then needed to be indexed by Elastic Search, so it becomes possible to filter on them. Then the Photo album front-end team would need to add the front-end that allows you to filter on tags from the user interface. The work of all four teams came together and the feature deployed.
In short, all four teams needed to coordinate to get the new feature live, as each was responsible for a component. Any delay in a component, delays the release of the feature. The more components and teams involved, the more coordination problems and unnecessary waiting can arise. If just one sprint of one team fails, the feature cannot be released.
The alternative: feature teams
Instead of making development teams responsible for components, you can make teams responsible for features. When using feature teams you assign one or more features to a team, instead of components.
Imagine you would have single team responsible for filtering & searching and another team responsible for adding metadata to pictures. When delivering the filtering on tags feature you suddenly have only two teams that need to coordinate their efforts.
What makes feature teams hard is that ownership of components is shared. Multiple teams may be working on the delivery of a new feature requiring the changes in the same components. The coordination efforts are moved from the feature level to the component level.
Managing dependencies on the feature level is hard, but easier than doing it on the component level. You need to make sure changes to components are reviewed by the people in the company who know the most about those components. There also needs to be adequate knowledge sharing between teams about all components.
Switch to feature teams when component teams hurt your development speed
Most companies start with component teams, because when the product is unclear at least the components are clear. At some point component teams might start preventing you from moving fast.
There are clear symptoms when your component team structure might be slowing you down:
- Small features take much longer to develop than expected, because they cross components owned by many different teams.
- Finger-pointing when there are production issues. Nobody owns any feature.
If this happens, you might consider switching to feature teams. Working with feature teams presents it’s own unique challenges. You need to figure out how to best share multiple components between multiple teams. Knowledge transfer between all teams is often required. It might also be necessary to adjust your architecture to decouple all different components as much as possible.
Actually one of the hardest part of working with feature teams is actually switching over to feature teams. How do you structure the teams? How do you transfer knowledge? How do you make sure you get buy-in from the organisation to change the teams?
I will follow-up with another article how you can make the switch to feature teams.
Update: 23rd of May 2018
Added link to follow-up article explaining how to roll out feature teams.