A Brief Explanation of Why Processes Don’t Scale

I received a comment on an earlier post from Guido Schneider taking some exception to a Scrum example I used. His thoughtful response got me thinking, and the thinking turned my response to his comment into a whole blog post.

Guido makes the point that in the real world, you often have Scrum teams that are part of larger projects. The result is that we try to scale our processes by creating things like scrum-of-scrums (essentially doubling-down on the practice). We need to create high level iteration plans that look at hundreds of work items across multiple Scrum teams. Sometimes we need to roll up utilization numbers and progress through WI hierarchies, for example. The current Scrum template in RTC doesn’t solve all of these scrum-of-scrum, project, or portfolio level issues.

One reason for this is because Scrum isn’t a project level practice, it’s a team practice (as Guido indicates). But organizations often try to scale Agile techniques like Scrum to larger teams and projects, placing a demand on an RTC Scrum template to do more than just Scrum.

Is it Possible to Scale a Process to a Larger Team Size?

So my concern is, how much can Scrum (or other Agile process) be scaled to larger organizations? Are we just doing what we’ve always done with rolling up the project management information, giving it a new name (Agile), and expecting different results? What evidence is there that you can scale any or all Agile techniques?

Interestingly, Agile was not initially defined as a set of techniques that only small teams can use. But almost all the work that’s been done inventing new Agile techniques focuses on the work of small teams. For example, Scrum uses developers and product owners, not project managers, to prioritize change requests. But larger projects make the project manager responsible for prioritization.

I believe this is because it’s easier to fulfill the Agile Manifesto with a small team. For example, face-to-face interaction is much easier, and gives you much more productivity, on a small co-located team then it does on a large distributed team. More documentation/artifacts are required for that larger team because 20 people can’t have face-to-face communication every day.

When you look at scaling a practice to create scrum-of-scrums, are we really scaling Agile? My experience in recent years is that most Daily Scrums are not actual Scrum meetings. They are status meetings with free-flowing discussion. They don’t keep to strict questions about what was or will be completed. Not only is this not Agile (as defined by Scrum), it’s not scalable. When you get to a scrum-of-scrums, you probably have a worse situation since those Daily Scrums will be filled with project managers who are used to discussing status and roadblocks instead of keeping to a Scrum script.

It’s frustrating, because one of my favorite parts of Scrum is the efficient and effective Daily Scrum format.

Agile Isn’t Everything. Neither is Anything Else

I’m not arguing here that we don’t need to roll up estimates and amount of work completed. I’m making the argument that these aren’t necessarily Agile techniques. They’re certainly not traditional Scrum techniques. And we should consider the cost/benefit of using a practice like Scrum (or any other type of process) when we need to make decisions in a way that is not Scrum-like.

For example, the Agile Manifesto says we prefer responding to change over following a plan. That’s an excellent perspective, but it’s a harder perspective to execute on when you have a large project with lots of moving parts. Customers need delivery dates, dependent teams need to accept new code changes, payment is only made after certain functionality is delivered and the business is counting on that money, etc. Four people working in adjacent cubes can coordinate with quick conversations. Larger projects gotta have a plan to follow or they sink under their own weight.

The Agile Scaling Trap

I assert that there are techniques that make you more effective on a small team that don’t scale to a larger team. The techniques lose their effectiveness in larger teams. The dynamics of larger teams are different, not just bigger. There are things you have to worry about with a large team that you don’t need to worry about with a small team.

What’s a small team? Surprisingly there’s no good research on that in our industry, last I checked. But based on some research with entrepreneurs, and using the n(n-1) formula that illustrates the Law of Diminishing Returns, I believe that a small team is no more than 5 people. More than that, and the communication overhead for each new person on the team increases geometrically. It’s also easier to “hide out” and do less work, which is why I join the largest teams I can find.

And the team needs to be relatively isolated. That is, it has few dependencies on other teams and other teams have few dependencies on it. As you depend more on other small teams, the meaning of “team” changes and you find yourself part of a larger team.

So moving from a small to medium sized team involves more than just adding new people. Everyone needs to communicate with each other differently, and the overhead increases dramatically. You can no longer be successful doing the same thing you did when the team had 3 people on it.

By the way, this doesn’t mean you can’t use Agile techniques in larger organizations. You absolutely can. For example, Story Points is something that an organization of almost any size can use. But we need to be wise about what we use and what can be scaled to larger teams.

Adapt and Overcome

So, what’s this all mean? Guido was making the point that at the project level, you still need to deal with lots of planning work items because you have multiple teams on your project. He’s correct. But looking at a WBS rollup is not Scrum, and it’s not even an Agile practice. It was hard to manage this kind of information 30 years ago, it’s hard to manage it now, and it’s going to be hard tomorrow. Tools make it a lot easier, and tools continue to get better. But it’s not a surprise that total automation eludes us.

We still need to adapt processes to our circumstances, history, and goals. Often, adaptations force us to change the way we view or interpret information, which makes us go back and re-think the complex ways we need to evaluate the progress of our teams and projects. It’s no wonder we haven’t come up with a one-size-fits-all process or project management schema.

We still need to understand our own organizational needs and goals, which means we still need to understand how practices (like Scrum, Story Points, etc) give us value so we can choose wisely, adapt appropriately, and get the most value for the energy we put into defining our processes.

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

2 Responses to A Brief Explanation of Why Processes Don’t Scale

  1. I’m dying to see a future post on how you’ve seen Story Points used effectively at scale. I’m not sure there is an agile practice people are more confused about. I was hoping to see a discussion of SAFe or DAD in terms of adapting agile to larger formats, finding a way to scale the good stuff while dealing with the other stuff.

  2. Guido Schneider says:

    Good comment, Millard. Regarding Story Points and scaling, I see in our company the tendence to define a “What is one” criteria in terms of hours. I’m not very happy with this but do not have good answers yet.
    Because according SCRUM a story point is a team internal definition defining the complexity of a reference story. But how can Project Leaders on bigger projects do forecasting, reporting etc. if each of the team has another reference story? Yes, you can now argue, why does a Project Leader needs a forecast. With SCRUM you just develop until you have enough functionality and then you go onto market.
    Here we have the dilemma where processes don’t scale. SCRUM is a great team process. But the team is very seldom alone on the planet. In a larger organisation you have 5 or more teams working on same project. On higher level you need some traditional project management practices. But this practices must be feeded by the teams with figures. But the teams you want to work as much as possible self organised and agile.
    Back to the beginning, “What is a Story Point” in a large organisation? How are other enterprises deal with this challange of team process vs. project management in an enterprise?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s