Uncategorized – Karney Li

Cross-Functional Squads

A few years ago, Spotify popularized the idea of organizing engineering teams into squads and guilds. This isn’t actually a new concept, it’s an old one given a new name; Amazon has been organizing development teams into two-pizza teams along the same principles at least as far back as the early 2000s. Organizational design is something that most engineering managers need to tackle with at various points in their careers.

When teams are small, they often index for hiring full-stack developers (i.e. jack-of-all trades developers). This often serves them well, especially considering when all they can afford is 2-3 developers. As teams grow, individuals tend to specialize. You’d be really hard pressed to find a developer that’s good at mobile, web, backend, distributed systems, devops, data engineering, systems engineering, etc. I’ve never actually met a developer that was actually excellent across all these engineering areas. I’ve met some that were fearless and willing to dive in and learn, but in most cases, they’re just average in the areas that aren’t their main areas of expertise.

The purpose of a cross-functional team is to outfit the team with the right tools to complete the job without the need to depend on outside help. The more a team needs to collaborate and coordinate externally the slower they get. Decoupling teams gives the team autonomy and the ability to own the situation.

Analogy time! Let’s take a basketball team, everyone on the team should be able to dribble, rebound and perform layups, these are basic skills. But individual players will have their areas of expertise, some will be excellent 3pt shooers, some excellent at posting-up under the net because of their size, and others great at play-making because of their speed/agility. A team with 5 centers in the starting lineup would likely be a pretty bad team, they’d be slow to move down the court, have their passes constantly intercepted, and balls stolen whenever they dribbled against a smaller more agile player. Each person on a team is filling a position, a good team is a team of complementary players that help each other be effective at what they do best.

We want developers that in a pinch could debug a web form or look at why an API is suddenly slow, this isn’t necessarily a full-stack developer, that’s just a willingness to do something when called upon, a lot of it just basic debugging skills and understanding of how software works. In a pinch, a center can shoot a 3-pointer, but you probably don’t want them to do that all the time; if you want your teams to be effective, get folks to focus on what they do best.

Deconstructing Agile

To really understand something, we need to read beyond the surface, beyond the rituals and understand the underlying motivations. The Agile Manifesto for developing software isn’t prescriptive about methodologies. The manifesto only has four points:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

There are many methodologies that teams can adapt to develop software in an Agile manner, Scrum, XP and Lean are amongst some of the more popular methods. Often teams adopt practices they like with varying degrees of success.

Software development is often over-complicated. The goal of creating software is to build something that people want. To determine what people want, we need to get customer feedback. Time spent creating things that the customer doesn’t value are a waste of time and effort.

The whole point of Agile is to release software more frequently because the more frequent software is released the sooner we can get customer feedback.

The objective of Agile is to optimize for customer feedback.

Without loss of generality, Scrum is really a compressed waterfall, one compressed into a 1 to 2-week “sprint”. When time is compressed, it puts constraints on a team, this is by design. By constraining the amount of time allotted to sprint planning, the team is forced to find other ways of describing and estimating work. They no longer have time to collect detailed requirements, write comprehensive design docs, or come up with detailed work plans and effort-estimates. The team is forced to optimize for what is deemed high-value for the customer. At the end of the sprint, the team gets feedback from the customer. At worse, they will have worked on something for 2-weeks that the customer doesn’t want, they can then pivot the very next sprint. This is pretty good, it’s unequivocally better than working on something for months or years only have the customer reject it.

If you try to optimize for customer feedback, you’ll notice that there’s still a potential 2-weeks worth of wasted time. The thing that gates customer feedback is a team’s frequency of releasing software. By releasing even more frequently, the team can get feedback even sooner. The feedback enables customer collaboration, a key value of Agile.

Lean can push this to the extreme. Assume developers are expected to deploy to production what they started on that morning. Now instead of releasing only at the end of a sprint, they release changes every day so that customer feedback can be solicited even sooner. This constraint means the changes deployed to production need to be small, small changes also limit deployment risk. Instead of a worst-case scenario of wasting 2-weeks, the worst case only a wasted day.

Doing stand-ups, sprint planning, kanban, kaizens, demos, velocity tracking, retrospectives, are just rituals, they don’t necessarily make your team Agile. I find many of these so-called Agile rituals downright wasteful. Understand the rationale behind the rituals and what to optimize for and it will lead to appreciation and enlightenment.

Thinking Distributed

The best analogy I can think of for how to think about scaling an organization to think distributed is the contrast between communism and capitalism methods of government. Objectively speaking, one is not necessarily better than the other, but in practice one scales better in our current world.

Communism is centralized command-and-control, it is a non-distributed organization. In theory, if you had perfect knowledge and ability to react in real-time, there would be minimal waste and demands would always be met with the right amount of supply. In practice, it hasn’t played out this way, probably because we don’t live in a world of perfect data and instantenous data processing abilities. Capitalism in contrast, relies on that invisible hand, it argues that in efficient markets, supply-and-demand will balance itself through price discovery. There might be some degree of loss in the beginning but it will come to an equilibrium fast and the overall oversupply or undersupply will be minimized.

In capitalism, when the government wants people to spend money, they lower interest rates. Instead of directing, they guide. They create incentives for the market to move in the desired direction but they don’t direct. In communism, there’s a required a feedback loop to central command so it can process the data and tell each type of producer the next steps. This often leads to either over-production or under-production because it’s easy to have incomplete information or data processing delay.

Leading a distributed organization is similar to working with monetary policy. We want to create the right incentives, we want the default thing to be the obvious thing. We want to build the necessary tools and safety-nets so that people can focus on their objectives and not worry about duplicating work. We want teams to operate independently, instead of relying on upper-management (central-command) for next steps.

Leadership cannot scale to have perfect knowledge of everything teams do, otherwise the leadership team ends up having massive reporting apparatus just to track work output and teams will end up idle waiting for the next instructions. Instead leadership needs to empower teams to want to act in their own best interests because those interests. Leadership can guide teams by helping them clearly define KPIs, but how they go about achieving those KPIs should be left in their hands, doing anything differently will only disempower.

Disagree and commit

We live in a world of inter-subjective realities. Humans created religions, economic models, monetary systems, government, corporations, and moral values. None of these things exist in nature, they were created from our imagination and we applied human value to them.

When a person joins a new company, they will often discover that the company has a set of values. These values were chosen by the leaders of the company because they want these values to guide the company’s actions. These values are frequently repeated and made highly visible to keep people aligned. The company’s values might actually differ from an employee’s own personal values, this leads to some degree of cognitive dissonance. The employee may disagree with some of the values, but the company doesn’t need them to agree but to commit to it.

Humans are masters of overcoming cognitive dissonance. When someone converts from one religion to another, seldom do they buy-in 100%. There are likely beliefs, values, or rituals they don’t agree with or understand, but they still converted to this new religion and go through th rituals because they’ve committed to doing so.

Imagine someone immigrating to a new country. This person might immigrate because they like the way of life, the natural landscape, the employment opportunities, the people or the social benefits. They might not agree with the taxation system and how that money is spent (cognitive dissonance) but the government still expects them to commit to paying their taxes.

As a leader, it’s important to recognize the difference between personal emotional comfort vs operational need. We often want people to agree with our values when it is more imperative that we get people to commit and deliver.