A Problem-First Approach To Building Engineering Teams
Companies Split Engineering Teams By Technology Because They Assume That Software Engineering Is About Writing Code. It's Not.
Editor’s note: My talk on How to build payment applications in Python is now live on Youtube!
How should engineering teams in tech companies be split?
Many people would say by the technology they use. That’s what most of us are used to: Backend on the right, Frontend on the left. But I think we can aim for something more ambitious: that tech companies should be split into teams by the problem they solve.
It is extremely difficult to meet deadlines. And I think that’s because information isn’t being channeled correctly. Expertise isn’t accumulating where it should, and feedback transmission is lossy.
That could be because team boundaries are misplaced. We’re having too many cross-team meetings for tiny problems.
In Designing Payment Applications, I made the observation that successful software teams are given enough and high quality context to do their work:
Savvy CEOs don’t veto technology they don’t necessarily understand. Instead, they provide access to the multiple and overlapping contexts on which that technology operates. It’s also access that engineers most often don’t have. Who are the stakeholders? What can they do with that technology? How is the company positioned in the market? How does it make money? Who is the user they cater to?
Though correct, I missed the opportunity to address another communication problem: what if the necessary context is something much more tacit, beyond what the CEO needs to be concerned with?
I’ve attended cross-team meetings where the biggest obstacle to overcome is how two teams have attempted to solve the same problem independently and have implemented two solutions so slightly different, but incompatible.
Understand: The most useful insight within tech companies is passed through informal channels.
Spreading that information is the unspoken role of Staff Engineers.
Welcome to Money In Transit, the newsletter bridging the gap between payments strategy and execution. I’m Alvaro Duran.
Recently, we’ve looked at
The foundations of an open source payment orchestration library
Why payments must be built in-house, and how to do it effectively
They’re all free to read.
Want to be notified when there’s a new post? You know what you have to do.
Unlike other engineers, Staff Engineers are tasked with the preservation and dissemination of the context and cultural cues that engineering teams need to do their job.
Until very recently, I thought that this was an impossible task.
Over the years, I’ve realized that programming isn’t about producing code. It’s about developing a tacit understanding of what’s the system ought to do, and how best to modify it.
See, in today’s world, tech companies are built “as a Service”. Software modifications on running systems is all that engineers do. The codebase as it is now is much less valuable than the ability to enhance it.
That is, programming should be seen as building a theory.
And because it’s a know-how, Staff Engineers can’t transmit it completely or accurately. They “just know”.
That may explain why Staff Engineers are very often just good engineers who were part of the team that built the legacy systems. They just accumulated that know-how. They’re Inuits in Greenland.
Can you see the problem? Many engineers simply don’t stay close to the problem long enough to develop a theory. Most often, they jump from one domain to another, unable to dive deep into anything but the technology they use.
Most engineers are technology-centered. But domain-centered engineers are the ones that make it to Staff.
Companies don’t necessarily know any better. They could be hiring engineers who have solved similar problems before. But instead write job descriptions that tend to say things “$INDUSTRY experience is a plus, but not required”.
High turnover and shallow domain expertise. It’s not surprising, then, that engineering teams go through a never ending cycle of attempts at replacing legacy systems. By the time the “replacement” project ends, the early team members are gone, and those on the team have a shallow understanding of the problem.
I believe engineering teams would have a much better chance at succeeding if they were split by the problem they’re solving, and not the technology they’re using.
If Staff Engineers are tasked with focusing on a single problem, rather than communicating bits of several ones across the org chart, they would be able to develop the expertise the company needs.
They narrow their sphere of observation, so that…
Are exposed to more nuanced decisions, so that…
They can iterate and enhance the system without replacing it.
You might have heard of Domain Driven Design. I’m not referring to that. DDD is the codification of tactics with the purpose of building internally consistent software. And that is a good first step to building expertise. But what companies are unknowingly starving for is the domain expertise itself.
What I want you to consider instead the possibility that your “backend” and “frontend” split may be harmful.
Businesses are developing more software in-house than ever before. AI has made it cheaper to get software tailored to your use cases rather than having to rely on SaaS solutions.
It made sense to split your engineering teams by technology when the obstacle was learning how to use it. But now, the problem at hand is the bottleneck, not the technology.
Make your engineers experts on the problem. Who knows? Maybe they find a way to solve it.