A Ledger is 3 Products in 1
Finding out where the real cause of your ledger's "bugs" really is
A couple of days ago, in one of those meetings that are going nowhere, someone casually coined one of the most wonderful terms I’ve ever heard: the root-root cause.
Not root cause, no. Double-root. The root of the root cause.
It was a moment as memorable to me as it was to all those journalists who were in the room when US Secretary of Defense Donald Rumsfeld first talk about unknown unknowns.
It was corporatespeak in the making.
I’m half joking. The term “root-root cause” is superficially absurd: if there is a root to a root cause, then it wasn’t a root cause to begin with. But on the other hand, I’ve witnessed problems caused by an underlying bug that was caused by something even deeper. Problems whose actual cause wasn’t slopiness, carelessness or a misunderstanding of the mechanicals of the problem, even if they were looked like, and addressed as, such.
Problems that were caused by something more fundamental. A root-root cause.
Bugs in ledgers, like in any other software system, are usually treated as engineering problems. When ledgers fail, they do so because they’re slow or because the calculations are off. And when engineers inspect them, they search for performance bottlenecks, off-by-one errors, or the use of floats to represent money.
That’s one way to look at ledgers’ problems. But it’s incomplete. When you look at ledgers as if they were simply technical problems to solve, you’re overlooking the fact that they’re a critical piece of business technology for fintechs. Which means that there are multiple areas of the company using the ledger in fundamentally different ways.
“Fixing” a bug may involve a tradeoff that benefits one of those areas at the expense of the others. And, in time, the other areas will notice how their way of interacting with the ledger has gotten worse, and will demand it “goes back to how it was behaving before.”
Spice this up with two different engineers looking at this problem at different points in time, and you’re going back and forth between doing and undoing the “fix”, never going anywhere.
Because none of them are really seeing the root-root cause.
I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.
But you don’t want to get a job; you already have one. What you want is to learn how to be great at your job. Especially as a payments engineer, where stakes are sky-high, and the margin for error is razor-thin.
In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful, and more successful payments engineer. And we do that by cutting off one sliver of it and extracting tactics from it.
In today’s article, we’re looking at the 3 products subsumed in a ledger system, their conflicting requirements, and some of the “scalability” or “performance” issues that arise from it. It’s one of those topics that makes it clear that being an engineer takes a special kind of courage, the courage to admit that you don’t know everything, and the courage to tell others that they don’t either.
Here’s what you can expect in today’s article:
The 3 root-root causes of all of ledger’s problems
What you will be forced to do if you ignore this for too long
Enough intro, let’s dive in.
