Uber has rewritten its ledger systems five times in the last ten years. And at least one of those rewrites, if not all, could have been avoided.

That’s because the root of each generation of money software at Uber was driven from bad incentives. Each started with a brand new proposal, approved as the definitive solution; in time, a fatal flaw was surfaced; and finally, a new proposal came along to replace it.

Every rewrite was someone’s promotion project.

At least one of them could’ve been avoided: the one where Uber moved to DynamoDB. In 2017, Uber launched their new payment platform on it, and the critical factor that everyone involved seemed to miss was that DynamoDB is a consumption-priced database.

You pay for every read, and every write.

With each trip generating multiple ledger entries, and Uber as a whole processing 15 million trips per day, it didn’t matter that DynamoDB was great because of high throughput at global scale. The proverbial bean counter should’ve stopped this madness from happening.

Within 2 years, the cost became prohibitely expensive:

At Uber’s scale, DynamoDB became expensive. Hence, we started keeping only 12 weeks of data (i.e., hot data) in DynamoDB and started using Uber’s blobstore, TerraBlob, for older data (i.e., cold data). TerraBlob is similar to AWS S3. For a long-term solution, we wanted to use LSG.

— Migrating a Trillion Entries of Uber’s Ledger Data from DynamoDB to LedgerStore

A redesign that gets replaced 2 years later is a catastrophe.

And yet, history remembers Uber’s ledger on top of DynamoDB as a masterpiece. As late as 2024, ByteByteGo has an article praising it.

And that’s what concerns me. Uber’s design was a failure, but nobody seems to remember it that way.

That ends today.

I’m Alvaro Duran, and this is The Payments Engineer Playbook, the only newsletter on Earth tailor-made for engineers of money software. Every week, more than 2,000 subscribers from companies like Stripe, Coinbase and Modern Treasury get a dive deep on how to build software that moves money around. Not to pass interviews, but to do their job exceptionally well.

When money is on the line, stakes are sky high and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

Here’s what you can expect in today’s article:

• Why DynamoDB works for payments but breaks when you use it as a ledger

• The napkin math that would have saved Uber millions of dollars

• And one shocking conclusion from all of this

Enough intro, let’s dive in.

The Consistency Trap

But first: is DynamoDB a bad choice for financial software?

Not necessarily. I’ve already covered DynamoDB as a potential data store for payment systems, and it has many features that are worth it: zero-downtime migrations, low latency for a global audience, and built-in replication and failover.

If you’re accepting payments at scale around the globe, DynamoDB is a great choice.

It is because DynamoDB, while not enforcing full linearizability across partitions, can guarantee consistency within a Region. DynamoDB gives you strong consistency on a per-partition basis, but not across partitions — and for a global-scale payments system, that’s a trade-off worth making.

PostgreSQL can; DynamoDB cannot.

This is quite a property when it comes to payments, because they are independent from each other. You can interleave the authentication of one with the capture of another. There’s no need to maintain full linearizability across your data; causal consistency is enough. DynamoDB trades off the linearizability that you don’t actually need for all those nice features I mentioned earlier, which means that for large enterprises that serve customers all over the world in high volume and frequency, DynamoDB is better than PostgreSQL.

But a ledger isn’t a payments system.

A ledger cannot simply say “hey, this account and that account can be dealt with independently”. The scope of a ledger system is The World; a data store that can’t enforce full linearizability isn’t going to cut it, no matter how good at throughput and latency it is.

In other words: DynamoDB works well in payments because payments can give up global consistency for better availability. But ledgers can’t give up global consistency, even if that means they get worse availability.

The Napkin Math (Because Someone Has To Do It At Some Point)

DynamoDB capacity pricing is based on two main models: Provisioned and On-demand. You can buy reads and writes in bulk, buy reads and writes when you need them, or both.

And then, there’s the storage and add-on features. For most, throughput is the real deal, and storage is just the cream on top of the invoice. But for data-heavy applications, such as ledgers, storage can become the dominant cost. That’s why reserving capacity is important, but it demands from you some predictive abilities, or at least making an educated guess about how much you’re going to need in advance, because you’re going to get a discount of more than 50 percent if you do it right.

If you use DynamoDB at scale, you need to do some napkin math.

In 2017, Uber was doing around 11 million trips a day. Assuming 10 entries per trip and 5 WCUs per entry1, that’s 550 million writes per day, and at $1.25 per million writes, that’s $687 per day.

$687 per day doesn’t sound like a lot. But that’s $250K a year, just in writes.

With 3x annual growth, the math is unsustainable: by year 3, we’re talking $2.25 million a year. I don’t have visibility into reads, indexes and global tables, but at Uber’s scale, the read side likely costs as much as the write side.

Which means that Uber was burning 5 million dollars in a freaking ledger.

Based on Uber’s data, by 2020 they had accumulated 1.2 petabytes of data. That, at $0.25 per gigabyte is 300K per month. And assuming the same 3x annual growth from 2017 to 2020 with a final size of 1.2 petabytes, that’s a cumulative cost of $3.5 million.

No wonder they switched to storing only the last 12 weeks of data in Dynamo, and stored the rest on premises.

Add the writes to the storage, and you’re looking at an 8 million dollar bill for a ledger that didn’t need to exist on DynamoDB in the first place.

Somebody Should Have Been Fired For This

What do you do with an 8 million dollar bill? You turn it into a case study.

Since 2020, Uber has migrated away from DynamoDB into their own internal ledger, called LSG (Ledger Store...Gateway?2), built on top of their own internal distributed database called DocStore:

Docstore is a general-purpose multi-model database that provides a strict serializability consistency model on a partition level and can scale horizontally to serve high volume workloads. Features such as Transaction, Materialized View, Associations, and Change Data Capture combined with modeling flexibility and rich query support, significantly improve developer productivity, and reduce the time to market for new applications at Uber.

— Evolving Schemaless into a Distributed SQL Database

Why not use an open-source alternative? Because Uber builds in-house. That’s what Uber does.

You could argue that DocStore provided the features they needed in a way no other alternative could. But you would be wrong!

Our homegrown Docstore was a perfect match for our database requirements, except for Change Data Capture (CDC) a.k.a., streaming functionality. [...] We decided to build a streaming framework for Docstore (project name “Flux”) and used that for LedgerStore’s Manifest generation.

— How Uber Migrated Financial Data from DynamoDB to Docstore

So let me get this straight: DynamoDB was a bad choice because it was expensive, which is something you could have figured out in advance. You then decided to move everything to an internal data store that had been built for something else3, that was available when you decided to build on top of DynamoDB. And that internal data store wasn’t good on its own, so you had to build a streaming framework to complete the migration.

And nobody got fired for this?

But nobody was optimizing for cost. They were optimizing for their next promotion. Each rewrite was a new proposal, a new design doc, a new system to put on a resume. The incentive was never to pick the boring, correct choice — it was to pick the complex, impressive one.

This isn’t Metaverse-levels of disaster, but relative to Uber’s scale, gets pretty close.

Catastrophes Disguised as Case Studies

What bothers me the most is this: by 2019, it was painfully obvious that Uber had made a terrible decision when they built LedgerStore on top of DynamoDB.

And yet, when AWS invited Uber to present at re:Invent 2019, they said yes.

I’ve written about Uber’s testing practices before, and praised them for it — the article hit the front of Hacker News.

But let’s call a spade a spade: when you actively disguise an atrocious decision as a case study for a database technology, you’re no less fraudulent than one of those hedge fund managers talking their book on TV.

It is the technological equivalent of an arsonist writing a fire safety manual.

On a second level, there’s the publications that regurgitated this case study without looking at the full picture: ByteByteGo has an article on LedgerStore praising “The cost savings from this migration” with yearly savings “exceeding $6 million due to reduced spend on DynamoDB”.

I can’t possibly comment on this.

When you’re tasked with building a system of any kind, not just ledgers, the technology is never enough. If you’re building a system that makes the economics of your company impossible, you’re better off not building it.

Focusing solely on the technical requirements, and not seeing the costs, is a disservice to the business that employs you.

Uber didn’t make a ledger mistake. It set the wrong incentives.

And it paid millions of dollars for it.

This was The Payments Engineer Playbook. I’ll see you next week.

Financial transactions aren’t database transactions.

Sure, they are the inspiration for database transactions. Turing Award-winner Jim Gray defined the standard measure of database transaction processing in terms of debits and credits. They also follow a layman’s intuition that, when money leaves an account, it immediately arrives at another.

But that’s not what usually happens.

When you make a transfer from your bank account, those funds almost always take some time to arrive. It may be a few seconds, or a business day, or a whole month (if your bank is settling payments made to you in Brazil1).

What happens during that time is treasury. Whoever is holding the funds in transit can use them as a (very) short-term collateral. It can deposit them and earn some interest, it can use them to improve their cash-flow, it can do as they please, as long as the funds arrive where they’re supposed to, when they’re supposed to.

That’s not how database transactions work at all.

Nevertheless, modeling ledger transactions as atomic units can work fine. You wrap debits and credits, make sure they’re double-entry safe, and work under the assumption that all those entries are all pending, or all posted. And if something goes wrong, you archive those entries, knowing that you don’t have to do anything else. It’s clean, safe, and predictable.

You may very well do that; Stripe doesn’t.

Stripe processes billions of events per day through a system they call Ledger. And in Ledger, they don’t assume that both sides of a money movement arrive together. After all, they don’t have to come from the same service, or in the same order. Each event is its own thing, with its own timestamp, source system, and moment of arrival; the only thing they share is an identifier.

That’s one limitation we already saw in stock markets: the risk of quickness versus completeness.

This risk is central to stock market systems. In order to scale, multiple matching engines are running in parallel, processing massive amounts of buy and sell orders with low latency. As soon as two orders are crossed (a buy order matches a sell order for a given price), there’s an execution, and that information gets disseminated quickly and fairly.

But what if something bad happens? What if regulators step in and ask “what happened here first”?

Like, two buy orders for the same price get sent at the same time by two traders, but one gets crossed and the other doesn’t, or does at a higher price.

How can you be sure that you were fair to both orders? Which one is “technically first”?

That, weirdly enough, is an ambiguous question. Because you haven’t clarified which notion of time is being used to sort the orders.

It’s not that transactions in a ledger can’t be atomic; they can. It’s that you don’t need them to.

And the real question is: what are you missing out when you force transactions to be atomic?

I’m Alvaro Duran, and this is The Payments Engineer Playbook, the only newsletter on Earth tailor-made for engineers of money software. Every week, more than 2,000 subscribers from companies like Stripe, Coinbase and Modern Treasury get a dive deep on how to build software that moves money around. Not to pass interviews, but to do their job exceptionally well.

When money is on the line, stakes are sky high and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

Here’s what you can expect in today’s article:

• Why computer science trained us to think of transactions as atomic, and why that assumption breaks down in distributed ledgers

• How Stripe’s Ledger posts individual entries instead of atomic transactions, and gets away with it

• The trade-offs between atomic and non-atomic models, and when to pick which

Enough intro, let’s dive in.

Read more

There’s a playbook for building successful companies that contradicts everything Paul Graham ever wrote.

Most business advice, what circulates on X, on Hacker News, and on Youtube, is modeled on PG’s essays. “A startup is a company designed to grow fast” (Startup = Growth); “the main value of whatever you launch with is as a pretext for engaging users”; “put a big piece of paper on your wall and every day plot the number of users” (Startups in 13 Sentences).

But the reality is that there are companies out there designed to grow slow, to launch late, and to deliberately restrict sales, that not only survive, but thrive.

The tech industry business model feels played out. Silicon Valley is both a place and a comedy show. It has grown so much that we now have a term for the point when these platforms have gone too far: enshittification.

Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die. I call this enshittification, and it is a seemingly inevitable consequence arising from the combination of the ease of changing how a platform allocates value, combined with the nature of a “two-sided market”, where a platform sits between buyers and sellers, hold each hostage to the other, raking off an ever-larger share of the value that passes between them.

— Cory Doctorow, The ‘Enshittification’ of TikTok

But as far as I can tell, nobody has articulated an alternative way of building online businesses. There’s a collective aloofness in the industry about this: “sure, enshittification is bad,” they seem to say, “but it’s the only way to build a startup.”

PG’s advice is battle tested: if you ignore his essays, you’re probably going to fail. But I argue that you can do something else: not to ignore PG, but to deliberately disobey him.

Let us call that approach The Luxury Playbook.

I’m Alvaro Duran, and you know I like playbooks. This article of The Payments Engineer Playbook, however, isn’t about the technology that moves money around. Today is April Cools: a celebration of what doesn’t fit the mold, so instead, I’m discussing business strategy.

But not your usual MBA pablum.

Since most startup founders learn how to build businesses from tech adjacent media, they usually treat PG’s advice as gospel. Thus, alternative approaches are poorly understood and underutilized in tech, and for exactly that reason, can be a secret weapon for those who wield them.

This is tricky advice, though. Most investors won’t understand it either, so funding will be more difficult. This article therefore has two main goals:

• To make you understand The Luxury Playbook and apply it to your company.

• To be a reference for investors who want to fund companies using this approach.

Enough intro, let’s dive in.

Make Something People Want, and then what?

It is impossible to build a successful business if you don’t make something people want.

That’s why both luxury brands and startups are, initially, very much alike. They both reject building for “everyone”, because that’s the same as building for no one. And both are designed to amplify the intensity of their founders.

Startups call it “founder mode”. But luxury founders go as far as naming their companies after them: Ferrari, Hermès, Yves Saint Laurent, Beats by Dr. Dre.

So the startup and the luxury brand agree on the starting conditions: make something real, stay close to it, and find Product Market Fit.

What separates them is what happens next.

The startup, once PMF is achieved, treats it as a milestone. From then on, it’s all about growth, speed, and distribution. The luxury brand, however, treats PMF as a horizon. Startups leave product market fit behind; luxury brands never fully reach it.

Luxury As The Anti-Playbook

You probably don’t dream of buying a Birkin bag.

But Samantha from Sex and the City can’t simply pay $4,0001 to get it. It’s not that she can’t afford it; it’s that she has to wait 5 years.

Luxury seems to operate in a completely different universe: rising prices famously spur demand (economists call these Veblen goods), websites do not show the price of the items, and physical shops are empty most of the time, usually because there’s a bouncer at the door.

Paul Graham’s advice is widespread because it makes sense. People understand in their bones that the right product must win, that being close to the users is a good thing, and that charts going up and to the right are indications of valuable companies.

On the other hand, Ferrari cars are uncomfortable, way too expensive, and you have to wait a few years to get one.

And yet, every kid wants a red car for Christmas, and every man wants a red Ferrari when they turn 40.

Different concepts of value

Luxury brands aren’t built to maximize growth. But that doesn’t mean they don’t make sense. Growth, for startups, vindicates the product. It’s evidence that a lot of people value what they’re selling.

Luxury brands, however, maximize the desirability of the product. It is very much the way consultants sell their services, and it’s not a coincidence. Luxury products have delayed gratification built in, which justifies the price in the mind of the buyer.

Startups make something PEOPLE want; luxury brands make something people WANT.

Move fast, or never break things

A Cartier Tank watch you buy today is largely unchanged to the one introduced in 1918.

Startups iterate and move fast. Their founders understand that every iteration is both a product to sell and a quest for market data; continuous change refines it, adapts to the changing environment, and sharpens the model of its ideal customer.

Luxury sees frequent change as instability, and the product as a promise. The Rolex you’ve waited years to buy isn’t just for yourself; in time, your kid will grow up, and you’ll give it to them, as if to pass the baton.

Startups move fast; luxury brands make timeless things.

Feedback as contamination

Startups iterate quickly, because the point is to gather feedback.

Their founders have gaps in their knowledge. These gaps are hard to spot, because your brain papers over them with assumptions. It is best to find paying customers, and figure out as much as you can from them, invade their privacy if required, so that the product feels tailor made to the ideal customer.

Luxury brands could not care less about that.

Obsessive attention to users is anathema in luxury. The creative genius is the source of all product decisions, and customers have little to say about that. A luxury brand dominates their customers the way a parent dominates their kids: “this is what is best for them, even if they don’t like it at first.”

Startups build bottom-up; luxury brands build top-down

The value of friction

Stripe’s entire thesis was to accept payments “in 7 lines of code”.

Nascent companies often position themselves against a big, but well-known competitor. Inertia and status quo are their biggest barriers to growth, and friction is the conversion killer. Startups, therefore, iterate on the way users interact with their product and remove friction every step of the way.

You can’t have startups without User Experience.

Luxury, on the other hand, embraces friction. A dream is the negation of the obstacle to attain the desire, but the obstacle must be there, and it must hurt. It is precisely the fact that not many people can have what you’re buying that makes it valuable. It says something about why you’re not just like anybody else.

Startups search for a moat in their business; luxury brands build a moat in their products.

What this means for online businesses

The luxury logic can’t be confined just to handbags, watches and sports cars.

One of the reasons I’ve been obsessed with luxury recently is because I have a thesis of my own: that luxury can be applied to online businesses, and digital products.

Since in digital markets the marginal cost of selling a product is near zero, everything naturally drifts towards commoditization. Things are free to sell, so we might as well give you cat videos for free, and aggressively pay for it with ads.

But this can’t be the only way to build successful businesses online. I have a visceral reaction to pointless ads. So, I’ve been mulling over this idea of online businesses that eschew commoditization.

And that brought me to luxury.

Luxury is taste, and there isn’t anything as tasteless as advertising. Therefore, online luxury brands, if they exist, build on the ideas I’ve described:

• Maximize the desirability of the product, not the user count

• Move slowly, and expand their offering only when it adds to the overall experience

• Manufacture scarcity, even when the Internet has removed it

And you know what? I believe they exist, even if they don’t call themselves luxury. In fact, I believe that the first luxury digital product that has ever existed was the Bloomberg Terminal.

The Terminal is an informational product, accessed online, that signals that your financial company means business, and that it’s profitable enough to afford the price tag: $30,000 per user per year.

Why so expensive? Well, why not? Wall Street pays a premium for high-quality business information, delivered instantly, and that’s become even more true over time.

The Terminal is complex, intentionally dense, not optimized for casual users. And yet, sales from the Terminal account for more than 85 percent of the company’s revenue. The product itself is largely unchanged since its inception in 1982. Sure, “if it ain’t broke, don’t fix it.” But that sounds a lot like a Cartier watch.

Bloomberg made financial data feel like a professional credential. A founder following Paul Graham’s advice would have never invented the Bloomberg Terminal.

Superhuman is a more recent and interesting case. It launched with a wait-list, onboarding gatekeeping, and a high price for an email client.

That scarcity worked as a growth mechanism in the early days; everyone wanted in, precisely because not everyone could get in.

And yes, eventually Superhuman abandoned scarcity in order to keep growing. But that doesn’t invalidate the luxury approach. Superhuman simply abandoned it.

What does a digital product that remains faithful to The Luxury Playbook look like?

Beyond Y Combinator

Graham’s advice is not wrong. Making something people want is a valid way of building startups.

But I’m just tired of it. I’m tired of the same kind of CEO, selling their product onstage, sporting a black turtleneck in my memory, even when they actually dressed in something else.

I’m tired of a business model that puts the online user at odds with the company in order to build a business moat. Of treating users as the product, then customers as the product, then even the shareholders as the product.

The Startup Playbook feels played out at this point.

It’s time for companies to test if The Luxury Playbook makes sense for them. It’s not that the odds are in their favor anyway. So they might as well try something different.

In a big company, you can do what all the other big companies are doing. But a startup can't do what all the other startups do. I don't think a lot of people realize this, even in startups.

— Paul Graham, Beating the Averages

The choice between the two playbooks is a choice about the kind of value you want to create in the world. The Luxury Playbook is the natural consequence of people taking taste more seriously.

Go make your taste a differentiator.

This is it for this week in The Payments Engineer Playbook. I’ll see you next week.

PS: You’ve come this far, I have some secret to share with you.

I’ve written this article with a not-so-secret question in mind: how can I turn my newsletter into a luxury brand?

And after writing it, I’m convinced that newsletters are the perfect playground for building luxury brands online. This, as I said, is somewhat uncharted territory, and that’s why the most well-known publications out there are funded with ads, not with paid subscribers.

In this new model, newsletter articles are more like bottles in a wine cellar. That’s why they’re paywalled: because subscribers are the newsletter’s patrons.

I’m deliberately turning The Payments Engineer Playbook into something much more exclusive and much more valuable for a smaller set of highly interested individuals. The Playbook is the only newsletter on Earth tailor-made for engineers who build money software, and I want to speak to those who want to do their job exceptionally well.

For that reason, I’m putting together a Flagship Membership.

It’s going to be expensive, and it’s going to demand work from you. But it’s going to be worth your money and your time.

If you’re interested, please write down your name and your email in this form, and I’ll be in touch.

I’ll see you around.

Data warehouses are poor implementations of ledgers.

You may think that, since all a ledger does sometimes is store money data, you can do away with all the complexity that ledgers require (namely, append-only tables) and use a data warehouse to store all your company’s financial data.

This is a mistake. The purpose of a ledger isn’t simply to store financial data.

A ledger that only stores data is doomed to fail because, in finance, the history is as important as the end state. When auditors and financial analysts look at a company’s finance, they want to make sure that the numbers don’t simply look good, but also tell a good story.

A bad history with a good look has a name: fraud.

The purpose of a ledger isn’t to store money data, but to store its history. We call that integrity: the ability to see what the ledger looked like, and query its accounts’ balances, now and at any point in the past.

Actually, accountants often get this backwards: it is integrity that leads to double-entry, not the other way around.

Ledgers, first, have integrity, and then they enforce double-entry.

This is The Payments Engineer Playbook, the only newsletter on Earth tailor-made for engineers of money software. Every week, more than 2,000 subscribers from companies like Stripe, Coinbase and Modern Treasury get a dive deep on how to build software that moves money around. Not to pass interviews, but to do their job exceptionally well.

When money is on the line, stakes are sky high and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

Here’s what you can expect in today’s article:

• Why immutability is a tool to verify integrity

• Why double entry is its natural consequence

• How to achieve consistent historic reads even when you apply compensating entries

Enough intro, let’s dive in.

Read more

You don’t know what a ledger is for.

You think you do, but you don’t. You’re aware that ledgers “keep accountants happy” and that “keep track of money across the business.” But your company has been doing just fine without one, so it’s not entirely clear why all of a sudden it would need one.

If you don’t know what a ledger is for, then having one is pointless.

Not knowing what a ledger is for has disastrous consequences, whether you’re a manager or an executive. Managers who don’t know what ledgers are for can’t get their projects approved; executives who don’t know what ledgers are for build ineffective ones.

Vague awareness of what a ledger is for isn’t enough. Only when you know precisely how a ledger fits within the context of your business, the project gets approved, the engineers understand the goal, and the resulting system is very effective, unlocking new avenues of growth and better financial planning and analysis.

But you don’t know what a ledger is for.

Let’s change that.

This is The Payments Engineer Playbook, the only newsletter on Earth tailor-made for engineers of money software. Every week, more than 2,000 subscribers from companies like Stripe, Coinbase and Modern Treasury get a dive deep on how to build software that moves money around. Not to pass interviews, but to do their job exceptionally well.

When money is on the line, stakes are sky high and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

Here’s what you can expect in today’s article:

• Why ledgers suddenly go from pointless to top priority at many startups

• The Accounting-Engineering tension at the core of every ledger

• Why all ledgers have to fulfill at least one of 4 use cases

• How to know if your company needs one

Enough intro, let’s dive in.

Money is a Skill

Money requires judgement, accuracy, and ownership.

Knowing how much you have in the bank is good, but not enough. The skill of money means acquiring the capacity to understand how money was used in the past, what insights to gather from that usage now, and how to make the most of what you have in the future.

In other words: the skill of money is about cash flow, budgeting and ROI. And these subskills are built incrementally.

In the beginning, startups don’t require ledgers at all, because the only metric that matters is “the runway”. Just as airplanes need a runway of a minimum length to take off, startups must have enough cash to reach Product-Market Fit. The only money skill that matters at this stage is “how much time can I buy with the expenses I have right now?”.

The 70 percent of startups that don’t get to achieve PMF don’t get to build a ledger either.

The startups that live to tell the tale face a more sophisticated money problem: how to make the most with what they have. Past survival, every post-PMF startup’s CEO has to allocate their money in such a way that the company achieves explosive growth. Sales are no longer founder led, employees cannot simply “do things that don’t scale”, and engineers have to titrate up the software used to satisfy customers’ needs.

Given the opportunities opened by product-market fit, the money skill that matters is “how much money do I allocate on each opportunity to achieve the most growth?”.

This is the part where, all of a sudden, knowing where the money goes, and what for, becomes necessary. And, depending on the complexity of the money flows, whether having everything on a spreadsheet will cut it, or whether a fully-fledged system is required.

What ends up happening is often a mix of the two: money inflows are tracked in a PSP dashboard, and the codebase preserves some data regarding transactions on their database, with some basic reporting built on top. This is good: the last thing you want is to waste everybody’s time getting strict with your finances at the expense of the startup’s actual goals.

However, there’s a point when startups can no longer ignore their finances anymore, because profit starts to become as important as revenue. Past a threshold, and every startup has it, ROI suddenly creeps into every conversation with investors.

Growth isn’t the only thing that matters. And, by then, if you don’t have a ledger in place, you’ll be falling behind.

The Accounting - Engineer Tension

At scale, there’s a tension between correctness and edge cases.

When a company starts handling a large volume of transactions, across multiple accounts, currencies, or business units, tracking balances manually with a spreadsheet becomes inefficient, error prone and opaque. At that point, a ledger system provides a structured, auditable way to record everything money-wise, ensuring accuracy, clarity and compliance.

Some companies never reach this stage though. That’s why most of us don’t need a ledger to manage our finances: there’s X amount in the bank, we know the outstanding credit card debt, and your precise portfolio balance doesn’t really matter in your daily life. In simpler businesses with few accounts and minimal volume, building a ledger is unwarranted.

If a spreadsheet is enough, use a spreadsheet. Ledger adoption is a matter of timing and scale.

Spreadsheets work because they’re simple, and simplicity is a great tool for correctness. Having your finances right in front of you is the best way to spot errors and missteps. It is not the lack of features what makes a spreadsheet inferior to a ledger system. Rather, it is the complexity and the scale of the business that makes a simple, but human dependent tool, unwieldy.

This is the tension between the Accounting and Engineering systems within the ledger: accounting is focused on correctness and precision; engineering is focused on scale and throughput. A ledger system provides (and demands) both. If manual work at low scale is sufficient, choose the spreadsheet.

This tension is also the biggest reason why ledgers don’t get adopted until late: its necessity is felt by accountants, not engineers.

When to ditch the spreadsheet and build a ledger

Here’s a good framework for deciding whether to build a ledger for your company:

1. Transaction Volume: Your company is approaching 1,000 transactions per month

2. Audit Readiness: Your company takes around 40 hours of man-work to close the book for the quarter.

3. Error rates: Your company consistently has at least 1 percent discrepancy rate (as in “1 in 100 transactions aren’t correct down to the cent).

4. Reconciliation Effort: Your company’s finance team spends 20 percent of their day in reconciliation.

5. Forecasted Growth: Your company’s transaction volume is projected to grow more than 50 percent in the next year.

These indicators are different manifestations of the same need, and often reinforce each other. They’re symptoms of scale, and signal that your company will soon need a more rigorous approach to its finances.

A ledger isn’t an accounting tool

Ledgers aren’t compliance checkboxes.

A ledger is a system that builds the skill of money in your company. In the beginning, it may look like a ledger answers the question “where did money go?”.

But the trick is that, when built correctly, a ledger ends up answering “where should money go next?”.

Startups that treat money as a skill build ledgers slightly before they need them. Their executives are aware of the usefulness of spreadsheets, but can look into the future and see the necessity for budgeting and ROI conversations.

In other words: they see beyond revenue growth, and the inevitable refocus to profit.

This is it for this week in The Payments Engineer Playbook. I’ll see you next week.

I’m not a visual learner.

When I onboard into a new team, and I go through all the available documentation, the hardest part for me is understanding the system diagrams. These invariably consist of many boxes representing entities, or even individual services, linked to each other by interactions (HTTP or SSE requests, Kafka queues, you name it) represented by arrows. They always leave me more confused than before.

I’m not a visual learner. But the thing is: neither are you.

The biggest clue I have as to why is the fact that nobody seems to be bothered when I ask further clarifications about the diagram. Elaboration is expected. Despite the thoroughness of the engineer who spent all that time putting those diagrams together, it is never enough: a meeting to make those diagrams resonate is assumed to be needed.

Diagrams never seem to be enough. At best, they make meetings shorter. At worst, they waste everybody’s time.

The Cognitive Style of the Miro board

Arrows and boxes aren’t completely useless, though.

They’re very effective for conveying flow, which is why they get used all the time by people already onboarded. Diagrams are good reminders for how components interact, aiding in high level system design. You’re able to see all at once.

But the more sophisticated, multivariate, and formal your task is, the more damaging the arrows and boxes become.

That description fits accounting like a glove: although cash is intuitive and simple, it is only a fraction of what a ledger has to keep track of. Accounting is the process of tracking ownership, not money, over time. And ownership isn’t concrete enough to be stored in boxes, and moved with arrows.

Arrows and boxes serve engineers when they map code flows. No argument there. But their visual emphasis is misleading when it comes to accounting.

This week, we’re looking at the limitations of arrows and boxes to explain accounting to engineers. I should know: a significant part of my job consists of clarifying accounting to engineers, from the very junior to the CTO. And, being an engineer myself, I initially used arrows and boxes to try to convey how to represent common use cases in accounting terms.

No longer. When you step into accounting, arrows and boxes are glorified squiggles.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to do your job exceptionally well. Especially as a payments engineer, where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

Here’s what you can expect in today’s article:

• The shift from “moving money” to “transferring ownership”

• Why using arrows to represent balance changes is confusing

• The “arrow from the outside” misconception

• And a better alternative to diagrams to explain accounting

Enough intro, let’s dive in.

Read more

Ledgers are a “get your shit together” tool.

The goal isn’t to record account balances, or where money comes and goes. A ledger’s purpose is to make the company trustworthy. All the strictness and the weird complexity isn’t to make life harder for engineers, or to prevent software bugs, but to keep accountants and auditors happy.

If a ledger can’t be trusted, neither can the company.

In order to interact with your ledger, all the other services have to make sure that they’re sending the right data in the right way. Sure, you can create a translation layer between the other services and the ledger, but that defeats its purpose: a service that initiates, validates, or completes money movements must be, implicitly, as strict as the ledger.

And the way that strictness is enforced is with a ledger that tolerates no bullshit through its interfaces.

All the strictness and complexity isn’t there to slow you down. It’s to create consistency, traceability, and proof.

• Proof that every money movement is intentional

• Proof that nothing can quietly disappear

• Proof that what the company says in aggregate matches the collection of what every transaction whispers individually

A ledger is discipline encoded in software.

This week, we’re looking at the most fundamental building block of ledgers: immutability. In most engineers’ mind, it translates to “append-only”. And sure, append-only databases are immutable because you can’t UPDATE or DELETE rows on them.

But that’s only the beginning of what immutability means for a ledger.

Here’s what you can expect in today’s article:

• A practical definition of immutability for ledgers (and why you can have it with mutable databases)

• The difference between Journaling and Posting ledger entries, and why it matters

• Why git is a nice metaphor for what a ledger’s immutability is about

• And for the database nerds, read skews, and why are relevant

Enough intro, let’s dive in.

Read more

The biggest obstacle to multi-currency ledgers is a misunderstanding of what the balance rules really are.

Currencies can’t be balanced; you’re comparing apples to oranges. In theory you can balance them using comprehensive exchange rate lists, so that currencies can be comparable. In practice, this never works: it’s like trying to join database tables by timestamps. ASOF joins might work, but time series data is never aligned perfectly. Slight delays between cause and effect can make the reconciliation process unwieldy.

What we need is to reinterpret the balancing process when multiple currencies are in play.

I’ve hinted at this idea in Ledgers of Catan, but the reality is that virtually every company that deals with more than one currency compiles a list of exchange rates (or gets one from an external provider) and uses that as a “best effort” to reconcile numbers, accepting that a compensation will be needed at the end of the period.

They’re doing ASOF joins with their money.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to do your job exceptionally well. Especially as a payments engineer, where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting insights from it.

In this article, I’m going to use a multi-currency wallet company as a practical example of how to balance multiple currencies without the strictness of the rule that “every transaction must be balanced” that every accountant uses in practice. This is dangerous territory: nobody does it.

But you’ll see that it can be done, and that it makes sense.

Read more

Stablecoins will have truly achieved adoption when we no longer need to talk about stablecoins.

When you pay with your credit card, the last thing you want to know about is the infrastructure involved in it. What you want to see is the green light, the check mark, the “Accepted” message. The end user of a card doesn’t even want to think about the payment at all.

The best payment experience is the one where you think that the money was instantly transferred. Even when it wasn’t.

The biggest promise that stablecoins are making is that it won’t change anything in the real world. That stablecoin’s actual goal isn’t to upend the payments industry, but to clean house. Ideally, using stablecoins to move money around might change the players, but leave everything else the same.

Sweeping clean the messy back, but leaving the facade alone.

On another level, the same thing has to happen to payments engineers. Stablecoins have to feel exactly like ACH, FedNow and Wire: battle-tested protocols for transferring money that, to some extent, are interchangeable.

So, apropos of their launching of a new integrated PSP for fiat and stablecoins, I sat down with Sam Aarons, CTO and co-founder of Modern Treasury, and Dan Mottice, founder of Beam, and now Head of Stablecoins at Modern Treasury following Beam’s acquisition.

The team at Modern Treasury and I have been circling each other for quite some time since I published an article on their approach to ledger authorization using events a while ago.

Finally, the timing and the topic have aligned completely. As I told them, I wanted to have a conversation so that we could talk shop and have a good time, with complete freedom to write an article afterwards that was valuable to readers.

This article is the result of that conversation.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. It is the only newsletter on Earth tailor-made for engineers of money software. Every week, we investigate the technology that moves money around, so that you become a smarter, more skillful and more successful payments engineer.

And we do that by cutting off one sliver of it, and extracting insights from it.

This week’s article is the first interview-based article I’ve done in The Playbook. Following my skeptic take on stablecoins, I sat with two of the engineering leaders of Modern Treasury to put my skepticism at ease, and see what the future holds for payments as this interesting piece of tech gains adoption.

Enough intro, let’s dive in.

A new form of liquidity

Seems like I hit the nail on the head a couple of weeks back when I said that stablecoins were the international transfer method that you would’ve come up with if you really didn’t want to deal with banks at all:

The good thing about crypto, and the reason why the transaction costs are so low, is because there’s nothing central about it. The recipient may be on the other side of the world, and they have the same level of access and visibility to it as you. It may be slow, but not SWIFT slow. And it’s way cheaper.

So what we probably need is something that’s crypto-like, to retain the global aspect of such a ledger, but without the volatility of cryptocurrencies. One way to do that is to have a blockchain that distributes the value of a predefined pool of resources, something that doesn’t change in value against a fiat currency.

So that’s what stablecoins really are. And they’re now facilitating $30bn of transactions daily.

When I mentioned this to Sam and Dan, they both laughed. “We internally call that Stablecoin Sandwich.”

It’s so visual: you have fiat on both ends of the transfer, but there’s a black box in the middle, responsible for moving value from left to right, in which your Canadian dollars get converted to stablecoins by an on-ramp partner, and then into dollars by an off-ramp partner. Modern Treasury then can send those dollars with traditional rails like ACH or FedNow.

Fiat on both sides, with stablecoins in the middle.

The stablecoin sandwich is what bridges the crypto and the fiat world. It’s what makes stablecoins “feel” like fiat, even though they aren’t.

Cross-border payments have been clunky since time immemorial. SWIFT, or more generally correspondent banking, was devised when most “long distance” transfers were made by flying checks in bags all over the US. Western Union, or money transmission, was the first private approach to solve cross-border transfers, by setting up a massive enterprise with agents in multiple countries around the world on the ground.

Then came Wise, whose founders realized that money didn’t need to move cross-border at all. Payment aggregation simulates “sending money” by finding and crossing supply and demand for foreign currency within the country of origin and destination separately.

Still, there’s always a company in the middle. Unlike other cross-border transfer methods, stablecoins are interoperable: the blockchain is publicly accessible, and moving money on top can be done without the need for intermediaries.

Stablecoins are a bit more difficult to use, and that’s why you may benefit from using liquidity providers to make things easier. But you don’t need them, not in the sense that you need banks to send money via ACH.

And not needing a bank is precisely what Modern Treasury is aiming at.

No bank? No problem

It’s not that banks are a problem. It’s that dealing with them makes everything way slower:

Too many companies are held back by unreliable payments infrastructure and slow, complex bank integrations. With Payments, we’re giving teams a faster path to market than BaaS or securing their own bank sponsorships.

— Matt Marcus, CEO of Modern Treasury, on the Payments announcement

MT’s strategy is to move from nothing to providing financial services as fast as possible. But payment orchestration and ledgering are just the technical steps; non-technical steps like bank partnerships and compliance with regulators across countries are also required.

There are two ways to approach this problem.

One is to Bring Your Own Bank, an acronym that I keep seeing in the fintech space even though it was first used elsewhere. With BYOB, you handle the non-technical side of things yourself, and bring a technical partner like Modern Treasury or a BaaS into the mix. BYOB sounds complex, but it’s sometimes the only option available. Most BaaS work exclusively with one bank, coupling your choice of BaaS with your choice of bank.

On the other hand, an integrated PSP handles everything for you. Modern Treasury’s PSP handles most of the paperwork and the banking relationships, and you deal with MT directly.

You trade control for go-to-market speed.

Problem is, in the beginning, you may not have support for some features, even when you’ve done everything you could to start. That’s where stablecoins come in. Since no bank is required, you could theoretically start transferring value in stablecoins right away.

This was the missing piece in the PSP puzzle that is now available thanks to stablecoins. But there are more things you can do with them.

The Flow and The Meta-flow of Funds

Yes, cryptocurrencies were built on speculative use cases. But that’s a boring form of skepticism.

Given where crypto has evolved to, stablecoins represent a maturing phase of decentralized finance. One that acknowledges that fiat isn’t going away, and attempts to bridge some of the functionality that’s useful about blockchain (decentralization, trust-less value transfer) to the world of the normies.

There’s the flow of funds, and there’s the meta-flow: moving more of fiat into usable crypto.

• Bank accounts become crypto wallets that store stablecoins. You can now hoard dollars, even when you can’t hold them.

• Treasury management becomes “hybridized”: moving actual dollars may be difficult, or incur taxes, but moving stablecoins isn’t.

Yes, there was a time when private keys were much more exposed and easier to steal; the industry is now aware of this problem. Blockchain is trustless, but you can trust someone with your keys and never have to fend off hackers anymore.

That’s not the thing you must be alert about stablecoins.

The Skeptic’s Last Question

Stablecoins are cheaper and faster than fiat. They’re not volatile like Bitcoin, and are even programmable, so you don’t have to deal with lawyers, but with transparent code.

So what’s the catch?

In what way are stablecoins inferior to the traditional way of moving money across countries, that accounts for more than 99 percent of the total flow of funds right now?

The answer is: usability.

“Stablecoins are unforgiving” is how Sam put it. There’s no takesies-backesies in the crypto world. That’s what removes banks from the equation. In ACH, you get your funds returned when you send money to a dead person; in crypto, you have to ensure that everything is correct in advance, because once you hit Pay, there’s no going back.

In a world where the rules are written in code, comments are for information, not control.

“The plus side of this”, Sam followed up, “is that some players may like this trade-off, which wasn’t available for them before”.

Stablecoins are consistent payments experience

For crypto to become usable, developers must not have to use a different set of APIs to deal with them.

This echoes an idea that I’ve continually come back to: the mistake that Stripe made when they assumed that all payments were card payments:

Payments are deceptive.

Representing a Payment with a single object sounds simple, but that’s because you’re moving the complexity elsewhere. The Token/Charge architecture was the obvious choice, but it made integrating other payment methods way more difficult.

And that could work if all you need is credit card payments. But that is less likely than it used to be.

In time, Stripe’s engineers understood that 7 lines of code wasn’t everything. They looked at the problem as a whole, and asked how they could make adding new payment methods easier.

That, in the end, is what’s making developers more productive. Which was Stripe’s goal all along.

What MT is trying to do here is something similar: build an API that’s consistent across all forms of money transfer.

Sure, faster go-to-market is a good thing for many companies. But the real boost is in the technical integration and expansion of those who are interested in crypto, but deeply embedded into fiat.

How can they test the waters with minimal developer effort?

Major businesses, even when curious, aren’t going to deploy a team of engineers just to see if stablecoins are hype or not. They want to get their feet wet, and therefore won’t commit to it unless it requires almost no effort to do so.

Such an approach is only possible when both fiat and crypto use the same set of endpoints to operate.

Thanks to Sam Aarons and Dan Mottice for taking the time to answer my questions. You can find more about Modern Treasury’s Integrated PSP for Fiat and Stablecoins here.

That’s it for this week in The Payments Engineer Playbook. I’ll see you next week, on Wednesday, as usual.

Stablecoins are the ultimate ledger.

A tool that moves money globally, quickly and cheaply. A transfer irrevocably linked to a piece of software that serves as the payment contract. No lawyers, no bankers, no humans inbetween two parties exchanging value for goods and services.

Like that’s ever gonna happen.

Reality is much more surprising and interesting than a completely human-less money movement infrastructure can model. Human beings, even bankers and lawyers, will always be involved, because money is a game of trust.

And trust is inherently human.

A game in which trust is fundamental will always have humans at the center. You can have decentralized transfers; but you can’t have decentralized finance.

Stablecoins are the latest attempt at bridging the world of crypto, with programmable money, full transparency and non-reversibility, and the real world, with taxes, fraud and honest mistakes.

But they’re not going to deliver on a machine-only finance.

In fact, stablecoin isn’t meant to do that. I’d say that stablecoin aims at implementing as much as possible of the crypto world in a way that’s practical in the fiat one, but no more.

Which is probably the best move possible.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to get promoted. Especially as a payments engineer, where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting tactics from it.

In this article, I’m going to assume that you’re as skeptic about crypto as I am. Sure, I’ve bought Bitcoin in the past (who hasn’t?). But it’s been 18 years since the Bitcoin paper was published, and since then, it has failed to live up to the expectations.

Plus, there’s also the grift, the scams and the speculation.

So what I’m going to do today is to explain what stablecoins are, while being realistic about what can be done with them. You don’t have to trust me, or agree with me. But here’s the thing: that’s the whole point of crypto anyway.

Here’s what you can expect below:

• What makes stablecoins fast, cheap, global, and programmable

• What’s stable about them

• Where do they fit in the history of moving money, both domestically and internationally

• What can we do with stablecoins that we wouldn’t be able to do otherwise

Stablecoins aren’t going to give us a world without bankers and lawyers. What they’re going to give us, if they become a significant chunk of the money flow worldwide, is a different kind of bankers and lawyers.

More cerebral, more technical, and less sales-y.

Subscribe now

Tech-savvy lawyers and bankers

Stablecoin is programmable money.

They’re digital tokens designed to maintain a 1:1 value relationship with an established asset, typically the US dollar. The two largest are Tether (codename USDT, with $184 billion as I’m writing these words) and USDCoin (USDC, with $73 billion as of right now).

Which sounds like a lot except both combined are roughly one tenth of a percent of the $200 trillion global payments market. Still, adoption is growing, but unlike cryptocurrencies, stablecoins’ adoption is driven not by speculation, but by practical usage.

And it’s the stable side of things that makes a dramatic improvement over “classic” cryptocurrencies:

How would you send somebody money internationally if you couldn’t use a bank?

Cash over email, or over a handcuffed briefcase might work, but you run the risk that it gets lost. No problem; you get creative. How about you buy shares of Apple, and instruct the recipient to short sell them, and then you can just transfer those shares to them?

That works. That’s actually brilliant. But the transaction costs are probably too high: there’s the difference between the buy and sell price, the broker’s commissions, etc.

Let’s go crypto, then: you buy some Bitcoins, they sell a bit of theirs, and then you can transfer those Bitcoins to them. The transaction costs are somewhat fixed, and low, so this is an improvement. But Bitcoin’s price is much, much more volatile than Apple’s. The risk of having to offset the price difference makes this approach unwieldy at scale.

The good thing about crypto, and the reason why the transaction costs are so low, is because there’s nothing central about it. The recipient may be on the other side of the world, and they have the same level of access and visibility to it as you. It may be slow, but not SWIFT slow. And it’s way cheaper.

So what we probably need is something that’s crypto-like, to retain the global aspect of such a ledger, but without the volatility of crypto currencies. One way to do that is to have a blockchain that distributes the value of a predefined pool of resources, something that doesn’t change in value against a fiat currency.

So that’s what stablecoins really are.

Stablecoins are programmable because they’re fiat represented in a crypto form. Unlike cash, and also unlike money deposited in a bank, stablecoins can be controlled by code.

You can write smart contracts that controls whether money can be moved, or is forcefully moved. Sure, this opens a can of worms: we’re opening the door to strict adherence to rules, no matter how absurd or immoral.

But it also opens the door to a few interesting products that we’ve never had access to, because of the number of intermediaries involved in moving money in the fiat world:

• Conditional payments: Insurance payouts come to mind, but also gambling. There’s an active community of people "predicting” outcomes at sites like Polymarket, and stablecoins are often used to enforce that the money is paid appropriately.

• Lending: For the longest time, we’ve needed banks to intermediate between lenders (who lend short time with their deposits) and borrowers (who borrow long with their loans). But we can now aggregate pools of money into stablecoins, and securitize a loan among many lenders, who earn interest on a pro-rata basis.

• Compliance: Programmable money means money that can blacklist certain senders or receivers, bringing KYC to a whole new level.

This list isn’t exhaustive, by the way. Crazier use cases will probably spring up as stablecoins gain further adoption.

Stablecoins are Triffin Stable

The world unpegged from the gold standard in the 1970s, and we’ve tried to go back to it ever since.

And the reason the US abandoned the gold standard was because at the core of every currency used for international trade resides a paradox that no one has really figured out how to solve.

Here’s the issue: gold is impractical to move, for obvious reasons. A much better alternative is to have some pointer to it that’s easier to move, so that everyone can trade knowing that what they’re really exchanging isn’t the pointer itself, but the underlying gold it represents.

After World War II, due to the fact that virtually all the gold was in the US, the world came to accept dollars (which were pegged to gold) as that pointer.

Now, the problem is that, in order to supply the world with enough dollars so that they can trade with each other, the US had to incur trade deficits consistently. There had to be a net outflow of dollars so that everyone could participate in the new era of globalization.

At this point, the US government had an incentive to issue dollars, even if they weren’t really backed by gold, because the expectation was that a) nobody really needed the gold, and b) everybody trusted that if they went to the Fed, they could get gold in exchange for their dollars.

As time went by, that consistent trade deficit bubbled up into a sudden crisis of confidence. Both a) and b) suddenly weren’t true anymore. And like a bank run, everyone tried to get their dollars exchanged for gold.

And that’s how the gold standard was abandoned.

The story has happened before (with the Roman empire) and after (with the Argentinian peso). My point is that something similar may happen with stablecoins as well.

Let me explain.

There are two ways in which stablecoins are stable. One could be called price-explicit, the other price-implicit.

Tether and USD Coin are price-stable. They’re accepted as if they were dollars, and in a legal sense, they are1. The price is enforced explicitly. Then, there are all the smaller stablecoins issued as staged, close loop wallets, which are pointers to a pool that’s dynamic in nature, but that maintains a 1:1 relationship because the tokens minted and burned correspond to the amounts of dollars deposited and withdrawn. They price is enforced implicitly.

Staged, closed loop wallets have existed for a while now, and Starbucks is the business that has made the most of them. As long as you have a tight ledger that controls that there are no cents dancing around, you’re fine. But history says otherwise about currencies that keep a 1:1 relationship to another asset by the sheer force of the law.

A peg is a story about why two things which are not the same are, in fact, similar enough to be treated interchangeably.

— Patrick McKenzie, Stablecoin mechanisms and use cases

The paradox between having to supply the world with a currency for trade purposes, and the loss of confidence on that currency because of the trade deficits required to do so is called the Triffin Dilemma.

As stablecoins gain adoption and become a significant chunk of money movement worldwide, storing a sufficient amount of dollars to back them up will be increasingly difficult.

By then, we will have to come up with a contingency plan. Or else.

I’m so excited about crypto

Nevertheless, my enthusiasm outweighs my skepticism.

True, there’s risk in assuming that we won’t find problems as stablecoins start running more of the international trade show.

But there’s a lot to be said about moving smaller amounts of money around without the need to deal with banks. There are simply too many intermediaries going on. Sure, they’re doing work that’s necessary. But that’s only because the current design makes them so.

Stablecoins can change all that overnight.

I’m skeptical about stablecoins conquering the world. Cash isn’t going away, and neither is SWIFT. But I’m hopeful about stablecoins being tremendously useful for a huge chunk of the day to day economy.

You can’t have decentralized finance. But you can have decentralized transfers.

That’s it for this week in The Payments Engineer Playbook. I’ll see you next week (on Thursday).

Double-entry accounting is still uncharted territory.

On the one hand, it’s ancient technology, older than the concept of negative numbers. But the expectations of what a ledger needs to do are more ambitious; what was OK to do monthly now needs to be done daily, or in real time.

Ledgers have outgrown their medieval assumptions.

It is clear that one of those assumptions is the necessity to consolidate all money movements into a single currency of reference. Traditionalists have extended the old model by layering ledgers: a single system gets partitioned into a USD subledger, a EUR subledger, and so on.

The expectation is that transactions between two of those layers is rare, and manual reconciliation between the two is a necessary evil to keep each layer simple.

That assumption is also breaking down, quickly:

Even startups and small businesses have the same problems as multinationals now, but needing far less scale and therefore much more granularity.

• Being remote means now being global. Clients are everywhere, and so are providers and employees.

• Banking apps not only handle deposits, but also investments in stocks, payments when we travel abroad, crypto, even angel investing.

• To the extent that the USD is no longer the only reserve currency, businesses that operate globally need cash stored in more than just dollars, all fluctuating against each other.

In some sense, this article is the practical follow up to the point I made at TigerBeetle’s Stablecoin Mistake: a 1000x faster ledger is something great to have, but the real deal is the business on top.

This week, I’m going to push the boundaries of what can be done with modern accounting. What can I do when I have unlimited processing power?

And in order to do that, I’m going to play The Settlers of Catan.

In Catan, players represent settlers establishing on a fictional, hex-split island, and build settlements, cities and roads by spending resources (wool, grain, lumber, brick, and ore). They obtain these resources on each player’s turn, when they roll two six-sided dice to determine which hexes produce resources.

If you have a settlement or a city on one of those hexes’ corners, you get resources. If not, you don’t.

There’s also the chance that you roll a 7, and you get to place a burglar on one of those hexes, steal another player’s card and “block” that hex, which will no longer produce resources until the robber is moved again.

It’s the kind of game where you win by pissing off everybody else.

So why on Earth is Catan accounting even worth it?

Because there’s no money. All you can do is barter. It’s the perfect scenario to test modern accounting, in a familiar setting.

And see where it breaks.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to get promoted. Especially as a payments engineer, where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting tactics from it.

In this article, we’re going to play Catan, ledger in hand. I’ll walk you through most of the operations in the game (minus the special cards), which will help us see:

• How to balance wool and lumber, and the perils of fair value accounting.

• Contra-accounts and off-balance-sheet derivatives (yes, really).

• What it all means for multi-currency ledgers.

Enough intro, let’s dive in.

Read more

You can build a ledger that’s blazingly fast with a slow programming language.

The trick is to realize that improving the overall performance of the ledger doesn’t really move the needle. What moves the needle is preventing your biggest bottleneck from slowing everything down.

A particular kind of obstacle placed in a particular place in the ledger can make any ledger glacially slow, even if it’s written in C++.

So where’s that bottleneck?

Every time you add a new transaction, most ledgers lock for writes the accounts involved. “That’s the only way”, you may think, “to prevent race conditions.” With those locks in place, you can now check the balance, make all the validations you need, and then proceed with the changes. Finally, the locks can be safely removed.

Here’s the thing: one of those locks may not be needed.

There’s a nice little trick that most engineers won’t notice because they treat all accounts as if they were equivalent.

They’re not.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to get promoted at your fintech job. Where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money in order to make you an outstanding engineer, and move your career forward.

This week, we’re going to discuss the impact of cardinality in ledgers. I’ve put together a valuable list of concepts that payment engineers have to be familiar with, all for free, and cardinality is one of those concepts.

In this article, I’m going to explain:

• What are high and low cardinality accounts

• What is the impact of cardinality in ledger performance

• How to leverage this insight to make your ledger even faster

Enough intro, let’s dive in.

Read more

You’ve heard this one already: “skate where the puck is going, not where it has been”.

In order to innovate, companies have to build for the world of the future, where conditions will have changed. What is silly now, where the puck isn’t, may eventually stop being so, because that’s where the puck will be.

But what if the puck ends up being somewhere else?

See, that’s the problem with trying to predict what the future will look like: you aren’t crazy enough to do it. The most you can do is to put yourself in positions where you can take advantage of the opportunities that are presented to you.

So, it’s not about skating where the puck is going. It’s about being able to try again if the puck doesn’t go where you are.

I first heard of TigerBeetle a couple of years ago in a piece from Chris Riccomini called Databases Are Commodities. Now What?. Since then, I’ve been experiencing a Baader-Meinhof phenomenon.

TigerBeetle is always there, everywhere I look.

Readers of this newsletter first recommended it to me, then demanded that I wrote about it. There’s a Primeagen video about it. They hosted a conference where Hillel Wayne gave a talk, and I’ve recently learned that matklad works for the company building it.

So it’s either The Algo really trying, or TigerBeetle having great marketing people. Because this is The Database For Financial Technology, and my newsletter is for engineers using databases (among other tools) for that very industry.

And let me tell you, TigerBeetle is a fine piece of engineering.

Yes, don’t take this article as software criticism: TigerBeetle is a marvelous thing. I’ve played around, tinkered with it, and I can say lots of positive things about it. It’s the kind of database that’s needed in many financial institutions desperately, and the only reason there isn’t more of them adopting it is likely because of banks’ atrocious IT procurement and contracting practices, and not TigerBeetle itself.

However, in trying to predict what the future would look like, TigerBeetle has painted itself into a corner. One that is increasingly clear isn’t the right corner to be.

Databases, like many other systems, are built on tradeoffs. A while ago, we explored QLDB, the now defunct Amazon’s ledger database, and I cheekily compared it to the spacecraft that the star character of The Martian had to propel itself to space with:

QLDB is like that vandalized rocket that Mark uses to take off from Mars and onto the spaceship that would bring him home. If postgreSQL is a fully fledged spacecraft, a BMW of luxurious interplanetary travel, with all the amenities and guarantees, QLDB is stripped from control panels, from backup systems, and all that doesn’t move the system closer to achieving its mission.

To boldly go where no database has gone before.

The problem with these design choices is that they’re so narrowed down on solving one particular aspect of the problem space that they’re very fragile when these solutions don’t make a business difference.

In QLDB, for example, the problem wasn’t really cryptographic verifiability. Sure, you can verify that the data hasn’t been tampered with independently, but does that really align with a system that’s owned by a cloud provider?

I guess that if Amazon couldn’t figure that out, the answer is probably “no”. You may want to verify immutability with Merkle trees, but trusting that the central authority isn’t doing anything with your data is probably a better business decision.

So it is with TigerBeetle. They’ve built a financial database that’s faster than any other alternative out there. And performance is something you desperately need when you build a ledger, since the whole architecture is predicated on lots of data that are accessed very frequently.

But TigerBeetle is very bad at handling multiple currencies. And the world seems to go in the direction of multi-currency ledgers, even if they aren’t the fastest.

TigerBeetle may be a database placed “where the puck is going”. But it’s lost the ability to get back if the puck ends up somewhere else.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to be great at your job. Especially as a payments engineer, where stakes are sky high, and the margin for error is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money. All to help you become a smarter, more skillful and more successful payments engineer. And we do that by cutting off one sliver of it and extracting tactics from it.

Today, we’re looking at TigerBeetle’s approach to multi-currency transactions. It sounds a weird topic to dive into, but if you notice how often you’re sending money abroad, pay for goods and services from other countries, or simply buy stocks from international exchange markets, you have to admit that this is becoming something that we’re doing more and more frequently.

And that’s putting a lot of pressure on single-currency ledger systems. Even when they’re 1000x faster than the competition.

In this article, we’re going to look at

• What is TigerBeetle’s approach to double entry accounting, and why.

• A crash course on stablecoins. What are they, and why everyone is talking about them.

• The “impedance mismatch” between TigerBeetle’s design and stablecoins.

• The unbounded future of ledger systems.

Enough intro, let’s dive in.

Reference Currency

For the longest time, ledgers have been denominated on one currency, and one currency only.

Our world is already testing this design decision to its limits. We’ve lived in a multi-currency world for quite some time, but the scale of multi-currency transfers was so minimal, humans in between single-currency ledgers were enough to smooth things over. Never needing to answer questions about these transfers with much granularity, nor in real time, also helped. As long as you get the answer by the end of the month, the quarter, even the year, you’re doing pretty well.

Not anymore.

Even startups and small businesses have the same problems as multinationals now, but needing far less scale and therefore much more granularity.

• Being remote means now being global. Clients are everywhere, and so are providers and employees.

• Banking apps not only handle deposits, but also investments in stocks, payments when we travel abroad, crypto, even angel investing.

• To the extent that the USD is no longer the only reserve currency, businesses that operate globally need cash stored in more than just dollars, all fluctuating against each other.

If this sounds intimidating, it’s because it is. Or it would be, if we keep relying on design decisions made in the times of Medici’s Florence.

Transfers, Not Transactions

Transaction is a loaded word.

There are financial transactions and database transactions. There’s also the transactional interpretation of quantum mechanics, and there’s transactional leadership.

But there are no transactions in TigerBeetle. There are transfers.

A transfer is a single movement of money, from a single debit account to a single credit account on—and this is crucial—the same ledger. They cannot be modified, though you can add new transfers to correct them.

And if you want to do more complex money movements, you have to compose multiple transfers.

The actual reason why TigerBeetle has transfers and not transactions isn’t just to mess with you, it’s because a transfer is a fundamental building block of the database.

When I discussed QLDB, I casually mentioned that SQL and JSON weren’t supported because “QLDB does away with everything that’s on the way to its difficult goal.”

SQL is a battle-tested way to get and update data from a general purpose database. But it relies on generic data structures meant to store any kind of data type.

TigerBeetle’s most fundamental optimization is precisely here: we don’t need to store any kind of data type. We just need to store movements of money. So why not make that the fundamental data model, and build everything around it?

In fact, all movements of money boil down to transfers, and two more things:

• Two phase transfers: Having the ability to reserve funds (pending), and resolve (post, void, expire) them.

• Linked Events: Having the ability to chain multiple transfers as an atomic block.

TigerBeetle leverages these 3 primitives into a set of recipes that allow engineers to do everything with this database as long as it’s about moving money.

But, like I said, a transfer must happen within the same ledger. And that’s the core of TigerBeetle’s mistake.

Stablecoins, and Multi-Currency Ledgers

How would you send somebody money internationally if you couldn’t use a bank?

Cash over email, or over a handcuffed briefcase might work, but you run the risk that it gets lost. No problem; you get creative. How about you buy shares of Apple, and instruct the recipient to short sell them, and then you can just transfer those shares to them?

That works. That’s actually brilliant. But the transaction costs are probably too high: there’s the difference between the buy and sell price, the broker’s commissions, etc.

Let’s go crypto, then: you buy some Bitcoins, and they sell a bit of theirs, and then you can transfer those Bitcoins to them. The transaction costs are somewhat fixed, and low, so this is an improvement. But Bitcoin’s price is much, much more volatile than Apple’s. The risk of having to offset the price difference makes this approach unwieldy at scale.

The good thing about crypto, and the reason why the transaction costs are so low, is because there’s nothing central about it. The recipient may be on the other side of the world, and they have the same level of access and visibility to it as you. It may be slow, but not SWIFT slow. And it’s way cheaper.

So what we probably need is something that’s crypto-like, to retain the global aspect of such a ledger, but without the volatility of crypto currencies. One way to do that is to have a blockchain that distributes the value of a predefined pool of resources, something that doesn’t change in value against a fiat currency.

So that’s what stablecoins really are. And they’re now facilitating $30bn of transactions daily.

Impedance Mismatch

How would you represent a transfer made from a bank account in USD to another in EUR made via Tether, a well-known stablecoin?1

In TigerBeetle, ledgers are denominated on a single currency, and transfers between pairs of accounts in different ledgers aren’t allowed. So what you have to do is to create two linked transfers in two different ledgers, each one moving money within the same ledger (from or to the “liquidity source”).

Using Tether complicates all this even more. We’re going to need 6 accounts:

• A source account in USD on ledger 1

• A source liquidity account on ledger 1

• A source sweep account in Tether on ledger 2

• A destination sweep account on Tether on ledger 2

• A destination liquidity account in EUR on ledger 3

• A destination account that belongs to B on ledger 3

and a transfer trio:

1. A transfer from the source account to the source liquidity account (USD)

2. A transfer from the source sweep account to the destination sweep account (Tether)

3. A transfer from the destination liquidity account to the destination account (EUR)

Here’s another problem: the exchange rate must be made explicit prior to the transaction. The price risk and any costs derived from the fluctuation of the exchange have to be recorded, as per TigerBeetle’s docs, “as a separate transfer from the source account to the source liquidity account”.

That’s a compensating transfer. Which becomes the fourth in this scenario.

TigerBeetle is built for speed. But makes multi-currency scenarios unmanageable.

Databases are (indeed) Commodities

I’ve been building financial technology for quite some time now, and I’m still as excited as when I started.

TigerBeetle isn’t a failure. On the contrary, it’s the signal that ledger technology isn’t done yet. There’s still so much to do. And that we haven’t nailed down the ultimate financial database yet.

Perhaps we never will.

Most payment engineers think that “rolling out your own ledger” is an unnecessary rite of passage (I already talked about my own), but every financial database that claims to be general purpose I know fails in some particular aspect that makes it work tremendously well for some use cases, and terribly bad for others.

It’s as if rolling out a ledger system isn’t a rite of passage, but the only valid way to build financial technology that truly fits your business.

It sounds absurd, but that brings me a lot of joy. There’s so much that we can do with financial recordkeeping that goes beyond tracking money. We might track a contractor’s hours as inventory. We might see the offices not just as something tangible, but as collateral for a loan, becoming both asset and liability.

We might track who gave us cash, and what we used that cash for.

A ledger then becomes less a cold record of balanced transactions, and more like a map of intent. Relations, obligations, and possibilities find a way into our books, and companies make more informed decisions based on them.

And ultimately, we free ourselves from thinking that only money is valuable, and that valuable can only be represented as money.

I’ll see you next week with another article from The Payments Engineer Playbook.

“Don’t Repeat Yourself” is vague advice.

It’s not that duplicating code is a good thing, no. But multiple generations of software engineers have learnt to avoid writing code, to Keep It Simple Stupid, and to prefer ways of building software that are terse and “elegant”.

This isn’t doing you any good.

Let me give you one example of what I mean: enums.

When we have a set of states in which some entity can be, engineers often choose to represent that status in the form of an enum: a finite set of values that it can take.

An account may be an asset, liability, equity, income, or expense, and most often the codebase includes one object with an attribute status that looks like this:

type Kind string

const (
	Asset     Kind = "asset"
	Liability Kind = "liability"
	Equity    Kind = "equity"
	Income    Kind = "income"
	Expense   Kind = "expense"
)

type Account struct {
	Kind Kind
}

This is probably what your favorite AI would suggest. Your senior engineer would, too!

However, this design choice forces you to verify the kind of your account every time you do something with it. How to calculate a balance, or what to do when you withdraw or deposit take different forms depending on whether the account is an Asset, Equity or Liability, Income or Expense.

The blocks of code that must always be present in order to verify that are called guard clauses.

They take the form of a bunch of return statements at the beginning of a Ruby method, or if statements in Python, or the kind of unbearable mess that Java engineers are now numb to.

Again: engineers don’t even see this as an issue. The links on the previous paragraph even laud the “readability” and the “cleanness” of guard clauses.

Multiple generations have been lost in the pursuit of elegant, non-DRY code.

The problem with this approach is that engineers are more likely to introduce integrity bugs.

This design demands that you use guard clauses all the time. But nothing in the software will tell you which validations you need to do, and what are the consequences of not using them. Engineers who jump into the codebase for the first time risk making real withdrawals with real money because nobody told them that they need to guard against being in production with a guard clause.

Yeah, I’ve seen that happen.

Even the “readability” that these posts claim should be challenged. Using guard clauses, the part where an Asset account is different from a Liability one isn’t apparent in the code; it’s disseminated throughout the codebase, wherever there’s a guard clause. Forming a mental model of what a particular kind of account is about is impossible.

And that feeds into more integrity bugs. Invalid, catastrophic states of your system lurk, undiscovered, silent, and harmful.

A few years ago, Ürgo Ringo from Wise published an article titled Implementing entity states as separate classes, in which he raised the issues with using enums in order to separate entities:

There are three problems with this design:

1. mutability. Even though business concept Order is mutable does not mean we should give up all the benefits of immutability when implementing it

2. it mixes a lot of behavior into a single class and hides away important concepts like difference between a pending Order and a confirmed Order. We want significant business concepts to be more prominent in our code.

3. if we try to do something we should not be doing we will only discover it at runtime when a validation exception is thrown. This is not aligned with the idea of a mistake proof design.

— Ürgo Ringo, Implementing entity states as separate classes

But the solution he proposed is to represent different kinds as different objects. An Account that could be one of 5 kinds becomes 5 objects named AssetAccount, LiabilityAccount, EquityAccount, IncomeAccount and ExpenseAccount.

I don’t like that design either. Yes, I’m done with guard clauses, because which kind of account I’m using is clearly separated from the others. But there’s going to be a lot of duplicated code, because even though they’re different, many of these kinds share functionality.

LiabilityAccounts’ balances are increased with credits, and decreased with debits. But so are EquityAccounts and ExpenseAccounts.

AssetAccounts work the other way around. And so do IncomeAccounts.

I want to teach you a different approach. One that leverages generics.

Generics is just a way to define an extra parameter in the object or function that links inputs and outputs. The change is that you can assume certain inputs to be of a certain type, and enforce that certain inputs are of some type as a result.

For example: you know that adding strings and numbers in Javascript is a mess:

"" + 1          === "1"
"1" + 10        === "110"
"110" + 2       === "1102"
"1102" - 5      === 1097
1097 + "8"      === "10978"

If I create a function that adds two variables, and I’m not careful with using strings or numbers, then all hell can break loose.

I can use Typescript to build a function that adds two items, while ensuring that the two items are of the same type (integer or string) with generics:

function add<T extends number | string>(a: T, b: T): T {
  return (a + b) as T;
}

const s = add("Hello, ", "TypeScript");
console.log(s); // Hello, TypeScript

const n = add(10, 20);
console.log(n); // 30

// Error: Argument of type 'number' is not assignable to parameter of type 'string'.
const invalid = add("Age: ", 25);

It is possible to use guard clauses and get the same result, but notice that generics don’t require writing any extra code in the body of the function. The type system enforces that the two items are of the same type, and thus every other part of the codebase that uses this function is protected.

“But what has this to do with enums?”, you may ask. More than you think.

After nearly a decade building and maintaining large-scale money software, I’ve seen what works (and what doesn’t) about software that moves money around. In The Payments Engineer Playbook, I share one in-depth article every Wednesday with breakdowns of how money actually moves.

If you’re an engineer or founder who needs to understand money software in depth, join close to 2,000 subscribers from companies like Shopify, Modern Treasury, Coinbase or Flywire to learn how real payment engineers plan, scale, and build money software.

Generics are worth it when domain rules are strict, invalid states are expensive and APIs must be hard to misuse. Money software is the textbook example of all three.

In this article, I’m going to show you two examples of how ledgers benefit from using generics.

• The Easy One: How to enforce that money amounts are always positive on Entries, while letting Balances be negative.

• The Hard One: How to separate functionality specific to certain Accounts but not others based on kind.

I’ll do this with generics, without duplicating classes, and without using guard clauses.

Sounds interesting? Then read below.

Read more

In reality, no two bank notes are the same.

When we say that money is fungible, we’re suspending our disbelief for a moment. Of course, I can tell that my bank note is not exactly the same as yours. But we go along with it because, well, everyone else is also playing pretend. To the point that we even forget it.

Fungibility is such a pervasive belief; I would have to pause for a moment to even consider that it isn’t really true.

This, however, is problematic for engineers who handle money. We’ve embedded this white lie into the software. And, sure, most of the time it doesn’t do any harm. You’ve probably never encountered any such situation where having to differentiate two bank notes was needed.

But that’s probably because you only handle money online.

I was watching a talk by Formance CEO Clément Salaün the other day, called The color of money1. In the talk, he argues that assuming that all money in a ledger is fungible is a bad idea.

Say you accept cash at your Starbucks joint in exchange for topping up a gift card. We’ve already covered what a great idea it is to have gift cards, or store credits, and Salaün goes as far as calling it “the best business in the universe”.

But there’s a catch. Days later, when you deposit that cash, along with everything you’ve earned from selling Macchiatos and Frappuccinos, your bank rejects some of your bank notes because they’re fake.

You thought you were getting Washingtons, Jeffersons and Lincolns, and in reality you were getting Monopoly money.

Soon, you’ll realize that you actually have two problems:

• You don’t know what amount to subtract from which gift cards

• You don’t know which sales to cancel

Because you mixed the legit and the fake money, now you can’t tell which one is which.

The solution that Formance CEO offered is actually pretty clever: make fungibility intentional.

Intentional fungibility means to trace the payer of every bank note the same way you trace its amount and currency. That gives you the choice to treat those bank notes as if they were any other bank note, but retaining the ability to trace back who gave them to you.

This, in other words, means treating bank notes from different people as not directly balanceable with other people’s notes. They’re currency from a different country, so to speak, and only become truly fungible when you decide so.

With that tiny bit of extra information, you retain your sanity when it comes to fake bank notes, even if, for a brief moment, you have to suspend the widespread notion that all bank notes are created equal.

I find this idea very interesting.

But I don’t usually deal with physical money when I build software. For me, it’s usually credit cards, APMs and bank transfers. When I see cash, it is rare, and in the form of Cash On Delivery.

Fraud exists online, but it’s not in the form of fake bank notes.

Does that mean that this idea is cool, but pointless in the online world? Is this another idea that’s clever, but wasted effort?

It’s not. Making fungibility intentional introduces a whole new approach to money, even online.

This is an idea that changes everything. And I’m not even sure if the people at Formance are even aware of.

I’m Alvaro Duran. After nearly a decade building and maintaining large-scale money software, I’ve seen what works (and what doesn’t) about software that moves money around. In The Payments Engineer Playbook, I share one in-depth article every Wednesday with breakdowns of how money actually moves.

And I show you how top fintech companies build their systems.

If you’re an engineer or founder who needs to understand money software in depth, join close to 2,000 subscribers from companies like Shopify, Modern Treasury, Coinbase or Flywire to learn how real payment engineers plan, scale, and build money software.

This week, we’re looking at how money gets represented in software. You probably use one of those open source, third party packages that are now standard in your language: Go’s go-money, Python’s py-moneyed, Ruby’s money, whatever.

This is fine (at least you’re not using floats to represent money).

But you’re leaving a lot of information on the table. All of these libraries work under the assumption that all money is fungible. And we’ve just proven this to be limiting, and sometimes harmful.

In this article, I’m going to argue that:

• That you should build your own money library

• That building such a library isn’t that hard, and very useful down the line

• That using a third party money library is akin to installing left-pad

Enough intro, let’s dive in.

Read more

It’s been one of the most rewarding years of my life.

I’ve been in William Goldman’s camp since he said that “every time out it’s a guess and, if you’re lucky, an educated one.” As 2024 was coming to an end, and propelled by an overwhelmingly positive reception of The Payments Engineer Playbook, I decided to make it a paid publication.

Since then, not only the newsletter has continued to grow, now shy of 2,000 overall subscribers, but a strong number has decided to join me in the goal of becoming better engineers of money software.

I wasn’t sure if I was making the right move. Turns out, content that’s well-researched and specific to payment engineers is hard to come by, and it’s worth paying for.

But The Playbook is also niche in another important aspect.

Most stuff you read or watch online is targeted to the occasional Internet wanderer. Tantalizing thumbnails, clickbaity titles, and emotional but ultimately worthless content.

This newsletter, instead, speaks for the regular reader.

With more than 100 articles in the archive, I’ve written enough to fill a couple of door-stoppers. You can read through them all you want. But the most valuable thing that The Playbook provides is the personal growth that you gain in the 10 minutes a week of reading.

I’ve explored many aspects of the job of payment engineers: databases, event sourcing, orchestration, ledgers, and more. I’ve published two long-form series (code-first reliability and from stock markets to ledgers), and I plan on doing many more.

Yet, the newsletter format is the perfect fit for The Playbook. It’s transformative content.

10 minutes a week, 52 weeks a year.

This year, The Playbook published 50 articles, 17 of them free. They all dive deep into some particular aspect of building software for the payments industry, what I call money software.

This week, I summarize the most popular articles of 2025.

2025’s Most Viewed Articles

These are the 3 most popular articles on The Playbook.

4.55k reads

Deemed “one of the clearest writeups I’ve seen on where payment systems quietly break” by Reddit, this post was widely shared in the fintech community, way beyond those building payments.

3.73k reads

I was called a “clown” for suggesting that you should deploy every day, even on Fridays. If you side with them, get ready to be proven wrong.

2.61k reads

Another Reddit blowup. Consensus hasn’t been widely adopted because what we used to have (Paxos) was inscrutable and therefore impossible to build correctly. Raft is proof that correctness isn’t always the only thing that matters. Leveling to the average developer can do wonders for adoption.

2025’s Most Viewed Paid Articles

These are the 3 most popular articles that sit behind the paywall.

2.17k reads

Your payment system probably started by accepting cards only. Eventually, it adopted a few of those trendy methods, and now that you’ve got a few providers integrated, the code is a mess. Why? Because you didn’t build a taxonomy for it. This article dives into this problem and provides a comprehensive solution.

2.03k reads

What’s more important than choosing the right database for your payment system? Having the ability to switch if you chose the wrong one.

1.9k reads

My most referenced article in this newsletter, in which I look at ledgers from the point of view of Availability-only and Consistency-only, and what are their implementation differences.

Meet, Assess, Inspire

I’ve been exploring ways to turn The Playbook into a productized service. Something a bit more hands-on and impactful.

The idea is to spend a full day onsite at your company’s office, getting to know the team, digging into the product, and advising on what’s going on and how to take it to the next level. So that at the end of the day, I can wrap everything up with a talk designed to spark new ideas and inspire the engineering team into action. All powered by the learnings of this newsletter.

I’m calling it Meet the Leaders, Assess the Product, Inspire the Team.

I’m testing this format on 2026 with one adventurous company, and I’d love to meet others who resonate with this format. If that’s you, hit reply to this email and let me know. Even if it’s just a quick chat over DMs or something more elaborate, I’ll be happy to get to know what you do and what you enjoy about money software.

That’s it for this year in The Payments Engineer Playbook. I’ll be back on the 7th of January.

Merry Christmas!

It’s not hours of practice, or experience; Sebastian Vettel and Lewis Hamilton won the title when they were 23. And while having a father that used to be an F1 driver might help (like Max Verstappen, Nico Rosbergand Jacques Villeneuve), it didn’t help Nelson Piquet Jr., nor any of Jack Brabham’s sons Geoff, Gary and David.

It’s not money either. Ask Lance Stroll.

Being the occasional F1 tifosi, this is the question that I ask myself throughout the season. Is it you, or is it your car that makes the championship? Was Lando Norris really this year’s best F1 driver?

64 out 76 times, the driver that won the title was racing the car that won the Constructor World Championship. That’s more than 84% of the time. So there’s strength in the argument that the driver, in practice, doesn’t really matter. That Adrian Newey matters more than the drivers who race with one of his cars.

It might. But even if you have the best car in the grid, you have to at least be faster than your teammate.

And if it isn’t experience, nor the guidance of a father who was a driver himself, nor money, then what?

It must be the driver who can squeeze the most of the car he’s got that separates the good drivers from the championship material.

It must be the mechanical sympathy.

You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.

— Jackie Stewart, 3 times World Champion

Mechanical Sympathy is an innate feel for the car. It’s being able to break at the right moment at every turn, considering the tires wearing out. It’s being aware of everything, including checking the flag on the side for wind direction.

If there’s such a thing as mechanical sympathy for engineers, most of us don’t have it. Even worse: many might even believe they have it, but based on notions about the underlying technology that are not true, or no longer so.

Which is why this article, hell even this whole series, came to be.

Over the last month and change, we’ve looked at stock exchange technology from the point of view of ledgers practitioners. In From Stock Markets To Ledgers, we’ve looked at how the systems at NASDAQ, NYSE or Euronext work, not to understand them completely, but to apply some of its takeaways to the building of ledgers.

And in this article, the last one in the series, we’re coming full circle. We’ve seen why fairness is more important than speed, why fault tolerance is about understandable code, and not simply correctness, and the four notions of time. We then looked at why AMQP failed, and I encouraged you to deploy to production every day with a CrossFit analogy.

The overarching theme behind this series is this: you don’t get to make better software products by mindlessly using what already exists, or copying what already works.

The moral of the stock market’s technology story is: the next generation isn’t built as a continuation. It’s a pivot.

Those who have a superficial understanding of the tech stack are forced to follow the rules. They’re vibe coding, even if they don’t use AI. Only those who understand the tech stack and its rules can change former and break the latter.

This is the path of those who want to achieve mechanical sympathy.

In this article, I’m going to look at 3 books as a way to approach the process of achieving mechanical sympathy.

• Why I chose these books over everything else (other books, online tutorials, courses, etc)

• How I approach the material

• What each of these books represent so that you can pick up your own reading stack

The books in themselves don’t matter as much as the process. Your path may be different, and so will be the books that you pick up. But feel free to start with them, and see where that leads you.

Let’s dive in.

3 types of books

Valuable books fall into three categories:

• DIY books

• From practitioner to expert books

• Exotic topic for the uninitiated

If you have a book that doesn’t fall into any of these buckets, throw it away.

I’m serious about this. We’re surrounded by books promising results in 21 days, or a week, or even in an hour.

Such books are based on the idea that feeling good is indistinguishable from being good. Deep down, you know it’s not. But it’s often hard to know the difference.

So here’s my foolproof heuristic for figuring out if a book is good: if it doesn’t feel like fighting Bane from The Dark Knight Rises, then it’s not valuable.

A good book is a wonderful way to increase your confusion and your stress levels.

Especially when it tackles a problem you think you know well. Good books go right into the kind of questions you never thought of asking yourself, and proceed to explain why you didn’t ask them, and what are their answers.

Good books are also practical. After reading them (really reading them), you’re a different person. Good books are akin to Neo’s kung-fu training in The Matrix.

A kick in the butt.

I am a student of money software. This means that I’m the most valuable when I learn from books about distributed systems that must be available enough while strongly consistent, provably correct, and auditable.

Which brings me to the three books I want to talk to you about.

Read more

This one is going to sound like a pep talk.

I seem to find no way around it: whenever I argue that you, or your team, should be deploying code changes to production every day, I get crap for it. “It’s because you have no idea what my team is doing”, they say, “and that’s why what you’re saying doesn’t apply to me. Clown”.

So I’m going to try a new approach: Andrew Hiller’s.

If you don’t know Andrew Hiller, he is a youtuber in the CrossFit space that became famous because he was calling out famous athletes for breaking the rules.

This is a very controversial topic. Every year, thousands of athletes enter the CrossFit Open to become one of the 40 men and 40 women who qualify to compete at the CrossFit Games. But this doesn’t happen in a central venue. Athletes submit their scores, along with videos of them doing the workouts. Some judge is meant to validate the score, and the top athletes go to compete in the Games.

This is all public, and for a long time nobody paid much attention to the thousands of videos uploaded on Youtube. Probably not even the judges either.

Enter Andrew Hiller.

CrossFit, at least in theory, is a sport in which standards are very strict. Its main youtube channel is filled with 1-minute long videos explaining even the most basic of movements: the burpee, the lunge, the pushup, the squat. Even the box step up, which is exactly what you think it is, has a video.

In the burpee, you’re meant to jump at the end. In the lunge, your knee has to touch the floor, and then extend on top. In the squat, your hips must go deeper than the knee. And so on.

In practice, though, people tend to be very lenient when it comes to call a rep valid or not.

So what if I just squat deep enough for hips to go almost as deep as knee height? That’s a normal train of thought when you’re doing 100 of these.

Plus, you’re not competing to win anything, right? So why be so strict? What’s the point of calling out 65+ year olds who just want to be part of the fitness community? Shouldn’t we celebrate the fact that they’re working out instead of sit on their couch all day?

Hiller has none of it. When you step into the competition floor, the rules are The Rules.

The reason CrossFit is very strict when it comes to movement standards is because one of the main tenets of the sport is being able to accurately measure your own improvement. In CrossFit, you don’t do legs Mondays and Thursdays and upper body Tuesdays and Fridays; every day is something different. Workouts are also scored: you’re competing either to get a certain amount of reps for time, or to get the maximum amount of reps on a specific time span.

Eventually, you get to repeat the workout, and that’s when you compare your old score with the new one. CrossFit’s promise is that the newest score is going to be better.

But if you’re cheating the movement standard, you can’t be certain of your actual progress.

Say you’re running a marathon. But you’re allowed to use your bike at some random point, for a random amount of time. And then, you get to do that again six months later.

Would you be able to tell if you’re a faster runner?

You wouldn’t. That’s why Hiller is calling people out.

But the most important reason for doing so is because the movement standards are more difficult, and demand more mobility and strength from you. Over time, especially if you’re that 65 year old lady, and you don’t squat deep enough, you're training your muscles to get shorter, because they don’t get the right stimulus.

With every no rep, you’re not only cheating yourself. You’re actually harming your body.

This, however, isn’t a fitness newsletter. And I’m not trying to convince you to become a Crossfitter. The reason I’m bringing up Andrew Hiller is because I see tons of parallels between the excuses of people who don’t follow CrossFit standards and the engineers who don’t deploy their code changes every day.

“Finally”, you might be thinking, “I see where all of this is going”. Even if you don’t agree with it.

So let me cut to the chase: you might not be uploading a video of you saying that you’re not going to deploy this Friday because, well, it’s a Friday.

And I’m going to say that you’re only embarrassing yourself.

You’re the old lady who can’t squat.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. I’m not kidding, this is a newsletter targeting, not CrossFit athletes, but software developers building the systems that move money around.

This article is part of the series that I’ve called From Stock Markets to Ledgers, where I’m tracking a few fundamental ideas that people building stock exchange technology have figured out, but the wider fintech community hasn’t.

And this week, I’m talking Continuous Delivery. Which, to me, it just means “deploy to production every day”.

In this article, we’re going to look at

• The 3 most common excuses people often make to avoid deploying daily

• All the counter arguments

• And one all-encompassing reason why you haven’t acted on any of this yet.

Unlike the Accelerate book, I’m not going to “appeal to science” by mentioning surveys and statistics. You’re probably familiar with this line of research (even if vaguely), and it hasn’t moved the needle in your world.

Instead, I’m going to follow a line of reasoning similar to Andrew Hiller’s: I’m going to lay down why you’re embarrassing yourself with lame excuses, that deep down you know they’re excuses. And that, like people who don’t squat deep to go faster, you’re actively harming your personal growth in the process.

If you want the softer, free-to-read version, I wrote about how Stripe deploys 1,000 times every day.

But if you need something stronger, then let’s dive in.

Read more

Messages are atomic pieces of data. When you send messages, the recipient either gets them completely, or they don’t. What’s useful about this is that messages can be queued, which releases a lot of pressure on the receiver about the way these messages are processed. Queues also decouple the sending and the receiving of the messages, which is particularly helpful for systems that communicate a lot, but belong to different companies.

Banking systems are almost always designed this way, and that’s why the Advanced Message Queueing Protocol project was started by a bank consortium.

This being a series on stock exchange systems, you shouldn’t be surprised to learn that middlewares have historically been the main performance bottleneck. Processing north of 500K events per second and executing timely makes it impossible to use TCP because it comes with built-in safety mechanisms that affect throughput and latency.

[A] large part of the uncertainty and the variation that comes from TCP are these self-learning algorithms that are very concerned about network health. When we would work with the Linux kernel maintainers and have questions about variability that we saw, then they would say, “Well, you shouldn’t be using TCP. If you care about latency, you shouldn’t be using TCP. TCP makes trade offs all the time, for network health, and so on and so forth. And for the Internet that is absolutely necessary. And if you really have these super tight requirements, and you really want them to get there fast, and you have a controlled network with very little packet loss, and very few layers between participants, you should be using UDP.”

— Brian Nigito, Multicast and the Markets

Over time, banks have managed to create open technical standards to do business. You may have heard of FIX (Financial Information Exchange), which is used regularly in trading, but it’s so complex that you often need some tool to generate the message for you. Same with FAST (FIX Adapted for Streaming), FpML (Financial products Markup Language) and SWIFT1: these are terrible protocols to work with, and the only reason people put up with them is because there are no widespread alternatives.

Which begs the question: why not? Well, banks have quite a bad reputation when it comes to technology. The whole reason fintech is a thing is because, for the longest time, banks have considered technology a “non-core competency”. Tech is for nerds. Engineers are the water boys of the football team.

And then, all of a sudden, they weren’t.

It’s 2003. John O’Hara, then Distinguished Engineer at JP Morgan, has had enough. His job is to build front- and back-office processing systems, and it seems to him that banks live in “integration Groundhog Day”.

Every discussion on making things better ended in “the chosen middleware makes it really expensive”.

The issue is, of course, interoperability. Or the lack thereof. It’s the same issue that Modern Treasury’s CEO raised in Bad Software and Expensive Mistakes when he talked about that time Citigroup “accidentally” wired $900 million and then couldn’t get them back:

Building on these interfaces takes investment of time and resources, because bank cores are challenging to integrate with and understand. Some banks use Direct Transmission over SFTP while others provide SOAP APIs. Timings vary. There are different, competing, data exchange formats for things like reporting on errors, and the configurations can vary from banks on identical cores.

When pressed for resources, teams cut corners. It’s hard to justify investing in wonderful design and robust systems for internal processes, so much of what’s out there is half-built, inflexible, and non-intuitive, and rarely gets any attention from the design team.

The result of poor software is workarounds.

— Dimitri Dadiomov, Bad Software and Expensive Mistakes

Having waited 7 years for someone to do something about it, O’Hara took the matter into his own hands. Initially, as a JP Morgan-only initiative, then through a contracting company called iMatix.

Two years later, in order to increase adoption, JP Morgan approached all the right technology companies (Cisco, Red Hat, Microsoft) and the right banks (Bank of America, Goldman, Credit Suisse) and created a working group that included people from 23 different companies.

And quickly, things started to decline.

In 2008, the CEO of iMatix published What is wrong with AMQP (and how to fix it), highlighting the problems derived from too many cooks in the kitchen:

The specification seems to be out of control, getting more and more complex without a clear plan or architecture. AMQP products don’t interoperate except in some limited cases. Core protocol aspects are unstable, and more ambitious functionality not happening. The Working Group is getting tired, and many of the original developers have burnt out.

Today, most of my team at iMatix has written off AMQP as a “failure”.

— Pieter Hintjens, What is wrong with AMQP (and how to fix it)

By then, iMatix had “soft abandoned” the AMQP project, and had started working on a better alternative. One that turned all the assumptions embedded in AMQP upside down.

They called it ZeroMQ.

A Newsletter For The Engineers That Keep Money Moving

Designing payment systems for interviews is easy. Designing them for millions of transactions is not.

After nearly a decade building and maintaining large-scale money software, I’ve seen what works (and what doesn’t) about software that moves money around. In The Payments Engineer Playbook, I share one in-depth article every Wednesday with breakdowns of how money actually moves.

If you’re an engineer or founder who needs to understand money software in depth, join close to 2,000 subscribers from companies like Shopify, Modern Treasury, Coinbase or Flywire to learn how real payment engineers plan, scale, and build money software.

Subscribe now

This series is called From Stock Markets to Ledgers. We’re looking at the technology that powers NASDAQ and NYSE in order to build the software that moves money around. Faster, more performant, more correct.

This week, we’re looking at ZeroMQ, a protocol for message communication that, like Raft, looks at the problem at hand from first principles. In this article, we’re going to look at:

• Why ZeroMQ is a library

• The tradeoff between throughput and latency

• And a big takeaway to build ledgers with better throughput

Enough intro, let’s dive in.

Read more

Quantum mechanics’ main tenet, the idea that an electron can be both a particle and a wave, is actually quite intuitive. And there’s plenty of real-world analogies that make it familiar, even obvious.

Let me give you a very simple one: language.

Most European languages divide concepts into (roughly) two camps: nouns and verbs1. There’s oil and water, and you can run and cook. These are two separate camps that describe things in the world, and actions being performed.

But you can also oil your car and water your plants; and you can go for a run, and work as a cook. There’s a fluidity in the language where the action being performed and the object related to it revolve around the same concept, and both noun and verb are a manifestation of being the thing and doing the thing.

You milk a cow to get milk, after all.

Let me give you another: the economy. Money can be exchanged for goods and services: you can buy peanuts, and you can hire a VA to handle your email. But you can also pay a Netflix subscription, which is a product that consist of being able to watch movies on demand, or you can pay a consultant for one of their productized services. You can DoorDash food to your place rather than having to go to a restaurant (and DoorDash is now a verb, see what I did there?).

Money is the abstraction of being able to buy, and it becomes concrete in the things we buy: be it goods, services, or anything in between.

You probably see where I’m going with this, but let me give you one last example: your body.

When you get a blood test, the results tell you, among other things, an estimation of how many white cells there are in your body, and of which kind. This tells you a lot about your immune system, and could signal a lot, from having a cold to AIDS or leukemia.

But the thing is, the number of white cells in your bloodstream is pointless. What matters is that they’re there to do something. The presence (or absence) of white cells is meaningful to the extent that white cells are produced by your body to protect your organism from external agents. The only reason we count them is because it is impossible to see white cells in action, and we can only infer what’s going on from counting them. The presence of white cells and what they’re doing are two manifestations of the same thing: a bodily response to the environment.

Electrons being particle and wave is analogous to all that. What we perceive is an incomplete manifestation of a whole. We milk to get milk; we buy things and jobs with a number in our bank account; we have white cells to defend ourselves.

Objects are tangible manifestations of state. We see how they are now, and we see them doing things. Processes are elusive manifestations of time. We know that something has happened because the state is different from what it was before.

We perceive time by noticing that things aren’t the way they used to be.

Quantum mechanics isn’t Tenet levels of hard2. It’s just that humans are really bad at thinking about processes.

When I was talking about money, was it simpler for you to think about buying stuff, or hiring somebody? Doesn’t it feel like astronomy is more about stars and planets than it is about supernovas and the gravitational pull? Isn’t your body simpler to think about in terms of organs rather than hormones?

Yes, yes, and yes. Humans unconsciously prefer to think about the world in terms of objects, rather than processes. Structure, rather than behavior.

The same exact bias happens, of course, to software engineers. And that is a pity, because stock markets, and ledgers, aren’t about objects. They’re about processes.

A Newsletter For The Engineers That Keep Money Moving

Designing payment systems for interviews is easy. Designing them for millions of transactions is not.

After nearly a decade building and maintaining large-scale money software, I’ve seen what works (and what doesn’t) about software that moves money around. In The Payments Engineer Playbook, I share one in-depth article every Wednesday with breakdowns of how money actually moves.

If you’re an engineer or founder who needs to understand money software in depth, join close to 2,000 subscribers from companies like Shopify, Modern Treasury, Coinbase or Flywire to learn how real payment engineers plan, scale, and build money software.

Subscribe now

This article is Part III of the From Stock Markets to Ledgers series, where I’m going deep into the design and implementation of stock market systems, not only because it’s so interesting and fun, but also because there are lots of good lessons for those building ledger systems.

In today’s article, I’ll focus on one interesting aspect of stock market system design: how central time is, and how bad humans are at reasoning about it.

We’ll look at:

• What HR organizing the Christmas dinner and Apache Flink have in common

• The 4 notions of time (you’re probably storing only one)

• The trade-off between quickness and completeness

• Two useful approaches to engage in it: watermarks and snapshots

• And a useful takeaway for ledger builders

Enough intro, let’s dive in.

Read more