Why Low Cardinality Accounts Are A Ledger's Bottleneck

You can build a ledger that’s blazingly fast with a slow programming language.

The trick is to realize that improving the overall performance of the ledger doesn’t really move the needle. What moves the needle is preventing your biggest bottleneck from slowing everything down.

A particular kind of obstacle placed in a particular place in the ledger can make any ledger glacially slow, even if it’s written in C++.

So where’s that bottleneck?

Every time you add a new transaction, most ledgers lock for writes the accounts involved. “That’s the only way”, you may think, “to prevent race conditions.” With those locks in place, you can now check the balance, make all the validations you need, and then proceed with the changes. Finally, the locks can be safely removed.

Here’s the thing: one of those locks may not be needed.

There’s a nice little trick that most engineers won’t notice because they treat all accounts as if they were equivalent.

They’re not.

I’m Alvaro Duran, and this is The Payments Engineer Playbook. You’re already subscribed to free newsletters that “teach” you how to get a job as a software engineer.

But you don’t want to get a job; you already have one. What you want is to learn how to get you promoted at your fintech job. Where stakes are sky high, and the margin for errors is razor thin.

In The Payments Engineer Playbook, we investigate the technology that transfers money in order to make you an outstanding engineer, and move your career forward.

This week, we’re going to discuss the impact of cardinality in ledgers. I’ve put together a valuable list of concepts that payment engineers have to be familiar with, all for free, and cardinality is one of those concepts.

In this article, I’m going to explain:

• What are high and low cardinality accounts

• What is the impact of cardinality in ledger performance

• How to leverage this insight to make your ledger even faster

Enough intro, let’s dive in.