Pyments: How to Design Payment Applications in Python
Why Building Payments, Why Doing it in Python, and How To Do It Effectively
Editor’s note: This talk was delivered on March 2024 at PyCon Slovakia. The video can be found here.
Do you think your work feels like an adventure?
I suspect you don’t.
How about this: do you believe that the code that engineers at your company write is beautiful?
I asked this question while presenting this talk at PyCon Slovakia last week, and the answer was a decisive “no, haha, are you kidding?”.
In this post, I’ll be talking about building software that collects payments. But I’ll also be talking about ownership, and beauty. Those are the key ingredients of effective software, regardless of its purpose.
You might be wondering why that is important for this talk.
To me, it is, because when I tell people that I build payment applications, they always ask me this:
Why Don’t You Outsource To a Platform?
It’s just a few lines of code, right?
And to be fair, small businesses should be fine doing this. If your company operates at a small scale, and you’re not technical, outsourcing payments to a platform is probably a good idea.
But you see, you ARE technical enough. Even if you are a solopreneur, this no longer applies to you.
Why is that? Well, first, because every company makes money by giving their customers the opportunity to give it to you. Handing THAT to someone who may not have your best interest at heart is a recipe for headaches and problems.
And second, because every API I’ve integrated with has ended up being less than satisfactory, and less than effective than it looked like in the beginning. And, in time, it had to be painfully migrated from, no matter how “platform agnostic” they claimed to be.
Every integration has a disintegration in the horizon.
If that’s not enough to convince you, consider this: do you feel that writing that bunch of code would make you a better engineer? That delegating, rather than building, is the mark of good craftsmanship?
Cause I believe it doesn’t, and it isn’t. Even the great artists of our time know that craftsmanship is not about cleanness, but about getting your hands dirty.
Craftsmanship is tied to feeling that your company is successful, and you contributed to it directly.
For us, craftsmanship means writing beautiful code, but also owning it.
That is how work feels like an adventure.
Welcome to Money In Transit, the newsletter bridging the gap between payments strategy and execution. I’m Alvaro Duran.
Recently, we’ve looked at the perils of building payments ignoring domain experts, an actionable way to classify payment methods, and what makes a good programming language for building payment applications. They’re all free to read.
This one dives deep into all of that, and more. I’m not pulling punches today, and I’m confident that you’ll learn a lot from it.
Want to be notified when there’s a new post? Smash that button below as if you were a heavyweight champion.
My name is Alvaro Duran, and I engineer payment applications at Kiwi.com, the leading global travel tech company headquartered in the Czech Republic.
Over the last year, we’ve worked on redesigning our payment application, fully in Python. We now give support to new payment methods with ease. That helped us give travelers the ability to pay the way they want, at scale.
In this post, I’ll show you how we did it.
There’s a dual nature in payments that Python is well positioned to address. But to make that obvious, I need to explain why payments are deceptive to build, and where engineers tend to make mistakes more often.
First of all, regulation. The payment network imposes very specific rules on how money movement should be handled. To connect directly to Visa, for example, you need to apply for a special membership that is time consuming and very expensive.
Instead, most companies choose to partner with intermediaries called Payment Facilitators, or PayFacs, which provide a lighter process. This is what has made modern e-commerce possible, but the downside is that you’re dependent on their API to accept payments.
Regulation means that we’re almost always going to integrate with at least one provider.
And when I say “at least one”, it’s because, as your company scales and starts doing business globally, you’re better off partnering with multiple providers. This becomes a “benefits buffet”, where every payment that goes through your system is routed to the provider that costs the least, or gives you the best acceptance rate.
If you want to go global, and most Internet businesses do, then integrating with multiple providers is a must.
But what trips engineers the most is the belief that payments online are card payments. That is not true. But this shouldn’t be news for you. The payments world has changed, and is changing, fast.
I suspect you’ve heard of PayPal or Revolut Pay right?
Here’s the thing: PayPal and Revolut Pay are not card-based payment methods. In both cases, you’ve got a balance that you can top up, and you can use that balance to buy things online.
And even if cards are still everywhere in countries like the US, they’re very expensive for the businesses that offer them. This is why some stores require a minimum purchase for customers who want to pay with a card.
However, if you go to Youtube and search for payment system design, you’ll find that the design is heavily card-centric. There is a disconnect between engineers who design payment applications for the simplest case, and how people behave nowadays.
Globally, everyone’s habits when it comes to money has radically changed, in no small part because of Covid-19. And there’s no way that the shift will stop there.
Engineers who choose to focus on cards only, nowadays, are fools.
The key to building effective payment applications is flexibility. Flexibility to change providers under strict regulation. Flexibility to adjust the implementation so that the payment gets routed to the optimal provider. Flexibility to accommodate every innovation in the payment landscape, at pace.
A beautiful challenge.
So why is Python well positioned to address this challenge? Because not all flexibilities are created equal. Total flexibility is bad, because Move Fast and Break Things has never sat well in any financial department. Payment applications must be flexible when it comes to the specifics of each payment method, but you don’t want them to be too flexible to be unstable.
After all, we’re talking about money.
Whenever you interact with a payment application, you want things to remain consistent, invariant and stable. The purpose of payment applications is to hide the nitty gritty details of what it takes to pay, right?
This fear of instability is behind the adoption that Java enjoyed in finance in the 90s, under the promise of Write Once Run Anywhere.
Java belongs in the camp of engineers who reject the Move Fast and Break Things by choosing to Moving Slow and Leaving Things Alone.
But that’s not the camp we’re in. If we tried to build payment applications in Java, we wouldn’t be able to adapt to the rapid change that the payments industry is experiencing. Flexibility is not idiomatic in Java. It’s too verbose, and if you ask me, it’s not beautiful.
On the other end, there’s Ruby. Lots of companies have succeeded using Ruby, at least during the early stages, because it provides a great deal of flexibility.
However, it’s been obvious that there’s a subset of problems that things like type systems and other stability features are poised to address more beautifully.
Rubyists though have more or less rejected that idea, trying to preserve a sense of playfulness at the expense of effectiveness.
Python strikes right in the middle of these two approaches. It’s flexible enough to move at pace, and has been refined by its widespread adoption. It now supports many desirable features for building stability into your system.
Python, and this is something that other programming language communities underplay, benefits tremendously from network effects. As far as open source libraries go, we Pythonistas are used to a level of support that is unheard of in other languages.
Pythonistas are the biggest pool of developers when it comes to language adoption, and it shows. That’s why, when a provider builds an SDK, they tend to build it for Python before any other programming language.
Python is therefore the Yin and Yan. It helps engineers Move Fast Enough, Improving Things Over Time.
It’s the Perfect Payments Language.
It bears repeating: there’s a dual nature in payments: what keeps evolving, and what remains stable. If we want to design payment applications effectively, we must therefore build them in two layers. A stable layer at the surface, and a flexible layer on the inside.
And importantly, these two layers must be related only in one direction.
What I mean by that is that the flexible layer knows about how the stable layer is built, but not the other way around. Abstractions should never depend on details.
The design should not only favor this relationship, but it also must prevent it from going the other way around.
For that reason, inheritance doesn’t work here.
That’s because inheritance allows for a bidirectional relationship between the parent and the child class. Having children “being a” parent means that they gain access to everything that belongs to the parent, polluting the code and making local reasoning impossible.
This is a mess! And therefore, we must employ composition.
This is the Strategy pattern. You can read all about it in its Wikipedia page. The key here is to notice that the parent “has a” child: the subclass is an attribute, and doesn’t have access to anything that belongs to the parent, whereas the parent can simply invoke the child and access everything it’s got.
If you’ve done anything with React, this may sound familiar.
In this example, two different strategies (sorting and reversing the list) are applied consistently by the same Context class using the same method. The Context doesn’t change, and the meat happens inside the Strategies.
How does that manifest in payments? Well, surprisingly, the Strategy pattern isn’t flexible enough.
When it comes to payment methods, copycats are everywhere. Can you tell me what are the differences between Apple Pay and Google Pay? Believe me, you have to implement both to find out. Many parts are just the same, but some things like the structure of the requests ad responses, the endpoints to call, and so on, are different.
If I wanted to implement this Strategy pattern for Apple Pay and Google Pay, how should I go about that?
Should I have a single strategy per payment method, and give up the obvious opportunity for code reuse? Or should I have a CompanyPay strategy of sorts, where I pass an argument that will explode into who knows how many “if” statements inside that class?
Didn’t I say that I want to write beautiful code? Neither of these options is beautiful!
So how can we solve this? Well, we can relax the boundaries of what a Strategy is. Instead of forcing the strategy to be a standalone class, we replace the strategy class for a list of classes that we can call Blocks.
Like a block chain? I surely wished the name wasn’t taken already.
Now the PaymentFlow has an attribute “blocks”, which is a list, and the method is meant to loop through all of the items inside that list, so that it calls run to each one of them.
Each Block can be a class that implements a “run” method and does whatever it needs to do, specifically built for the task that is meant to do.
Notice how easy it is to read the first half, and then the second half, as if turning a page from a novel.
The PaymentFlow is the context, and each instance IS A strategy. Depending on the blocks that you pass onto it, you get a different result.
Isn’t that beautiful?
We can now build lists of blocks for Apple and Google Pay, each one of them having parts that are common to both, and parts that are unique to each of them.
We no longer talk about strategies, because depending on the method you invoke from PaymentFlow, you will get a different one.
So the question now is, given that the PaymentFlow doesn’t change, and the methods defined inside PaymentFlow don’t change either…how do you define the PaymentFlow?
This only works if it remains stable across all payment methods. Those existing, and those that haven’t been invented yet.
And not that we’re running out of options here. This is what you get when you search for “payment method ecosystem”: word salads, tiny logos, a Cambrian explosion of ways to move money around.
Is the PaymentFlow that we want to build even possible? Isn’t some corner case out there going to bite us?
How Can We Define Payments in terms of Technology?
So my answer to this is that it is possible. The reason is that we figured out a way to classify payment methods that not only is simple enough for me to spend the rest of the talk talking about, it is also exhaustive.
Every payment method that exists, has existed, or will exist can be placed somewhere in this taxonomy.
Curious to see how? OK here we go:
A Payment is a Promise made by an Authorized Party. I chose those words carefully because both the concepts of Promises and Authorization are actually computer science concepts. One is related to Sync and Async communication, and the other is related to Identity Management.
And that means that we can place all payment methods along two axes: whether the customer needs to perform some action to become an Authorized Party or not, and whether the Payment finalizes synchronously or asynchronously like Promises.
Like good engineers, we’ve turned an impossible task, figuring out what’s common across all payments, into a tractable problem: figuring out a way to manifest these two dimensions in the code.
Let’s start with identity management. How do we make sure that both the with and without customer action flows can be supported? One possible way is to make the pay method private.
We design the application so that the pay method can only be called in one of two ways. At initialization, where every payment goes first, the block can give us three kinds of results:
Success, in which case the pay method gets called directly
Failed, and the method will instead return the failed response
And also Requires Action, which is special in the sense that it returns, but inside the response there’s some metadata for the client to show to the customer, so that they can do whatever they have to do.
Then, if the actions are completed, some other action data will be sent in the request to another method, let’s call it process_actions, which will be either successful, and therefore will lead to calling the pay method, or it will fail.
This is elegant because sometimes you have no idea whether the provider is going to require some action from the customer. If you pay with a card in Europe, you’re likely to be redirected to your banking app. If not, then that rarely happens.
This is the perfect segway to talk about sync vs async flow. You see, usually the customer isn’t immediately requested to complete the action. More often than not, the provider is the one asking the customer to complete the action in a banking app, out of your control.
Your payment application would hang, because now your user has to go to that app, spend some time there, and return, if they ever do.
Thankfully, the provider won’t leave you hanging. Instead, it does what everyone would expect. It returns a response with status pending, and will call the callback url of your choice with enough data for you to acknowledge that the payment went through.
In order to accommodate both flows, we can split the pay from the process_pay. When you call pay, you may go immediately after to process pay, if the flow is synchronous.
And you will have to return, and wait for the provider to call the callback url, which will trigger the process_pay method, if the flow is asynchronous.
Now that’s the theory. In practice, the provider will be messing with you, big time. Because these providers have agreed, contractually, that you will be notified of 99.999% of all payment request outcomes, and that means that they will bombard you with multiple notifications just to guarantee that.
This is actually good news: no payment gets forgotten. The bad news is that a payment may go through the sync and the async process at the same time.
This at-least-once delivery is prone to race conditions. And race conditions are the worst in payments! When that happens, multiple payments get accounted for when in reality only one has happened.
You would lose money on every transaction! Wasn’t this supposed to earn you money instead?
How do we fix this? One way is to place decision logic right in front of the calling of the blocks. The payment gets transitioned to “process_pay started”, so that if one thread invokes this method while there’s another one already running it, it will get rejected.
This is very elegant, because there are other situations in which a decision logic is very useful, like preventing over refunds, retry payments, etc.
The design can accommodate that, and more.
What’s more, it effectively makes this design horizontally scalable. Come Black Friday or any other peak event, more servers accepting requests concurrently won’t lead to concurrency problems unless the database gets overloaded.
In which case, the work that you will need to do to scale further must happen at the database level (like caching or sharding).
To bring this home one more time.
Flexibility, Python’s key feature, is the foundation of payment applications. It is what keeps developers in sync with the hectic payments industry, while maintaining control and stability.
Composition, and the chain of Blocks, is the manifestation of that flexibility in the code. Instances become the real world implementations of the abstracted Payment Flows.
And finally, a Design based on a Technical Definition of Payments, where we are
making the pay method private to accommodate customer actions,
splitting it into two methods for sync/async flows,
and wrapping them with a decision logic mechanism to prevent race conditions.
If this has resonated with you, you’re in for a treat. I’ve been working on a Django package that ties all of this together. It’s called django-acquiring. If you liked the talk, you should give it a star!
I hope I’ve convinced you that building payment applications not only is easier than you thought, it is also very exciting!
This is how we’ve redesigned our payment applications at Kiwi.com, and I couldn’t be happier about it.
Not because it’s very simple. It’s not. But because, thanks to this, for me, work feels like an adventure.
Hi, what happened to the django-acquiring library? Is it now private or just removed?