5 Traits of Payment Engineering
As with any engineering area, payments engineering has its own character and its own unique aspects. Creating and operating payment systems has long been a peculiar niche different from social networking, medical, or any other type of applied software engineering.
Some engineers come into the payment industry once and never leave, some, on the other hand, retreat after a short exposure. After spending close to 10 years in the payments industry and learning from some of the brightest minds I thought to share some peculiarities of the space.
This article aims to help engineers, technical executives and engineering leaders coming from non-payments background to quickly grasp key differences of payment engineering and be able to make an immediate impact on their teams and their businesses. This article can also be helpful for those who are exploring an option to come and join the payment engineering workforce and not sure if they will enjoy tackling payment engineering unique challenges.
What is so different about payment engineering?
The reason why many businesses exist out there is that they exchange certain services for customer funds (think Spotify) or the other way around, paying out customers for their services (think Amazon Sellers or Uber Drivers). This makes the money processing activities somewhat a universal part for most consumer and service companies out there. All these businesses have their unique payment processing status quo in part dictated by company history and growth. Some companies rely on off-the-shelf solutions like Stripe, Square, or Adyen, while others go deeper into the stack and invest in payment processing and optimization.
The reason payment engineering spanned out to be a distinct engineering space is dictated, on one side, by the fact that payments rely on a set of very unique integration protocols (like ISO8583, ISO20022, Batch Settlement File formats, 3-D Secure, etc.). On the other side, it takes time to understand the end-to-end flows together with all involved parties and their technical and financial relationships (acquires, issuers, payment gateways, schemes, etc.). To make things worse, regulatory requirements and regional mandates set a high bar when it comes to compliance and security of payment systems.
Specific traits of payment engineering vary depending on what type of organization you are in (merchant, gateway, acquirer, issuer, payment method provider, or other), and whether you deal directly with payment methods like bank accounts or credit cards, or surface some alternative payment methods (for ex. PayPal, Klarna, Trustly, etc). Across the board the specifics of payment engineering can be clubbed into the following five traits:
- The value of the payment system correctness
- High security bar and unique compliance requirements
- Cross-functional collaboration within the organization
- Integration and collaboration with other players in the payments space
- Peculiar challenges in building great customer experience
These traits can be represented as a pyramid where correctness is placed at the bottom — symbolizing the biggest level of direct control that the payment engineering team has. And going all the way up to customer end-to-end payment experience where payment engineering has the least amount of control (think about the fragmentation of experience throughout payment journey between an issuer, a payment method, an acquirer, and a merchant).
Trait #1. Payment engineering values correctness over speed
With processing payments data, high importance is placed on getting things right. Whether it is correct handling of decimal points for different currencies, or setting right flags in the payment integrations.
The correctness of a payment system is paramount from the early days of its existence. Correctness comes in different forms and flavors but most often includes correctness of monetary calculations, correctness of payment integrations, and correctness of a payment system state.
Correctness of monetary calculations ensures that payment systems can properly do basic arithmetical operations. While it sounds silly, things might get messy when you need to do tax calculations, multi-payment-method operations, or process cross-border transactions (transactions that involve more than one currency). Getting rounding rules right and consistent with your business practices is easy when you start from the beginning and hard when you attempt to fish out bugs later. So, you might decide to have an extra layer of validation and unit testing in place to avoid silly mistakes falling through the cracks.
Correctness of payment integrations ensures that whatever messages you pass to your processing partners are right and aligned with what your business expects. This aspect of payment system correctness is explained more in Trait #4.
Correctness of a payment system state guarantees that transactions you process are stored, visible, and match with what other players in the payment flow see. As a result, transactional processing becomes a crucial part of the design and operations of payment systems, where transactions move payment system from one consistent state to another (or don’t move at all should the transaction fail).
You might be asking: “Shouldn’t all computer systems value correctness?”. The answer is yes, but only to a certain extent. For example, if you look at the social networking applications, where a mismatch in a count of likes under the post is not as impactful as under-reported taxes for you and your clients. Additionally, it is hard to revert an impact of production mistakes when you post invalid data to your downstream dependencies losing the ability to change it.
To tackle the challenges of correctness in payment systems, engineers and engineering leaders need to possess meticulous attention to details and high-quality standards. It is not unusual to see payment teams strive to abnormally high test coverage in payment systems specifically for the reason that a benefit of preventing mistakes from going into production outweighs potential delays caused by adding an extra layer of unit and integration tests.
Regardless of where you are in the payment flow, building a payment processing system that is correct, reliable, and stable is a key to successful payments business. It is often better to slow down and ensure an extra set of checks and balances is in place before getting the payment system out to your customers.
Trait #2. Payments have high security and compliance bar
Payment processing requires stricter than usual security controls and impose an additional layer of business and security compliance that impacts engineering processes.
Information security and compliance takes a special place in the hearts of payment engineers and broader payment teams. A particular set of compliance requirements differ depending on the type of payment information you collect and process. It can materialize in either PCI DSS compliance if you happen to handle credit card data, or NACHA security requirements for ACH transfers, or many other regional and payment method specific mandates. And if only information security compliance is not enough, local governing bodies can enforce other business compliance requirements, for example, requirements to collect specific data from the entities you plan to send money to (KYB/KYC), or perform transaction monitoring and reporting for anti-money laundering, or special requirements connected with the type of financial license your organization holds.
A broad set of regulatory requirements and processing mandates becomes an important input into payment engineering activities. These inputs guide decisions on how to store information, how to process customer data, and what technical providers to use and what technology providers not to use (for ex. cloud services, open-source libraries, etc.). Engineering decisions can vary from small like logging reduction (to hide customer and payment method data from logs) and ending with software development lifecycle and operational procedures (payment method data access, a four-eye principle for infrastructure and access control changes, etc.).
Successful payment engineering teams embrace the best industry practices and compliance requirements from day one and make them business-as-usual. Setting high security bar and ensuring adherence to compliance requirements is the job of each person on the team. Continuous curiosity and willingness to look beyond your scope will power up your ability to deliver secure and compliant payment systems.
As a general rule of thumb, don’t do compliance for the sake of being compliant. Compliance guidelines have been built with the goal of building trust between payment industry players and building reliable payment systems. Most of the guidelines are based on reflections from past incidents and security breaches. Embedding compliance requirements uniformly into your organization will enable you to swiftly pass regular assessments and at the same time will make you a stronger organization.
Trait #3. Payment engineering requires cross-functional collaboration
Payments are interlaced with internal business processes. A complex net of dependencies multiplied by a variety of payment operations (a charge, refund, chargeback, adjustment, payouts, etc.) creates an almost infinite number of possible use cases that payment engineering teams need to wrap their head around.
Payment workflows can be tied to a diverse set of internal processes and events. A specific set of dependencies varies based on your location in the payment flow — whether you are on the merchant, acquiring or issuing side (or somewhere in the middle). For example, if you look at the dependencies within the merchant, payment operations are triggered as a result of internal physical (for ex. shipping, delivery) or virtual (for ex. recurring billing) events. While for an issuer, payment processing will be tightly integrated with internal fraud systems, customer support, and account management services.
The more complicated internal business processes are, the harder it is for the payment engineering team to navigate and manage payment system complexity.
A great variety of business processes and structures leads to very unique payment system designs that are guided by the company’s history and future vision.This leads to an expectation for payment engineering teams to know and navigate the internals of the company and its business.
Engineers and leaders who have an ability to navigate a cross-functional environment, understand constraints, priorities, and processes of other teams within the organization will get an edge in building payment systems that nicely fit in the overall organizational architecture. If you find yourself in the payment engineering department that has a reputation of a bottleneck, your skill and willingness to build relationships across the business units will dramatically improve your chances of building a payment system your internal customers love and pivoting to a reputation of an enabler and a reliable partner.
Getting a payment system to be well integrated into the rest of the business workflows is a challenging and often long process. An in-depth understanding of the internal corporate structure and business processes along with setting up well-defined boundaries of a payment system will supercharge an ability of a payment engineering team to build payment systems that are reliable and scalable.
Trait #4. Payment engineering involves significant integration efforts
Multitude of payment processing flows, payment service providers, acquirers, and issuers makes integration efforts to be a significant part of payment engineering job.
Payments don’t happen in the vacuum. Regardless of where you are in the payment processing chain you always need to integrate with somebody. Merchants integrate with gateways, acquirers and payment method providers, gateways integrate with acquirers, acquirers integrate with schemes and so on, and so forth. On top of payment flow integration, there is plenty of auxiliary services that a company might decide to integrate with. For example, an integration with transaction risk analysis services (like Ravelin, Ekata, Riskified, Kount, etc.), payment monitoring and optimization solutions (like ProcessOut or Pazien), compliance solution providers (like VGS), or many-many others.
Looking at the variety of payment players, it seems like every payment-related company uses its own lingo and a set of terms. Some companies call authentication a pre-auth, some call charge a debit, some have disbursements, some have payouts. Different languages are in part caused by specifics of particular industry the company is in, sometimes it is caused by legacy, and sometimes it has been heavily influenced by the first payment related product or system the company used. Matching an external set of concepts to your internal terminology is a challenge to start. On top of that, the variety of unique messaging, flows, and data formats creates a struggle of mapping this variety to your internal data models and architecture. As a result, it is often seen that external dependencies influence how payment engineering teams think and design their payment systems (adding asynchronous flows, adding keep-alive proxies, etc.).
Decrypting integration specifications and translating them into the code is an inevitable part of the payment engineering job. Rarely though the teams can figure integration by themselves. This forces payment engineering teams to master communication and interrogation skills to uncover valuable integration information from processing partners. When things go south and square peg doesn’t fit in the round hole, integration work needs an open mind to be able to rethink your payment system design and / or integration approach.
In the ever-growing payment industry with a new payment fintechs popping up almost every week, the integration work becomes to some payment engineering teams their bread and butter. The never-ceasing integration effort of payment teams fuels a gigantic network of global payments that enables frictionless and convenient ways for customers to shop globally.
Trait #5. Payment engineering is about end-customer experience
While often neglected, focus on customer experience throughout payment journeys (whether it is a charge, refund, or payment related customer support inquiries) builds long-lasting trust and partnership with your customers.
Looking at the dashboards of payment gateways with thousands of transactions processed per second, it is hard to grasp that behind every transaction there is a real person. These transactions often signify a culmination of some kind of journey — quick like a transit pass purchase, or multi-day effort like picking up a new fridge or a vacation package. Regardless of the case, it places a great responsibility on everybody involved in the payment processing.
The correctness and stability of payment systems for a long time have been valued higher than their visual appeal or convenience. Even today across the internet and brick-and-mortar retails you can see examples of terrible customer experience, whether it comes in a form of an excruciatingly long delays or credit card forms that seems to have been created in the 90s.
For any kind of merchant, payment processing plays an important role in finalizing the customer journey. A final click on a “Pay Now” button shows the success of many teams involved — marketing, logistics, retail frontend, etc. At this moment you are one step away from converting a prospective customer into a satisfied customer (or dissatisfied customer should the transaction fail).
Modern days payment engineering teams are tasked not only with meeting the requirements of implementation specs but also actively advocating on behalf of the customer. It is a pleasure to see companies like Stripe, Square and others pushing the envelope and setting the bar high in user experience, while simultaneously ensuring their systems are reliable, fast, and developer-friendly.
Successful payment engineering teams aggregate and process an enormous amount of fragmented and heterogeneous inputs ranging from business requirements and internal organizational specifics and ending with payment protocols and security and compliance requirements. Ability to hold, structure, share, and use knowledge from all these inputs and apply it in building payment system differentiates mature payment industry experts from inexperienced payment engineering teams.
When it comes to payment engineering leadership, those leaders that feel comfortable navigating the complexity of the organizational business processes and broader payment flows beyond one’s business will help their teams to be successful and move fast to deliver best of class payment experience to their customers.
Payment engineering is a fascinating and exciting field that offers unique challenges for software developers and engineering leadership. Whether you are thinking about switching to payment engineering or want to better understand what a payment engineering team does — I hope the shortlist of traits above shed some light on the technical and process constraints that surround payment engineering.
I would love to hear from you. Tell me what you see as unique characteristics of payment engineering, and if you are a payment engineer or an engineering manager — share your view on what it means to be one.
