Online marketplaces are complex systems. That’s true all the more when you decide to equip them with a custom payment gateway you developed (and there are many good reasons why you should do so).
As a development company, offering CoreWallet, a software foundation for payment and ewallet applications, we are familiar with the complexity of creating payment gateways. It’s important to approach the development process with a clear plan. To help you master the technical challenges, we have compiled the common stages of such a payment gateway project for you. And we’ve also collected a few best practices.
So, let’s walk the walk. Let’s build a payment gateway.
Stage 0: The Demand (Or: Do You Even Need One?)
But, wait a minute!
Before you write a single line of code or think about your payment application’s software architecture, you have to decide on a direction. You will have to make a number of decisions about your Payment-Gateway-to-be.
And that means you have to know where your starting point is.
Prepare for a phase of assessing the need for a custom-built Payment Gateway. When your company makes this assessment, you have to take the whole business model and strategy as well as your respective market into consideration. (Disclaimer: By “company” we do not necessarily mean the CEO, but also the decision-makers in the business and technical departments).
We can only hint at some general tendencies here. Detailing the ins and outs of payment systems in the modern platform economy would take a full-fledged article. (Good thing we already have up our sleeve, as Part 2 of our Payment Gateway series).
So we only will say so much here, regarding the business side: It is perfectly possible that a third-party solution suffices for your business model.
However, if you run an online marketplace, especially one for a specialized niche of the market, building your own payment gateway may be an attractive option. A custom-created solution can be adapted to your business’ specific needs and will save you frustrated customers aborting payment processes. In addition, self-built Payment Gateways are yours to expand. You own it, so you set the priorities for the feature roadmap and after-launch maintenance.
Stage 1: The Basic Questions
(Or: How Will the Payment Gateway Function)
Once you decide to develop a custom payment gateway, you will find that many possible roads to this goal open up. You are building a solution from scratch. That means you have to consider a high number of technical factors. And you will make many decisions regarding layout, features and architecture for your solution.
When drafting out the road-map and goal-set for your solution, these are the areas you should give some thought to:
1. Interaction (Buyers, Sellers, Marketplace Operators)
Formulating your business case, you will have outlined the basic interactions between the buyers, the sellers and the platform itself. Put these at the forefront, when you start designing your solution. The interaction flows have to be properly set up before the programming starts.
For starters, it’s important that you determine who sells what to whom. Who can take the role of the seller and who can take the role of the buyer? In any case, you have a merchant of record (MoR), who is authorized to call in and trigger a customer’s payment and who will be held responsible by financial authorities. In online marketplaces, this role typically goes to the company owning and running it. The selling party then acts as the sub-merchant, transferring parts of their revenue to the marketplace as a fee for the service. Alternatively, the marketplace, acting as the MoR, can collect all the money from purchases and distribute sellers their respective share of it.
The customer on your platform is the third party to consider when designing your transaction flow. Just take data protection regulations for example: How much data will your platform collect from your buyers? And will you have private customers, corporate customers or both on your platform? You might as well allow users to sell things to each other, eBay style, in which case customers can become merchants on their own. No matter what you settle for, it will impact your architecture and workflows on a fundamental level.
For instance, depending on whether you have a C2C or B2C or B2B marketplace, whether you are onboarding private persons or companies, there are many other aspects to consider as well: Onboarding processes, KYC processes, legal requirements for AML and so on.
Furthermore, depending on what user data is collected and stored in your systems your data protection policies will turn out differently. Your system must be able to handle customer and financial transaction data securely. On the part of your developers, that means using secure coding procedures during development in the first place. Compliance with financial regulations (like rules for preserving the financial data for a certain period of time) and data protection rulesets are big topics. GDPR adds some technical requirements in regards to data policies, like…
- Data retention periods
- Data anonymization
- Restricted access to the data for your employees and third-party providers
- Customers’ rights to know what personal data is stored for what purpose
- And finally: “the right to be forgotten”.
When you offer online payment on your marketplace, you have to decide how you want to integrate payment service providers into your payment gateway during development. This integration can take two forms:
- PSP integration via API, with an internal checkout page being part of your platform’s frontend.
- Redirection, leading the user to the Payment Provider’s designated, external checkout page.
Beyond that basic distinction, there is a lot of fine-tuning you can do here. It reaches into the fields of useability and technical practicability. Specific platforms or devices might dictate one approach and be incompatible with another. A mobile payment flow may differ from one accessed via desktop, for example, not to mention terminal devices like smart car computers. No user wants to busy themselves with credit card credentials while sitting in their car. Also, mind that some PSPs only support one of these two integration forms. This determines which PSP you will be able to use with your Payment Gateway – and which checkout processes you can implement.
With the PSPs integrated, you have completed the bulk of the work regarding Payment Gateway development. But side topics and add-ons demand your attention as well.
For example, there are other procedures you have to include into your registration, checkout and payment validation flows. Take KYC and KYB verification providers, anti-fraud and risk management solutions, for example. They put the same choice as above before you: Integrate a third-party provider or even roll with the fraud and risk checks of the PSPs. Or build your own solution, as the second option.
Not to speak of another important aspect of payment transaction management: Communication. You must make sure to integrate message providers, like email or SMS services to convey information to your customers and merchants.
All of the aforementioned services are just the bare minimum of procedures to incorporate. We’re sure you will think of other services your marketplace can provide, which require additional service providers.
It’s a common saying that as a founder you should “start small and think big”. What this clause leaves open, however, is what we define as “small” and “big” respectively. If you want to develop a payment gateway from scratch you should have a good idea about the respective numbers though. And about how fast you want your system to scale.
That means you have to get a grip on the projected workload of your gateway. That is, the number of transactions it will have to process in a given period of time. And that’s just one factor you have to account for. Others will be:
- The anticipated number of transactions in one year, in a few years etc.
- Expected maximum peak load in a day, an hour and a minute or at certain times. For example, let’s say you run a marketplace for fireworks and light show equipment. Be certain to see peaks in the weeks before New Year’s Eve and on Black Friday.
- The time schedule for when you want your system to handle a specific transaction load (should it increase its capacities gradually, like when you want to limit the number of initial users? Or do you want it to be all set from day one?)
Your estimations here will have an impact on the architecture and development process. Payment gateways dealing mainly with few high-value transactions do not need the same scalability as high-traffic systems with many small transactions. 100 000 transactions per day are one story, the same amount every 5 minutes is something completely different.
The devil is in the details here, in the individual processes within the system.
Let’s take an example: Say you want to build up a network of instances, which can be switched on and off to help with a higher transaction load during peak times. This would allow you to scale up temporarily should the need arise. Development of features, in this case, is much more complex of course as the transaction data would be distributed all over the system, but you still have to ensure consistency. All transactions must be matched exactly and flawlessly: From customer to instance, from instance to customer. You even have to route payment updates from the PSP to the appropriate instance – and that can get tricky when an instance is shut down due to low workload.
And that’s only hinting at the complexity and the minute decisions you must take when planning the scale of your marketplace’s payment operations.
4. Time to Market
We already hinted at this, but the factor time is vastly important. Depending on the requested functionalities, flexibility and scalability of the system, the implementation takes more or less time. The rule of thumb: The more complex the system and the more smoothly it should scale, the more time it will take to create it. And that means it will probably launch later.
In the end, product owners might go for one of two scenarios:
- They launch the product as fast as they can. Having a well-staffed team of software engineers at hand is a basic prerequisite to achieve this. And they will take a risk here because the product will perhaps not provide the full package of scalability options. You might even give access only to a reduced set of primary customers, starting with a deliberately small business case.
- They have the product finished. It’s then designed to be as feature-complete as it can possibly be, with all scalability options. If customers rush to your platform, flooding it with account registrations and transactions, that’s the way to go. But there is a chance that you will have spent resources on infrastructure that idles until your marketplace gains traction.
From our experience, the first option is suitable for most businesses. It adheres to the principles behind Agile Development, which we base our software development processes on. You just have to be able to make an educated guess about the expected amount of users and transactions. You can then build a Minimum Viable Product that can handle this amount. If it hits the market with full tilt and surpasses your expectations, you intervene as necessary, adding features for more scalability, like the aforementioned distributed system.
5. System Architecture
Once you have gained a good grip on your vision and the questions above, you can outline your payment gateway in detail. On the technical level, that means laying down a proper system architecture. It contains several crucial aspects:
If you have a system comprising distributed instances, deployment differs vastly from when you launch a monolithic, single-instance product.
That’s even more true when you want to deploy your platform and payment gateway in more than one country. As you do so regulatory variables are added to the equation. Some nations require you by law to deploy in data centres situated within their borders – Turkey, Russia and China for instance.
When you deploy in multiple data centres, you must decide: Do you want national instances to be connected to instances from other nations under the roof of one comprehensive system? Or do you want them to confine themselves to a strictly regional model?
In the case of eCommerce marketplaces, it’s basically the decision between allowing merchants to have a global account or if you require them to register one separate account for each nation they are active in. Have the product designers and legal consultants wrap their heads around this question, before giving software developers the direction.
Another important workflow to outline will be monitoring.
On the one hand, you will have pure technical monitoring. That means keeping track of the health of the infrastructure, checking that all components of your system are up and running at all times.
Secondly, there is business monitoring: watching the number of transactions, new registrations and other flows in the system. Typically you set a baseline number of expected actions and transactions. Once the current numbers differ from this baseline, you probably need to investigate why they dropped and enact countermeasures. Also, ensure that the system can handle the load when the number is higher.
In payment, security is the alpha and omega of a good product and you should strive for excellence here. Not only should you know the specific KYC and AML requirements and regulations for the regions your payment gateway will be active in. You also have to adhere to international rulebooks as well, such as the credit card processing standard PCI DSS.
What’s more, your development team is required to follow secure coding procedures during payment gateway development (if your developers need an introduction to those, we can help.)
Finally, don’t forget to devise a strategy on how to ensure private and financial data protection mentioned above and how to deal with security breaches. 2-factor authentication is just the tip of the iceberg here. (If you want to
6. Branching Out: API Design
Designing functional APIs for internal use is one of the most important aspects of Payment Gateway building. And when you apply some additional restrictions, they open up an additional source of revenue for your company: Giving other businesses access to your solution for their respective platforms. Of course, building an API to that end creates external dependencies and requires clear versioning and upgrading strategies. Having others using your API, you would no longer be completely free to choose the point when to change the APIs.
When designing your APIs, design them with maximum stability in mind. It’s preferable that one API can process as many different payment methods and data as possible. The alternative would be to offer one specific API for every payment method. But the parties wanting to integrate your payment gateway will likely not thank you for it.
Also, make it flexible enough to easily bring in new payment methods – you never know what outlandish way of payment is about to hit the financial system next. The same goes for onboarding of merchants and customers (don’t forget your 2-factor authentication functionalities). You don’t want to adjust the integration every time you want to change something.
Announcing API Changes
On the other hand, be certain how you want to implement changes, should they be necessary.
Once an API is released it’s relatively fixed. Avoid breaking changes and if you can’t, have profound API versioning and change management functions in place. If you decide to decommission your API at some point, be open to the parties using it and give them time to prepare. Don’t shut it down overnight, but set a deadline (“This API will be deprecated after X years”). Indeed, the change of a running API can be for the better, but you must draw in your clients with new features – if there is no benefit, they likely won’t switch.
Stage 3: Development
Having gone through the processes above, you have laid out your payment gateway before you. Not in all detail perhaps: During the development process you will find that a lot more design decisions will demand your attention. But for now, you can gather your software engineers and commence the coding process. Structured and with clear goals of course.
1. Assembling the Team
And that means: With clear priorities. As the product owner, having many features and functionalities on your wish list now, you must assign them priorities. This also includes specifying when your product will have reached the status of a Minimum Viable Product for launch and what you will consider as feature-complete. If you are working with the SCRUM method, you define epics (complete feature ranges) and stories (packages of tasks).
Due to our long-term experience as a fintech and payment software development company, we know that there are a few best practices for organizing your software development process.
One example would be: automate tests and provide a sample integration of your payment gateway and associated payment flows. From our own perspective we can say: If we hadn’t built a demo environment for our CoreWallet software foundation, we wouldn’t have gotten the full picture of its functionality.
In the same regard, make sure your team can identify failures in the product easily: Error messages should be clear and precise – not only for your own team but also for external clients you want to work with in the future. The same goes for your documentation. Have it updated regularly, both for internal and for external use.
As final advice, make sure to have code reviews on all changes as a quality assurance measure so that it becomes less likely for changes to wreck your gateway.
2. Coding the Solution
In the next step, the team and project owner consult, discussing the roadmap and splitting the stories into single tasks. Finally, the team gets to work, deciding which high-prio stories to include in the current sprint. Your company will cycle through this process several times until project launch and at any point, you might want to take in external teams and technical consultants to help you plan and implement everything.
We won’t go into more detail here, as development is the basic bread and butter of your software team. But one bullet point we want to add nonetheless, as we feel it’s too often overlooked.
- Automation: It’s important and it improves the productivity of your software team. During the programming process, even your top coders will produce bugs.
Automating parts of the development process, like build automation using Maven and Jenkins and automated unit, integration, end-to-end (2E) and security testing can help here. Automation procedures like these allow you to maintain consistency within your software. They make it easier to ensure the quality of your code, to discover bugs, should something go wrong or extend your code if you want to introduce a new feature avoiding possible side-effects.
3. Product Launch
Eventually, live deployment will happen. The moment of truth. The point in time when you have a feature-complete MVP ready. What “feature-complete” means exactly can be subject to debate. As a general guideline: The bigger your organization the more time it will take to find a compromise here.
And sometimes, features you badly wanted your payment gateway will not work as smoothly as you anticipated. Automated E2E, security, load and penetration tests are important to help prevent bad surprises after going live. Before you invite customers in, you must make sure your infrastructure will work as it should.
When your testing reports bugs and errors, make a risk-based estimation of how severe the problem is. Some may be fixed easily, others are even small enough that you can go live with them in the code. And then there are critical findings, which will have you postpone your launch. If the issue would cause serious financial consequences or damage the image of your company, you wouldn’t want to go live.
MVPs: Viable Enough
Aside from that, it’s good to realize one thing: Your MVP will contain flaws and bugs when you launch it. It’s practically impossible to code the perfect product, headfirst. There are many reasons why things can go wrong after launch or during development, be it workflow issues, missing data in the reporting or sub-par scalability. In many cases, we found that theory does not always equal practice when it comes to payment and marketplace operations. The given conditions and user behaviour “out in the field” might differ from those anticipated in boardroom meetings. And in such situations, you must be able to react quickly and flexibly.
So, the challenge for your developers in the after-launch operation phase is to make the best of the situation once the MVP has been released and you start the improvement cycle. To enable companies to do that most effectively, we put great emphasis on our delivery processes to our customers. Our Solution Architects share insights into every aspect of the software products we deliver with our client’s software teams, so they are completely set up to maintain and expand the product as required.
Stage 4: Operations and Maintenance
So, now your payment gateway is live, it’s time to lean back… adjust your office chair and get back to work. As your business is running, development goes on. It boils down to two main assignments:
1. Ongoing Development
Chances are that you did not implement all planned features into your MVP. And even if you did, there is one truth about software development we withheld when talking about feature completeness: In software, there is no such thing as feature complete. Even your payment gateway will never be finished in the narrower sense. The payment landscape is so versatile, there will always be new features and payment methods for your coders to integrate.
2. Operations and Maintenance
Aside from creating what’s planned, it’s even more important to keep what’s there: Operations. Bugs will occur, requiring your team to classify and prioritize them, and then resolve them.
In this stage, you should also ask yourself whether you want your own team to operate your software or assign this task to external teams. Perhaps you had your payment gateway developed by external software engineers in the first place, so continued operations as part of day-to-day businesses might be part of the contract (at trimplement, we offer such services).
Looking back on all the requirements for payment gateways, especially those for online marketplaces, it’s fair to say that developing one is not trivial. In order to launch a successful, well-performing payment gateway, many departments, from business to development, must interconnect and work together.
Fortunately, it’s not like a puzzle board, with each piece having to fall in place exactly. In software development, you always have the flexibility to try things out, or go back to the code and change, what doesn’t work – or improve on the things that do. In the end, your payment gateway will be judged by merit of its security, functionality and useability. This will reflect on your checkout process as a whole and in turn on the user experience of your entire marketplace.
So, while sticking to the requirements in data protection, KYC, PCI DSS etc. which are a necessity, don’t be afraid to think big and be creative when bringing your payment gateway in line with your business ideas. A skilled software team (ideally with good domain knowledge and financial software experience) is really all you need to bake your vision of smooth payment transactions into code. With the proper enablers, everything is possible.