If you’re the CTO or lead developer at a SaaS company, there’s a good chance a visionary founder or excited marketing director already refers to your technology as a “platform.” Now it’s up to you to make that promise a reality.
It’s not as daunting as it sounds, because platforms are nothing more than a collection of interconnected yet independent subsystems. Each subsystem is responsible for some discrete unit of work that takes a set of known inputs and produces a set of known outputs while adhering to a contract.
By building discrete subsystems, we can quickly tweak a subsystem, push the change into production, and even run A-B or canary testing on the new subsystem — all with no downtime. As long as we adhere to the contract.
Winner, winner, chicken dinner!
Bonus: You can build new features into a subsystem before developing the UI/UX, allowing separate engineering teams to work more independently.
At Leasecake, we recently started building out an integration subsystem. We already had integration points in our software, but they were largely soft, point-to-point integrations. That’s a great start, especially when you are moving fast and adapting to an ever-shifting market. But for our long-term health, it was time to build an integration subsystem.
Here’s the 30,000-foot-view of what we need to accomplish:
Abstraction of data is actually the simple ingredient in our solution. I learned a lot about data abstraction at my first startup, where we processed a lot of abstract product data.
So at Leasecake, we started by asking questions: What are some characteristics all our potential integrations will share? What does a calendar or email integration share with an accounting integration?
The answer is simpler than you may think. Let’s look at an abstract representation of an integration and break it down.
{
“requestGUID”: “”,
“topic”: “”,
“customerId”:””,
“document”: {
}
}
That’s basically it.
Requesting an integration is a simple JSON object passed in through a RESTful API.
Let’s start by agreeing on what I mean by “guaranteed.”
With a best-of-breed cloud implementation, we guarantee an acknowledged request will process as long as an availability zone (AZ) doesn’t get wiped out. Furthermore, we can scale this horizontally across AZs and effectively have a nearly 100 percent guarantee of processing.
On the front end, we have REST (although other services will likely lean towards GraphQL). Once we get the request, we quickly process it by handing it off to a message broker.
Modern message brokers are fast, scalable, and durable. Everything we need to guarantee processing once it’s been received. The message broker provides a receipt, we pass that back to the caller, and a state engine pulls the request off the queue and starts the workflow.
In the event of a catastrophic failure, we recover the message broker queue, recover the state engine, check for the last point of agreement, and then restart the workflow. Other pieces of the workflow do their part and inform the state engine along the way.
Piece of cake.
More abstraction, please.
What are the abstracted steps we need to perform? Again pretty simple (the devil is always in the details, though):
Of course, the magic happens in the steps. But there’s a lot of power in abstracting every workflow down to these steps. Configuring the topic is where all of the heavy lifting will happen.
Have a journal entry to pass on to Quickbooks? Write a configurator for it. What about a larger ERP like Oracle? Write a configurator. What about calendar entries in G Suite? Configurator. All the other steps are the same regardless of what you want to integrate with.
Sweet!
Arguably, the hardest piece is the actual call into a third party. Yes, RESTful calls are fairly easy, but authenticating on behalf of a client adds complexity. Additionally, what if we want to integrate with an on-premise solution?
We took a typical agile approach to this at Leasecake, crawl before you walk and walk before you run. Our “jacks or better to open” bet was to go with the most-requested integration. Our starting point was a fair amount of hard coding for the request. We learn from that, do another, learn some more — it’s your basic, “lather, rinse, repeat” approach.
This is a work in progress for us, so look for a future blog post on integration as we learn and adapt.
"*" indicates required fields
