Continuous Integration, Delivery, and Deployment: The Modern Assembly Line for FinTech Software
I won’t trouble you by explaining what continuous integration, delivery, and Deployment does best. In our constantly changing FinTech environment, plenty of companies want to release new features as quickly as possible. But practically speaking, the CI/CD implementation process is full of hidden reefs that can quickly derail these efforts.
Quick results are a house of cards
In an interview with Alex Sukhenko, Chief Technology Officer at Salsa Labs, a fundraising platform for nonprofit organizations, he shared an interesting example regarding the migration to continuous delivery. Salsa Labs had been in talks with one company about various routes toward a possible acquisition.
As Salsa Labs looked at the target company’s demo, the company reps said, “This is our existing platform, but by the end of this year, we plan to migrate over to the new one and have our old one shut down.”
“Great!” Sukhenko said. “Show us the new one.”
“Well, we can’t. It’s not ready to demo yet,” they responded.
“Well,” Sukhenko said, “how are you going to succeed eight months from now, when you migrate everybody off the existing platform, if you can’t even demo anything now?”
“So, they’re in this new architecture, going to Google Clouds, [and] they didn’t use it before. […] They’re telling us that they like the Steam code, and their sprints are two-weeks long. […] But it’s not going to work because it’s a house of cards. They can’t migrate everybody. They’ll migrate things, they’ll migrate the first set of customers, and [when they] get a whole series of problems, they’ll throw down.”
Continuous delivery tools are out there, Sukhenko says, and those are nice concepts. Everybody would like to implement them. But if you’ve got heavy new apps and a lot of involvement in your software, it’s very hard to do it all in time. It can be nearly impossible depending on your set of servers and applications. In other words, before following the crowd, a company should ensure that CI/CD is something that their product really needs.
Some companies have no reason to make changes. For example, Ryan Brucker, Chief Technology Officer at Portfolio Pathway, a web-based platform for RIAs, says that their data team is somewhat similar to that of DevOps . They ensure a continuous integration and delivery process. However, Brucker says that they aren’t using continuous integration fully because they do not release every cycle.
When does introducing CI/CD lead to success?
Nevertheless, positive implementation cases do exist. Let’s look how companies reached success and growth thanks to continuous integration, delivery, and deployment.
For example, Roland Collins, Chief Technology Officer at InvestEdge, a comprehensive wealth-management platform, built a process around Build Master from Inedo. Build Master is essentially a framework that handles promotion and delivery in different environments. InvestEdge offers development environments that are updated daily, directly from the development branches, for every development stream. As a developer checks code, it shows up on the server on the same day, generating tight feedback loops that are useful from a testing perspective.
One advantage of a continuous delivery approach is that you can flexibly react to market needs and correct product mistakes very quickly. Perry Moutzouros, Chief Technology Officer at Oranj, another platform for RIAs, told me that their super lean and agile development group releases very small changes every five days.
“I think that’s probably the most important takeaway [is that] to do the best you cannot make assumptions. And if you do make an assumption, you try to make it as small as possible, and you do the best you can to actually ship something that’s feature complete, that can either prove the assumption or allow you to correct [it] in a quick fashion.”
Henry Ford’s assembly-line concept for software
That said, continuous delivery is not just about fast releases. A rare company can make this kind of pipeline, and fortunately, I happened to meet one.
Trizic’s former Chief Technology Officer, Steve Mays, was appointed to his position to drive automation within the enterprise-oriented platform for RIAs. He decided to split every business process into its own set of services, each comprising at least two components — (a) a producer, who gets data from the data store and writes it into a queue, and (b) a consumer, who consumes this data and does something with it. There’s a contract between the two in terms of the message in the queue. Where the producer gets the data is immaterial to the consumer, and what the consumer does with the finished product is immaterial to the producer.
Mays had an “Aha!” moment when he realized that he could leverage Henry Ford’s assembly-line concept for software. By allowing engineers to contractually agree on format and data, his team can divide work into a constellation of producers and consumers (or collectively, “workers”). Under this model, many Trizic engineers can work on the same project without running into merge conflicts.
Mays also adopted “configuration flags ”— a concept wherein one engineer continuously writes little bits of software. They deploy these mini “chunks” every hour, as quickly as possible, without regard to completion order, and in a completely safe manner.
“You basically build up your prod like Legos as opposed to large deploys.”
Mays is not a big fan of short-term sprints, so Trizic adopted Kanban, under which the team builds a corpus of trusted code over time using configuration flags. This allows for substantial flexibility; when necessary, certain client features can be turned on or off.
“I’m still absolutely, 100%, categorically opposed to releases of any sort. You release code as it is ready. In any order. You build up code, Lego by Lego, until it’s finally all tested in pre-prod, and feature flags are enabled in prod.”
Thus, each feature can be released once it’s ready and has undergone all relevant testing. This tricky transition is possible due to Ansible, wherein all source code is covered by unit and integration tests. They also use Selenium Web Driver test for user interface and a steady progression of approvals through various environments. This ensures that it’s proven before being deployed to production.
This kind of arrangement is a great example of classical Kanban with continual delivery and a pure DevOps culture. Additionally, microservices and modular architecture can enable flexibility and fast time to market.
Continuous deployment experiences
Very few companies use the continuous deployment approach because it’s often very dangerous, especially for complex systems and high-responsibility projects like those in the FinTech industry. Sergey Matikaynen, Head of Software Delivery at INSART, a FinTech engineering company, says:
“We use continuous delivery [for] every project. It’s a really nice and beneficial concept. There’s another similar concept known as continuous deployment, which means that one doesn’t just prepare code to deploy, but [that] every change that passes automated tests is deployed to production automatically.”
In addition to these three concepts that complement each other, Matikaynen advises implementing continuous integration and delivery but not continuous deployment.
“The first two enable to keep the project stable, monitor its state, and see if something goes wrong quickly. They help to be always prepared for a release and stay one-click away from deployment to prod. Continuous deployment means clicking at a stroke, which requires compulsory post-production tests to check ‘happy paths’ at least. From our experience, we don’t recommend [making] a step to continuous deployment without thinking twice.”