Some Useful Tips for Decoupling Releases and Deployments

Pierre Bergamin, VP of Engineering at Assignar, outlines some useful tips for decoupling releases from deployment and increasing deployments by a huge factor, speeding up reverts and planning releases in a better way.

Problem#

Each deployment to production, with customer-facing changes, was very time-consuming and coordination between engineering, product management, marketing, and customer success was becoming exceedingly complex. As an engineering team, we set a goal to get closer to continuous deployment but have realized that the tooling and processes in place were a blocker.

Actions taken#

I broke down and mapped all of our current processes and tools, including Continuous Integration/Continuous Delivery (CI/CD), Jira workflow, existing in-house feature toggling, release notes, communication between different functions in the company, etc. I made sure that we’re all aligned on what problem we were trying to solve there: “Deploy when you want and release when you can”.

Our goal was to be able to deploy code to production several times a day without having to manage change for our customers. Product, Marketing and Customer Success should decide when we should “release” features. Another goal of going towards CD was to be able to revert a change as fast as possible.

It became quickly evident that our biggest blocker here was the lack of a proper feature toggling system. I did research on different tools, ran trials, evaluated them, and picked the most suited tool for us. It includes a Software Development Kit (SDK) for our core languages, a configuration as code platform, which allowed our developers to manage it through GitHub, and a kill switch feature that was critical for us in order to revert changes fast.

The next challenge was to get it adopted by our engineering team. Using it systematically for new feature development was quite a big change for our team. We also had to get product management driving the releases, which went really well.

Lessons learned#

• It allowed us to increase our deployments by a huge factor, speed up reverts and plan our releases in a better way without blocking anyone in the organization.
• Change management is hard for customers but it is also hard for the organization internally. Make sure you communicate why you’re doing this clearly, what you’re doing and how you’re going to do it well.
• The main concern I had with this new system was cleaning up old flags as it introduces a “flag” debt, but this could be viewed as an inevitable trade-off. We also had to make sure to set a style guide on how to name and structure our flags as we had early signs of inconsistency.