Why continuous deployment is better for emergencies
Multi-stage software delivery means no hotfixes, or lots of extra work.I was recently talking to a technical founder, who was concerned about some of the implications of continuous deployment. He acknowledged the dangers of GitFlow, so suggested rather than promoting feature branches from a devel to stage to master branch, why not just promote an artifact (such as a Docker image)?
Sure, build the Docker image immediately after merging into master, but don’t deploy to production yet. Instead, let that Docker image go through a QA process, for example, before promoting it to production.
There are times when this may make sense. But it doesn’t come without drawbacks. Specifically, (and very relevant to the situation of this founder), consider what happens when you need to make an emergency fix for a production issue.
If Bob has just merged his amazing new feature, and now Alice comes along with an emergency hotfix, we have three possible options:
- Use a separate “hotfix process”: Apply Alice’s hotfix to whatever version of the code is in production, build that artifact, then deploy it. Then also apply Alice’s fix against
master. - Revert Bob’s feature, and apply Alice’s hotfix, then fast-track the QA process to get Alice’s fix into production, then re-apply Bob’s feature.
- Wait for Bob’s feature to pass QA before Alice’s hotfix gets into production.
None of these are ideal. The first one requires maintaining and testing two separate relese processe. Additionally, both the first and second depend on multiple merges, which significantly increases the opportunity for mistakes to occur. And the last one is non-ideal for the obvious reason that it means that there’s effectively no way to do an emergency fix.
The solution I always advocate is to move all your checks, including any manual QA, to before your developer presses that merge button. In short: As soon as that merge button is pressed, everything should be automated until that aftifact is deployed and running in front of customers (or in some cases, such as a mobile app, ready to be uploaded to the app store). If you’re intrigued, you can learn more about this in my free e-mail course, The Lean CD Bootcamp.
