Database migration tips & tricks
May 12, 2022A few "obvious" database migration tips that I've run into, that I guess may not always be so obvious.
I’m not anything like a database specialist or guru. But I’ve run into a few tips and tricks over the years that make SQL migrations easier to handle. And as obvious as some of them seem to me now, I’m frequently reminded when discussing these tips that they often are anything but obvious to others, and in fact at one point they were not obvious to me, either.
I make no claim that these are all still “best practice” (whatever that even means), or that this is anything like a complete list. But here goes…
- Make your database migrations idempotent. In other words, it should be safe to re-run the same migration multiple times, with no change in outcome.
- If your upgrade isn’t idempotent, wrap it in a transaction. In other words, if you can’t write an upgrade that is safe to re-run, at least put it in a transaction. Then if it fails, your data is not left in an inconsistent state.
- Don’t write downgrade scripts. Most db migration tools allow you to specify both upgrade and downgrade scripts, which are expected to mirror each other. This allows a clean roll-back, in theory. Instead, I prefer to make my upgrade scripts idempotent (see above tips), then downgrading should never be necessary.
- Run large changes in the background. If you’re converting 10 million columns from an
CHAR, for example, don’t do this in an upgrade script. Instead, have your upgrade add the new
CHARcolumn, then have a background task handle the data migration from the old to new column. Then drop the old column some time later (see next tip).
- Isolate destructive changes. The destructive changes (such as
ALTER COLUMN, or
DELETE WHERE...) are actually impossible to downgrade anyway (aside from restoring a backup). But that doesn’t mean we can actually avoid them. Sometimes they’re necessary. So I always keep them separate. That is, I never do add a column and drop one (sometimes known as a rename) in the same upgrade script. Instead, add your new column in one migration, and preserve all destructive changes for a separate upgrade, ideally one that runs days, weeks, or even months later, when you’re highly confident it won’t need to be rolled back.
That’s all for today. What tips do you have to make database migrations easier?
Short-term alternatives to code freezes
Use a code freeze as a reminder that there are still quirks to be worked out, and make an effort to work them out.
Adventures in DevOps 114: Progressions Through Programming Languages
What are the types of program languages required in DevOps? What are the pros and cons of each?
Rose bushes and story points
Gardening with a 1-year old takes a lot longer. How should this affect our estimates?