Validate Your Migrations!

Migrating data is never fun. It is, at best, a PITA, and at worst, a reason to go find a different job. But hey, sometimes you have no choice, and you just have to Go There, and that’s where Scientist comes in.

Scientist?
Yeah, Scientist. But do wait, we’ll get to it in just a moment.
First, let’s just look at what you’re probably going to be doing to migrate your data between, oh, whatever it is — databases, DB versions, applications, whatever.

Basically, you’ll be doing some variation fo the Dual Write approach (•), which, roughly, translates to something like

1. 1. Do all your Writes (Actually, Create / Update / Delete) to both the Old, and the New targets, while serving Reads only from the Old target.
2. 2. Start serving some of the Reads from the New target.
3. 3. Eventually serve all the Reads from the New target.
4. 4. Turn off Writes from the Old target
5. 5. Go get some delectable mochi ice-cream to enjoy in peace.

The fun, of course, is in the details. I mean, did you refactor all of your reads and writes? And how do you know that you actually did it correctly, and something didn’t drop through the cracks? Because, remember, this is software, and if there is any possible way in which something can get screwed up, as sure as little apples, it will be exposed at the single most inconvenient time!

And that, my friends, is where Scientist comes in. It’s a remarkably nifty toolkit from GitHub that you can use to validate your code’s behavior in both the Old and the New environments. It was originally Ruby, but is also available in a host of other languages (Clojure, Go, Elixir, Python, …) (••).

Basically, (in Ruby) you wrap a use block around the Old behavior, and wrap atry around the New behavior. Then, experiment.run will always do the stuff in use (the control), but also do a bunch of stuff with try ( the candidate).

• 1. It’ll randomly call use or try
• 2. Measures the durations of each (‘cos timing related issues are a thing, y’know?)
• 3. Compare the result of try to the result of use,
• 4. Record any exceptions raised in try

And yeah, all this information is published, you get charts and whatnot.

It’s really good stuff, and if you’re doing any kind of migration, you better be using this, or doing something pretty much exactly like this!

(Incidentally, when migrating don’t forget about backups and archives. You’ll need some way to recover them, y’know?)

(•) For more on Dual Write, check out Migrating to a New Database by, well, me 
(••) For more on Scientist, check out “Scientist: Measure Twice, Cut Once” by GitHub Engineering.