Inpainting, Deep Learning, and Anonymization

AnonymizationImage recognition is one of the areas that Deep Learning excels at. It is, in fact, so good at it that these days you can pretty easily identify anyone from a photograph (especially if you’ve got a large enough corpus of identity information, like Facebook, Google, or, oh, the Feds).

So, how d’you go about anonymizing pictures? Let’s make this legit — you’re the newspaper (look it up, it’s a thing), and you want to publish a picture of a witness in your ScoopOfTheYear™ along with the story. You’d probably do something like this picture, right? An image with a nicely blurred face, can’t make out who it is, etc. etc.

Blurring: Not the AnswerWell, you guessed wrong! Back in 2016, McPherson et al. (•) showed that you can uniquely identify people from these types of blurred images. (You’ve got to remember, Neural Networks don’t give a s**t about whether a thing is a “face” or not, it just matches away…). What’s even worse is that Shiri et al. (••) figured out a way to actually restore the original image from the blurred one. So now, you can not only figure out who the blurred person is, you can even get back to the original image from this. Ouch!

A Way Out?
When you get down to it, there are two possible ways out — blackboxing, and inpainting

BlackboxingPretty much what it sounds like. Just slap a huge black box over the whole image. Mind you, depending on the subject, you might want to slap a black box over their body too (‘cos you might be able to identify them based on posture, body type, etc.). The problem with these types of solutions (called target-generic) is that while effective, they do tend to remove a lot of context from the image, and they’re also pretty grody to look at. I mean, it’s hard for a target audience to sympathize with a black rectangle, y’know?

InpaintingHere, you swap out the face with a different face, but one that works in context (Technically speaking, inpainting refers to the process of replacing lost/corrupted parts of an original image/video. Done well, this can be awesome. Done poorly, well, you can end up with what happened to Elias Garcia Martinez’s ‘Ecce Homo’.

The key to inpainting for anonymization though, is that you use it to swap in a different face! Mind you, to prevent it from looking goofy, you want to try and match certain things, like face orientation, body pose, activity, lighting, and so forth.
It’s all a lot harder than you’d imagine, and has, hitherto, largely been a manual thing (if done at all).

A new paper by Sun et al. “Natural and Effective Obfuscation by Head Inpainting” manages to do exactly this though! The best part is that they managed to come up with a target-generic solution (remember blackboxing above? Works against all identification mechanisms?), that still seems extremely natural.

Deep Learning FTW! Now, to move on to video, right?

(•) “Defeating Image Obfuscation with Deep Learning” — McPherson et al.

(••) “Identity-preserving Face Recovery from Portraits” — Shiri et al.