Posts

Showing posts with the label AI

Raise the MiniBatch Size!

Image
/via https://xkcd-excuse.com/ “ Training takes time ” ← That’s just the way it is, and the way it’s always been. The thing is, it’s not just the time you spend waiting for that bloody model to train, it’s the knock-on effects thereof. It is exceedingly common to spin off multiple workflows at a time, each with it’s own tweak, and that leads to any amount of chaos when it comes to keeping track of what exactly changed.  Reproducibility is, well, a bit of a PITA in this field   . In fact,  you can even get tooling just to work around the chaos of tracking all these pipelined workflows . Ouch! The underlying issue here is that typically training is done in one of two ways 1)  Throw everything at a single server/GPU (what you do if that’s what you’ve got) 2)  Throw it at a server farm (AWS, GCE, whatever) with their concomitant GPUs/TPUs/whatever Take  ResNet-50 . You’d typically break the images up into “minibatches” of 256 images each, and then have...

Breaking The Rules — Self-Driving Cars Edition

Image
So, this from  a recent article on self-driving cars . One woman said that she almost hit one of the company’s minivans because it suddenly stopped while trying to make a right turn, while another man said that he gets so frustrated waiting for the cars to cross the intersection that he has illegally driven around them. The anecdotes highlight how challenging it can be for self-driving cars, which are programmed to drive conservatively, to master situations that human drivers can handle with relative ease — like merging or finding a gap in traffic to make a turn. Let’s think about that for a moment. The car was — almost certainly — Doing The Right Thing. 1.  The woman who almost hit the minivan because it suddenly stopped? Well, if she was following the  two-second rule , and paying close attention, or was  even more conservative , she would not have been surprised at the sudden stop. 2.  And the dude who illegally drove around the self-driving car?...

Calibration, Trust, and Deep Learning

Image
You know how you really need to calibrate your drone before you start flying it? And if you don’t do that whole “ spinny ” thing with it, it really acts goofy? Or, sometimes Google Maps gets horked, and the blue dot shows you in the wrong place? And to fix it, you need to do that   “figure 8”   thing? These are examples of   calibration  — where the output of an instrument is compared against a reference standard, to make sure that it is correct (and/or set it appropriately). We rely on calibration all the time , heck, modern life would basically be epically disastrous if it weren’t happening behind the scenes. Everything from the standardization of tape measures (think of how that results in the building you are in not collapsing), to voltage regulation (your iPhone is still working, right?), to, well, everything, assumes that instruments are calibrated! Great, but what, you might ask, does this have to do with Deep Learning? The answer is in the form of a questi...

Precision, Recall, and Relevance

Image
“ How relevant is your response? ” A simple question to ask, but figuring out the answer can easily lead you down a rabbit-hole. Before we go down that though, let’s first take a look at what “ Relevant ” actually means. “ Relevance   is the concept of one topic being connected to another topic in a way that makes it useful to consider the second topic when considering the first.”  —  Wiki Or, to put it differently,   Relevance   is all about   connections , between ideas, concepts, topics, communications, whatever. In short, when we want to know how   relevant   something is, what we’re actually asking is how   connected   it is to something else you are interested in. Relevance   becomes, well, relevant when we start thinking about accuracy in responses. For example, let’s say we’re talking about cats (because,   cats! ). If you ask me how many pictures of cats I’ve tweeted, and I respond   211   (note: not an accurat...

Face Synthesis Outside The Uncanny Valley

Image
/via https://bit.ly/2KjRUCw Remember  Polar Express ? For those who don’t (lucky you!), it was  the  ChristmasKidFlick™ for 2004, one supposed to impart the Xmas spirit, good cheer, Santa Claus, and all that. Instead, it ended up becoming the living (!) embodiment of the  Uncanny Valley , filled with dead-eyed CGI zombie kids, and was — there is no other word for it —  creepy . The thing is, humans tend to be  very  good at identifying and classifying behaviors in others, especially base don facial expressions. That little twitch in your beloved’s lip means that they found your behavior adorable, the almost invisible wrinkle by the left eye — you goofed up real bad. And when the behavior displayed doesn’t match what we expect it to look like (“ huh? why did the eye twitch like  that ? ”), it’s like there is a blaring siren saying “ THIS IS ODD WHY IS IT THIS WAY I DON’T GET IT ”. In short,  humanoid objects which appear almost, but not e...