TPUv2 Benchmarks (Yup, they’re fast)

-
RiseML benchmarked Google’s TPUv2, and — surprise! — they’re fast. Also — surprise! — they’re cost efficient too.
I know, shocking, right? After all, if you consider selection-bias, of course they wouldn’t release something that wasn’t fast, and of course they’d price it so that it would be cost efficient. Then again, this wouldn’t be the first time a company has done something stupid, so, hey, we should probably keep that in mind .
Anyhow, it turns out that the TPUv2 is around 7.3x faster than an Nvidia P100, and around 2.8x faster than an Nvidia V100. Yes, it’s more expensive, but you actually end up saving on a $ per hour basis — it’s that much faster.
It’s important to note that the TPUv2 is not just a speeded up version of the TPUv1 — it’s quite a different beast. Unlike the TPUv1, which was focused on inference, the TPUv2 is actually focused on speeding up learning. Inside, it has systolic arrays (•) which do 180TFLOPS of matrix math, with 64GB of memory available — against the Nvidia V100’s 125 TFLOPS / 16GB.
Mind you, the whole thing is “cloud-based”. You don’t get a TPUv2 enabled instance, instead, you are assigned a TPUv2 that exists somewhere in the “cloud”.
(•) Systolic Array: Parallel input data flows through a network of hard-wired processor nodes, which combine, process, merge or sort the input data into a derived result. Because the wave-like propagation of data through a systolic array resembles the pulse of the human circulatory system, the name systolic was coined from medical terminology. The name is derived from systole as an analogy to the regular pumping of blood by the heart.

Comments

Post a Comment

Popular posts from this blog

Erlang, Binaries, and Garbage Collection (Sigh)

-
Image
The nice thing about garbage-collection is that it is makes memory management absolutely transparent to you Of course, the above is true right up to the point when it stops being transparent, at which point it does a phase change from nice to clusterf**k of biblical proportions .  There are entire posts, books, and suicide notes that have been written about this, and I have no great desire to add to them, save with the minor comment that paranoia about garbage-collection is still justified , albeit only in the edge-cases for most people. That said, I'll give you that Erlang GC is in many ways much, much better positioned in this regard than, oh, Java, but it still rears up and bites you in the butt when you least expect it. ( More information about Erlang's GC vis-a-vis Java here , and Jesper's truly excellent description of Erlang's GC here ).  A story from the past should serve to illustrate this. Once upon a time, in a magical land far far away - well, Her...
Read more

Spot The Difference — Deep Learning Edition

-
Image
You’ve played  Spot The Difference  at some point, right? (Of course you did. We  all  did  ). Think about what you actually did when you were trying to Spot The Difference — in fact, just do it again with SpongeBob juggling above. You’re probably doing something like 1. Quick glance to the left. 2. Quick glance to the right 3. “ Spatula on the left. Wonder if there is one on the right? ” 4. “ Yup, there is. Sriracha on the right. Is that on the left? ” 5. “ Nope! It’s a ketchup bottle on the left! And there is nothing on the plate below it! ” etc. To summarize, what you’re basically doing is looking at each image alternately, identifying objects in each image, and trying to find the same object in the other image. As it turns out, what most Deep Learning systems do is something quite different. They basically do something like this 1. Figure out everything in the first image (“ ketchup bottle at 1 o-clock, empty plate at 3 o-clock, spatula at 1...
Read more