Deep Learning, and Intel’s MKL-DNN
Intel’s MKL, for those of you who’ve never actually had to worry about math performance, is basically a library of hand-optimized code for seriousmath, including Linear Algebra (BLAS, LAPACK, etc.), FFTs, vector math, and whatnot. And yeah, it’s specific to Intel’s processors (hence “Intel’s MKL”) with effectively free licensing. If you’re doing any kind of heavy lifting in science / math / fintech / etc, you’re probably already using this somewhere in your stack!
MKL-DNN is a library built on MKL which adds in optimizations specific to Deep Neural Networks (hence the “DNN” part 😀). It’s been out since late 2016, but there are fairly frequent updates as new models and techniques become popular (and boy howdy, is the pace of change rapid in this field!).
At heart though, it takes a decent chunk of Neural Network primitive — things like convolution, normalization, tensor manipulation, etc. — and maps them to the SIMD instructions available on Intel processors (•). This mapping is key, since it can allow for operating on 16 numbers at a time (technically 16 single precision numbers, i.e. 512 bits)
At heart though, it takes a decent chunk of Neural Network primitive — things like convolution, normalization, tensor manipulation, etc. — and maps them to the SIMD instructions available on Intel processors (•). This mapping is key, since it can allow for operating on 16 numbers at a time (technically 16 single precision numbers, i.e. 512 bits)
It’s good stuff, has been available for Tensorflow, Microsoft CNTK, and Caffefor a while, and as of recently, is even available for Apache MXNet. If you’re developing locally sans-GPU (and on Intel, of course), you really should be using this.
That said, do note that you need to use versions of the tooling that are specifically compiled for MKL. For a variety of reasons (historical, performance expectations, compatibility, etc.), they “default” data formats are different across MKL-DNN and TensorFlow, MXNet, etc. Pretty much all of them are working on automagically switching formats between native and MKL, but till that happens you should make sure to use the MKL specific version!
(And yes, don't ask me how I found out 😡)
That said, do note that you need to use versions of the tooling that are specifically compiled for MKL. For a variety of reasons (historical, performance expectations, compatibility, etc.), they “default” data formats are different across MKL-DNN and TensorFlow, MXNet, etc. Pretty much all of them are working on automagically switching formats between native and MKL, but till that happens you should make sure to use the MKL specific version!
(And yes, don't ask me how I found out 😡)
(•) SIMD (Single Instruction Multiple Data) is a set of around 70 instructions that current Intel CPUs support that allow them to operate on multiple data elements at a time. It’s basically something like “add these 16 numbers to those 16 numbers” except that the “add” happens in one instruction instead of 16. (And yeah, it’s a bit more complicated, but…whatever).
Comments