Skewed Network Latencies on GCP — an Explainer

-
The folks at Azure are using FPGAs to accelerate network performance (and decrease host-CPU utilization) in their public cloud. There’s a nifty paper about this that was presented at NSDI ’18 (•). It’s worth reading (as is the writeup by Adrian Colyer), but the following graphs, showing the network performance of Azure vs GCP vs AWS, really caught my eye.
/via https://bit.ly/2HGZtBT
 Ignore the Azure bars (ok, don’t ignore them, they’re pretty nifty, but they’re not the focus here). Notice how GCP’s VM-VM latencies are better than AWS’, but the tail latencies are much worse?
Turns out there is a reason for this, and it’s embedded in the way GCP does networking, and specifically in the way their Andromeda network virtualization stack works.
/via https://bit.ly/2HGZtBT
You can ignore most of the stuff in the stack (shown above), except for the “Hoverboard”, which is, in effect a sidecar that is used to process any traffic flows that a VM doesn’t know anything about. The Hoverboard then proceeds to deal with these flows, up to a certain threshold, after which it basically offloads the flow back to the VM, and sets up a direct VM to VM connection for the traffic in that flow.
To put this differently, it goes something as follows
  1. 1. VM sees packet for a target it doesn’t know anything about
  2. 2. VM forwards packet to Hoverboard, and sez. “you deal with this”
  3. 3. Hoverboard processes the packet, and subsequent packets, until the traffic gets to a point where it could do with optimization
  4. 4. Hoverboard tells the VM to set up a direct connection to the target, and all subsequent packets go directly VM-VM
/via https://bit.ly/2HGZtBT
As a result, the hoverboard only deals with setup packets, and any traffic that is below thresholds. Everything else goes directly between the VMs, as shown above
In short, most traffic has low latency (VM-VM), while setup/infrequent packets have high latency (VM-Hoverboard-VM)
It’s a pretty nifty solution! If you’d like to know more about this, check out the NSDI ’18 paper on Andromeda (••), and/or this summary by (who else) Adrian Colyer.
(•) “Azure Accelerated Networking: SmartNICs in the Public Cloud” by Firestone et al.
(••) “Andromeda: Performance, Isolation, and Velocity at Scale in Cloud Network Virtualization” by Dalton et al.

Comments

Post a Comment

Popular posts from this blog

Erlang, Binaries, and Garbage Collection (Sigh)

-
Image
The nice thing about garbage-collection is that it is makes memory management absolutely transparent to you Of course, the above is true right up to the point when it stops being transparent, at which point it does a phase change from nice to clusterf**k of biblical proportions .  There are entire posts, books, and suicide notes that have been written about this, and I have no great desire to add to them, save with the minor comment that paranoia about garbage-collection is still justified , albeit only in the edge-cases for most people. That said, I'll give you that Erlang GC is in many ways much, much better positioned in this regard than, oh, Java, but it still rears up and bites you in the butt when you least expect it. ( More information about Erlang's GC vis-a-vis Java here , and Jesper's truly excellent description of Erlang's GC here ).  A story from the past should serve to illustrate this. Once upon a time, in a magical land far far away - well, Her...
Read more

Spot The Difference — Deep Learning Edition

-
Image
You’ve played  Spot The Difference  at some point, right? (Of course you did. We  all  did  ). Think about what you actually did when you were trying to Spot The Difference — in fact, just do it again with SpongeBob juggling above. You’re probably doing something like 1. Quick glance to the left. 2. Quick glance to the right 3. “ Spatula on the left. Wonder if there is one on the right? ” 4. “ Yup, there is. Sriracha on the right. Is that on the left? ” 5. “ Nope! It’s a ketchup bottle on the left! And there is nothing on the plate below it! ” etc. To summarize, what you’re basically doing is looking at each image alternately, identifying objects in each image, and trying to find the same object in the other image. As it turns out, what most Deep Learning systems do is something quite different. They basically do something like this 1. Figure out everything in the first image (“ ketchup bottle at 1 o-clock, empty plate at 3 o-clock, spatula at 1...
Read more