Rate Limiting vs Load Shedding
Ever wonder what the difference between rate-limiting and load-shedding is? Rate Limiting is all about controlling the rate of traffic sent or received by the network. In it's simplest form, it consists of rejecting any request that exceeds a specified count per time unit - think "If we get more than 10 customers per hour, we tell them we're busy". A slight variant of this would be concurrent rate limiting, which defines the maximum number of simultaneous requests that can be handled in a time unit - think "We can only handle 3 customers at a time". Load Shedding is all about preventing the system from getting overloaded in the first place. The idea here is that it is better to ignore some requests, rather than having the system crater and not be able to serve any requests - think "Let 911 calls through, and ignore the rest". This, of course, implies that decisions around load shedding - i.e., which requests to drop - are made based on...