Riak - Using Pipes to enhance MapReduce
In particular, I like the way they've gone all out on the plumber analogy, with fittings and all...
There are a few reasons you should be excited about running your MapReduce queries on Riak Pipe. First and foremost, Riak Pipe is designed as a work distribution system, and as such, it is better able to take advantage of the parallel resources available in the cluster. One small example of how Riak Pipe achieves this is simply by splitting the “map” phase processing into two steps: fetching the object from Riak KV, and transforming it. This allows the work of each step to happen in parallel; the next input will be fetched while the transformation of the last one is in progress (more...)
Comments