Posts

Showing posts with the label analytics

GPUs are coming to your database!

Image
Database technology for transaction processing and that for analytics have, kinda, moved in different directions. This has largely been from a “separation of concerns” perspective . Transactions are all about velocity, with people speeding up databases by making them in-memory (e.g. HANA), and/or adding columnar capabilities (e.g. Postgres, Oracle, and, well, everybody). Analytics, OTOH, are usually about volume — having   massively parallel   or distributed back-ends that can run through gobs of data very fast (e.g. Vertica, Redshift, Netezza, etc.) is pretty much table-stakes in this world. Mind you, when one talks about   massively parallel , GPUs should come to mind, and it clearly did for   Kinetica ,   SQream , and   MapD   who are all using GPUs to run the parallelized parts of SQL queries in their analytics databases. The result, as you might imagine, is a   huge   speed up in query processing for analytics. MapD, in partic...

BigData - Tornado Tracks edition

Image
Theres a ridiculously cool map out from IDV Solutions titled 56 years of tornado tracks, by F-Scale . It is, as you would expect, 56 years of tornado tracks overlayed on a map of the U.S. And boy oh boy, is it ever a doozy - check out the hard stop as you move west... Small version below, but click to embiggen...

Get your Minecraft server connections here! Live!

Image
stats.minecraft.net puts together a sweet redis + node.js + socket.io app to do this. Its pretty wicked - a realy nice example of BigData / Analytics in the modern style, go check it out ! (note:  its live, the image below is just that, an image, go to stats.minecraft.net   instead!)

BigData - Size vs. Structure

Image
Monash has a slightly different categorization of BigData, breaking it up into Size vs Structure .  The main idea is that while we tend to think of BigData as The Three (or Four) Vs , there is actually a distinction here between    Size : Volume, Velocity    Structure : Variety (,Variability) To  quote... Relational big data   is data of high volume that fits well into a relational DBMS. Multi-structured big data   is data of high volume that doesn’t fit well into a relational DBMS.   Alternative: Poly-structured big data. Conventional relational data   is data of not-so-high volume that fits well into a relational DBMS.   Alternatives: Ordinary/normal/smaller relational data. Smaller poly-structured data   is data for which   dynamic schema capabilities are important, but which doesn’t rise to “big data” volume. The point being that instead of thinking of all these in the same space, different combinatio...

BigData (Competitive Analytics Edition)

Image
Data Analytics is in all our futures .  Mind you, when I say Data Analytics , I'm not referring to complex Business Intelligence systems in the High Temples of Enterprises . I am referring to tools that make our lives radically simpler, tools like Siri, which have successfully used Analytics to make a ridiculously complex task even more ridiculously simple (*). Enter Kaggle -  a company dedicated to the proposition that Data Analytics can be a sport (albeit one for the brainy amongst us).  You give them access to your data, the problem, and a prize.  They (Kaggle), outsource the problem to their gaggle of smart people around the world, who compete with each other on finding the best solution to the problem.  The winner gets the prize. It hits all the right marks - competitive natures (FTW sucker!), prizes (money!), and the spirit of exploration.  Also, for those companies that don't want to give up sensitive information, Kaggle also has a serv...