Analytics within a company is typically about standards and processes. Data is generated and obtained from a variety of sources, painfully scrubbed and sanitized, and finally imported into a Business Intelligence system of some kind. The petitioners then line up at the doors, requesting guidance. Supplicants get chosen based on some type of appropriate criteria - Flavor of the Month, Marketing's latest brainstorm, whatever - and then encoded into complex BI queries in scenes reminiscent of an Asimov short story, except without the benevolent presence of
Multivac. The entire process can take weeks or months. Admittedly, once a specific process is put in place, turnaround is quick, but there is little room for serendipity, spontaneity, or synergy.
In the entire sequence described above, there is one area that organizations have gotten remarkably good at - generating vast quantities of data. Given the crazy decline in the cost of storage (A terabyte! In my laptop! And its not surprising!) nothing gets thrown away any more, and more and more data is kept around. This data is all over the place - in marketing, sales, support, IT, business systems, voicemail, email, social interaction - basically everywhere in the organization.
In fact, a lot of it is not even recognized as being data. How many organizations actually keep track of the interactions their employees have over email, voice, Facebook, twitter, etc. (lets not go down the Big Brother aspect here. Thats a separate discussion!)? Most organizations, if anything, tend to have stringent rules against their works employees interacting with customers (imagine!), resulting in all these interactions being "out-of-band", thus creating vast dark-nets of data, valuable
(
oh so valuable) data that is completely invisible to the organization but containing treasure-troves of relevance.
Take this a step further - consider the ancillary information that is not stored or maintained by organizations. When a customer calls in to place an order, whats the weather like where the customer called from? This is information that is probably trivial to obtain (even automatically), and could very well be relevant to the product being sold (umbrellas! thermostats! air-conditioners! sunscreen! pillows!) but one that isn't actually tracked. Would it not be nifty to be able to correlate pillow sales with inclement weather? (humor me here).
As Anjul Bhambri of IBM puts it -
Often, the clues that lie in big data can be the key to an enterprise’s future success, offering insights to optimize supply chains, uncover consumer behavior patterns, identify traffic and energy patterns, and much more. Rather than be a burden, all of this newfound knowledge can allow businesses to make effective long-term and real-time business decisions.
You got it - welcome to the world of BigData. And in this world - as you may already have figured out - organizations already have (or have access to) all the data they need. They already exist in a BigData world. There is
Volume, Velocity, and Variety in the higher-order-of-infinity amount of data that they generate. There is even an entire field known as
Predictive Analytics, whose whole aim in life is to analyze the present (the
now) to predict the future (the
soon to be).
"What", you may ask, "What is the problem with that?"
The problem, my dear Watson, is that
Math is Hard. Business Lines already have issues with their own P&L - imagine the entertainment when they try and justify hiring a couple of statisticians. And this is for scenarios where you
know what it is that you are trying to accomplish. Now try and imagine the
even greater entertainment when the Business Line tries to hire statisticians to figure out what they
don't know, based on data they
don't know they have.
Joy, right?
What they need to get better at is in
Collaboration. This is, in part, because Collaboration is Good - (as is sitting around the campfire singing Kumbaya). But this is good
mainly because organizations no longer have a choice. Collaboration is Necessary (with a capital N), since this data, which is so vital to the future of the organization, has meaning and context that is only relevant to parts of the organization that are almost certainly
not be the ones processing the data. But if these Business Lines start talking to each other about what they have, and have (easy! thats the key!) access to said data, well, that changes the game. In short,
Start Collaborating and The Tools Will Follow.
Seriously. It all started with Google Analytics - which pretty much anybody can figure out the basics of with about half an hour of messing around. Its moved on, with a newer breed of tools like
Bime,
D3,
Tableau, and
Qlikview, which bring visualization of complex data into the realm of the mortals. Math is still hard, but just like you don't need to understand the
Navier-Stokes theorem to go swimming, you don't
have to understand complex regression analysis to work with these tools.
Take a look at some of the
examples, they're mindblowing, and they're
easy.
And
that, in the end, is the key. The unfortunate truth of our world is that while
If you build it, they will come was great for Kevin Costner, it really doesn't work all that well in the business world. Supply-side economics didn't work in economics, and it doesn't really work here either. Demand, however, creates supply, and that too in spades. The new generation of BI tools are
easy, so easy that even non-practitioners are getting a handle on what it means, resulting in unintentional collaboration across various departments in organizations. I expect to see the fruits of this snowballing over the next year.
So yeah, this could have been a post about all the cool nifty analytics and BI tools that exist out there in the world of BigData, Business Intelligence and NoSQL, but it really doesn't matter. Cart before horse -
Start Collaborating and The Tools Will Follow...
Comments