What Calvin and Hobbes Taught Me About Big Data!

Fans of comic strips still fondly remember Calvin and Hobbes, the story of a peripatetic 6 year old and his stuffed (but oh-so-real) tiger. One of my favorite strips shows Calvin digging in his backyard, pulling up “a few dirty rocks, a weird root and some disgusting grubs.” His exclamation to pal Hobbes: “There’s Treasure Everywhere”

I’ve found myself thinking more and more about that phrase as I ponder this year of “Big Data”. Now Big Data certainly has a lot of promise, and it certainly has been hyped to death. But it’s also – to many of us – pretty scary. Big Data requires lots of other big things, including big computer hardware, big staff, big project timelines and big budgets. The promise of Big Data is certainly strong – but for many of us it’s proving daunting and confusing as well.

We’ve been working on our own “Big Data” project here at Discovery Digital Networks for the past year. It’s not been without challenges. We discovered that one of our trusted data sources has numerous inconsistencies, another supposedly “real-time” source really couldn’t be trusted until about 15 days after each month closes, and we’ve also had to revamp our staffing assumptions and timelines. We’re starting to get some killer stuff out of that big data project, but it’s taken longer, cost more and exposed more warts than we ever expected.

You’re probably in the same boat. The promise of big data is huge – but the reality often requires a priesthood of data scientists to implement, and more mumbo jumbo than you’ll find at the New York Witch Festival.

But a funny thing happened on our way to Big Data heaven. The more we thought about it, the more we realized we didn’t need to wait to get real actionable audience and other insights. There really is data everywhere that you can tap into right now that will help you understand how customers, suppliers and others view you, your company and your products.

For us, it started with Facebook, Twitter, Google+ and YouTube. We started spending more time first monitoring discussions (or lack thereof) with our shows and hosts, and then began reaching out and joining the communities of our fans. We discovered an important (but obvious in retrospect) fact: if you ask people a question politely, they will usually answer.

From there we started using simple tools like Survey Monkey to create basic questionnaires to put in front of our viewers, using YouTube annotations and other means to drive response. We got all sorts of insights that way. And although we haven’t done this yet, others have told me that just asking a single simple question can be even more effective – and will drive much a better response.

We also realized that available data from YouTube and our other partners could be parsed and interpreted by hand – and combined with our internal data –to create real insights. So while our team of Big Data folks slaved away in the corner, we hired a smart data analyst who began cranking out real insights with essentially the little scraps of data left on the table after the big boys finished eating.

All of that really started helping the business. We learned, for example, that our DNews audience liked certain sorts of stories on certain days. So we launched “Space Fridays” to capitalize on one of those insights. We started seeing real patterns that let us increase the shareability of our content dramatically. And we started using those insights to deliver dramatic growth among some of our newer and older properties, including Animalist, Rev3Games and Scam School.

And then, towards the end of the year, I started talking with a fascinating new company that promises to deliver much of the insight of big data, but through a cloud-based and rapid data ingest and analysis service called DBMiner. We’re still working on a trial with them, but I’m optimistic that this will be a great middle ground between adopting a priesthood of big data and digging around for insights on your own.

Oh, and don’t expect “Big Data” – even when you do get it all really working — to be the Holy Grail either. According to my friend Mark Anderson – CEO of Strategic News Service and one of the smartest analysts and futurist in the business – next year we’ll be worrying more about Big Visualization. Or in other words just because you’ve got all that data warehoused, you still need great systems to help you really “see” inside, and extract real actionable knowledge.

Don’t get me wrong, I don’t hate Big Data. We’ll see lots of value going forward from our initiatives there. But just because you’re working on a big project, don’t forget that all sorts of insights are there for the taking – or the asking – just by poking around. To your data scientists they might be rocks, grubs and weird roots. But for the rest of us those insights are great treasure.