Here's a guest post by Chris Knight, who recently returned from the International Consumer Electronics Show and he's convinced that Big Data needs to have some top chefs.

Guest Post by Chris Knight

With billions of smart devices coming online, the variety and volume of new data sources will require businesses to move from back-end IT and data science to hiring "top chefs" of data.

Delivering on big data's promise for strategic business edge is now far more daunting after a string of news and products coming to market to at the International Consumer Electronics Show in Las Vegas last week. As some pundits noted, big data was a "quiet star" at the mammoth annual gadget fest.

Everything at the show seemed connected in some way, for some purpose, it was clear to anyone that the future will definitely be an Internet of Everything. From sleep monitors that determine the best time to wake up based on sleeping patterns; to self-driving vehicles, smart lighting systems, home and public safety,  everything was Wi-Fi enabled.

Cisco CEO John Chambers predicted the Internet of Everything would be a $19 trillion market by the end of this decade, connecting 50 billion things. According to Cisco, 50 billion objects will be connected to the Internet by 2020. Market research from Gartner says that the  IoE boom will create a $300 billion services market.

There's lots of exuberance but there here are some concerns. Alistair Croll wrote on O'Reilly Radar recently: "The Internet of Things and big data are two sides of the same coin; building one without considering the other is a recipe for doom."

One of the big problems Croll predicts is the issue of "datamandering" a.k.a. constant wars over data formats between leading companies, such as we've seen between Apple, Samsung, and Google.  As they "jockey for position to be the central hub of our health, parenting, home of finances."

An increase in the variety of data sources and formats is a huge challenge from an enterprise point of view. Businesses must solve the challenge of unstructured data as billions of "things" communicating in real-time, generate data storms of massive complexities. Big Data enthusiasts constantly talk about the three "Vs" -- volume, velocity and variety of data.

There's been constant debate about which "V" is the most important in the big data equation. With an expected onslaught in unstructured data from billions, if not trillions of connected objects coming online, variety is emerging as the most critical "V" to solve if Big Data is to fulfill its promises. 

Veteran Silicon Valley entrepreneur Sharmila Mulligan, CEO of ClearStory Data, is among those that agree that variety will become the most important "V" in big data analysis.

In her recent 2015 predictions post, she wrote: "As data sources proliferate, they'll be increasingly daunting and more costly to wrestle. Data variety will be the #1 challenge businesses will seek to overcome." (SlideShare of Mulligan's 2015 predictions: http://slidesha.re/1KpXR83)

Mulligan believes that democratizing access to data analysis and business insights across organizations is a vital next step to address this massive increase in data variety generated in the age of IoT.

"Static BI [Business Intelligence] dashboards like Excel are dead," says Mulligan. "They need to be replaced by new platforms that can crunch big volumes of data faster and let employees in different job roles collaborate on various data streams and sources on the fly to get the real story and holistic point of view."

A recent survey of 500+ business executives at Strata + Hadoop World last October, found that the majority of them are struggling to combine a wide variety of data sources and formats and make them usable. Nearly half (49 percent) of those polled said they have to work with eight to more than 15 different data formats. And that's before the future world of IoT kicks in. [See Infographic below.]

We have "Data Scientists" but what we might need is "Data Chefs," experts who know how to mix many different data source "ingredients," into recipes that produce the nutritional sustenance that powers an organization's key execs in making the right decisions, and moving towards the right goals.

Not every data source is needed just because it is available, too much data can spoil a decision just as not having enough data. Combining the right mix of data sources and delivering it all at the right time is an art, and is similar to the work of a  top chef who knows how much of each ingredient is needed to bring a dish together in time, and create a great experience. 

- - -

Chris Knight is a founder and creative director of Divino Group, a new brand marketing and PR firm based in San Francisco that's launching this week. Follow him on Twitter at @DivinoGroup.

http://www.clearstorydata.com