Survey results: How businesses are adopting and dealing with data

A glimpse into enterprise use of big data.

On December 7, 2011, we held our fifth Strata Online Conference. This series of free web events brings together analysts, innovators and researchers from a variety of fields. Each conference, we look at a particular facet of the move to big data — from personal analytics, to disruptive startups, to enterprise adoption.

This time, we focused on how businesses are going to embrace big data, and where the challenges lie. It was a perfect opportunity to survey the attendees and get a glimpse into enterprise adoption of big data. Out of the roughly 350 attendees, approximately 100 agreed to give us their feedback on a number of questions we asked. Here are the results.

Some basic facts

While the attendees worked for a mix of commercial, educational, government, and non-profit companies, the vast majority (82%) worked for a commercial, for-profit company.

What kind of organization do you work for?
Click to enlarge.

Most of the attendees’ organizations were also fairly large — more than half of them had 500 co-workers, and 22% of them had more than 10,000.

How big is your organization?
Click to enlarge.

We used this demographic information to segment and better analyze the other three questions we asked.

Big data adoption and challenges

We then asked attendees about their journey to big data. Fewer than 20% of them already have a big data solution in place — which we clarified to mean some kind of massive-scale, sharded, NoSQL, parallel data query system that may employ interactivity and machine-assisted data exploration. More than a quarter said they have no plans at this time.

How soon do you expect to implement a big data solution?
Click to enlarge.

While it’s relatively early days for adoption, more than 60% of attendees said they were in the process of gathering information on big data and what it meant to them. This is a spurious result at best: we’re of course selecting an audience that wants to be an audience. Nevertheless, the volume of attendees and their feedback suggests that deployment is ramping up: if you’re a big data vendor, this is the time to be fighting for mindshare.

What's the biggest challenge you see with big data?
Click to enlarge.

When it comes to actually deploying big data, companies have plenty of challenges. The big ones seem to be:

  • Data privacy and governance.
  • Defining what big data actually is.
  • Integrating big data with legacy systems.
  • A lack of big data skills.
  • The cost of tools.

Analyzing a bit further

These results might be informative, but what we really want to know is how they correlate. After all, Strata is a data conference: we’d be remiss if we didn’t crunch things a bit!

First, we wondered whether there’s a relationship between the size of a company and the kinds of problems it’s experiencing with big data.

Obstacles by company size
Click to enlarge.

Our results suggest that governance and skill shortages are problems for larger companies, and that smaller businesses worry much less about data privacy and integrating legacy systems. Cost concerns come largely from mid-sized businesses.

Then we wondered whether adoption is tied to company size.

Big data adoption progress by company size
Click to enlarge.

Among our attendees, smaller firms were ahead of the game: none of the companies larger than 500 employees said they had big data in place today.

We also found that educational, government, and NGO respondents didn’t list cost as a top concern, suggesting that they may have a tolerance for open-source or home-grown approaches.

Obstacles by company type
Click to enlarge.

Of course, the number of responses from these segments isn’t statistically significant, but it warrants further study, particularly for commercial offerings trying to sell outside the for-profit world.

Finally, we wondered whether the things a company worries about change as it goes from “just browsing” to “trying to build.”

Obstacles by time to implement
Click to enlarge.

Concerns do seem to shift over the course of adoption and maturity. Early on, companies struggle to define what big data is and worry about staffing. As they get closer to implementation, their attention shifts to legacy system integration. Once they have a system, talent shortages and a variety of other, more specific concerns emerge.

While not a hard-core study — respondents weren’t randomly selected, the number of responses within some segments isn’t statistically significant, and so on — this feedback does suggest that there’s a large demand for clear information on what big data is and how it’ll change business, and that as enterprises move to adopt these technologies they’ll face integration headaches and staffing issues.

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Related:

tags: , , , , , ,
  • http://dannyboisvert.blogspot.com/2012/01/web-templates-for-developing-iphone-app.html Harris Shaw

    Really a nice post you have done a great job with providing exact scale and with graph. Thanks buddy i really appreciate you post.

  • http://matchendirect.typepad.com Karl

    Very interesting article, the charts are really interesting, although I am quite surprised by some figures regarding big data in the companies.

  • http://jogosdemotos9.org/ jogos de motos

    We again asked attendees about their adventure to big data. Fewer than 20% of them already accept a big abstracts band-aid in abode — which we antiseptic to beggarly some affectionate of massive-scale, sharded, NoSQL, alongside abstracts concern arrangement that may apply interactivity and machine-assisted abstracts exploration. More than a division said they accept no affairs at this time.