How many real Big Data applications are there?

I’d like to generate a bit of robust discussion and speculation about the use of analytics in the real world.

Off the top of my head, I would guess that:

  • The number of production grade, profit impacting big data applications worldwide is around 1,000.
  • The number of production grade, profit impacting optimization (prescriptive analytics) applications worldwide is around 10,000.
  • The number of production grade, profit impacting predictive analytics applications worldwide is around 100,000.

(By “profit impacting” I mean $50K or more per year. This means that startups with a website and a beta do not necessarily count.

    This thought exercise was prompted by the realization that the share of press of each category seems to be significantly different than the number of actual projects.
    The big question: am I full if it? If so, why? Either way, is this meaningful in some way?

Author: natebrix

Follow me on twitter at @natebrix.

5 thoughts on “How many real Big Data applications are there?”

  1. Good point! There aren’t enough “big” applications to go around. There are lots of little applications in the “long tail” – but they tend to be bits and pieces, and often they only need solving once. If you start a consultancy to solve lots of little applications for lots of small companies, the optimisation problem you’re going to be spending most time on is getting the necessary time with busy people who are daily subjected to a barrage of noise from other people who will also be fighting to sell them business services.

    I would be inclined to sell optimisation and analytics services as AI: I might be wrong, but I think that people are finally beginning to believe in it – might as well be first on the bandwagon!

  2. This makes sense, but also likely influenced that I am already biased in that direction. To give this post more weight it would be good to spend a couple of lines going through your thought process for those Fermi Estimates.

    1. Yeah, you’re right about that. I will mull it over a bit over the weekend and add a bit more to the post. The post as it stands is basically an expanded tweet!

  3. This would be crisper if you defined big data data. Indeed, we see more and more papers where big data is used as a synonym analytics.

    I also think you underestimate the number of optimization applications if I judge by the numbed of cplex licenses sold. What inflates the number of optimization applications are the supply chain packages that embed optimization . Should we count them for one application, or should we count the number of installations of such software? If we select the latter, which I think we should, then I’d say you are off by 2 or 3x here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s