Big Data Raises Big Questions

In today’s Information Age, the rate at which data is growing is mind boggling and it is literally an explosion visible in all fields.  Our DNA, a whooping 3.2 GB, is dwarfed when compared with the size of data that we generate in one single day - 2.5 exabytes! This data, is coming from a multitude of sources such as social media sites, weather sensors, medical data, transaction records, etc.

The 3Vs of Big Data

The data size is not only humongous but it is also growing at a tremendous rate. Moreover, the data is of a wide variety and is coming from a number of different sources and the speed of data in/out is very fast. Collectively, these define the characteristics of Big Data - the so called 3Vs:

  1. Increasing Volume
  2. High Velocity
  3. High Variety Information

The Value in Big Data

The important question, related to Big Data, is how to handle such volumes of data.  That is, how to store, curate, manage and process it, how to enhance decision making, gain rich insights, improve system agility, and devise wise ways for the overall optimization in the face of complex inter-relationships between data of such different types. Big questions indeed. If we look at the above questions, they all boil down to one single word- value; how to get real value out of this behemoth of data? Of course, the term “value” has a lot of different interpretations in different scenarios.


Big questions translate into new technologies on the answer part. To tame this tsunami of Big Data, big tools are coming up. Apache Hadoop is one example. Cloudera, Hortonworks, Datameer are some examples of Hadoop down stream companies.

On the ethical front, issues of privacy and the danger of big brother watching over the shoulders have been raised. The real question, however, is how we, the homo sapiens, are going to apply this huge data and the enormous power that the analytics and visualizations has put into our hands for the overall good of this world and the universe at large.