12 Apr What Exactly is “Big Data”?
The Information Age is not a new concept. We have been inundated with data for a long time. But the concept of Big Data is more recent. The term has been in use since the 1990s and was more accurately defined in the 2000s with the now well-known three Vs of Big Data:
Volume. There is simply a great deal more data coming into organizations than ever before. The sheer volume presents challenges for turning that data stream into useful information. Fortunately, new technologies, like Hadoop, are making it ever more manageable.
Velocity. Real time data streaming or near real time data streaming is now quite common. When combined with the sheer volume of data, you have a veritable fire hose of information incoming at any one time.
Variety. In the early days of the internet, most data came in the form of text. But now data comes in a wide variety of formats. In addition to text, it can be numeric, video, audio, email, and any number of other formats you are likely to trip across on the world wide web. This adds yet more complexity to the problem space of how to turn huge volumes of raw data into useful information that an organization can turn into strategic intelligence.
It is common for various sources to add additional traits to this list. But these three core traits are the ones most widely recognized as central to the definition of Big Data.
One of the reasons the term Big Data came into use is because this level of volume, velocity and variety of data requires different special handling. Because of the amount of information involved, one common tool used for grasping and communication about Big Data is data visualization. Like the saying “A picture’s worth a thousand words,” data visualization is particularly well suited to try to capture and convey the amount of information in a dense and efficient manner.
A few common myths:
Bigger data is better
Although large data sets can have their advantages, quality of data can trump volume. The term Garbage In, Garbage Out very much applies to Big Data.
Big Data is only for big organizations
In the world today, even small organizations can have access to enormous amounts of data. Today’s technologies, such as Hadoop, make it feasible for small organizations to tap into the power of using Big Data to make better, more informed decisions.
Every problem is a Big Data problem
Plenty of problems involve examining smaller data sets. The tools for Big Data can be unnecessary overkill when applied inappropriately.
Big data can empower organizations, but it also has its pitfalls. Among them, privacy issues. Just think of any number of data breaches at large organizations that have been in the news recently. So much information can give more opportunities for mishaps.
For more in depth discussion, see Harvard Magazine’s article Why “Big Data” Is a Big Deal