What is Big Data?

What if somebody asked you: “What is Big Data?” If you really understand something, you have to be able to say it in one word, and “variety” is the word that best represents what Big Data is. You might assume the defining word would be “volume,” but volume is a relative term—I may have ten megs or ten terabytes of data. So, the biggest difference between “small” versus “big” data is variety. Variety means we have different types of data to consider—and not only data that we have managed so far, which was structured data, meaning it contained metadata (data about the data, e.g., customer name, customer address, account number) which can neatly be put in rows in column and for which names can be given to the attributes or fields.

Whereas, how do you deal with data derived from music? You can use defining terms such as “symphony” or “aria,” but how do you play that music? Or, looking at piece of art, how do you express it? You can say that it contains these colors, it’s this particular size, it’s a rendition of a Van Gogh that’s hanging in the Museum of Modern Art in NYC, but that still doesn’t give you the feeling for it, and that is the variety—called unstructured data because it does not contain the attributes that describe it. Description is a big problem with Big Data, when you are dealing with a variety of data that includes unstructured data.