Big Data Solution – Where Art Thou?
Did you know 90% of the world’s data was created in the last two years? Given the digital enablement of various business artifacts, this proliferation of data will continue, but more data are becoming neglected or not utilized. This growing torrent of information has created a new market opportunity called “Big Data”. With the increased amount of data, traditional technology solutions do not meet the demand economically or technologically. Thus, we are witnessing various innovations across many of the technology verticals particularly in: (i) storage; (ii) database / data warehouse; and (iii) analytics.
Despite these innovations, unstructured data presents a new challenge within this market. According to IDC, there will be three times more total unstructured data than total structured data by 2014. In an attempt to address this, we have witnessed many of the storage companies consolidating and expanding their capabilities with database functionalities. This trend was primarily driven by people’s concerns of the inability to capture all of the information being generated. However, what good is the data, if you do not know what you have?
This year we have seen two companies, Autonomy and Endeca, acquired to provide the ability to search through stored data. Although this solves the question of what data is available, given the volumes of information, this search capacity only presents a partial tool. You need another tool that will allow you to analyze which of these data sets are more important to you — analytics. This chain of new challenges does not stop there. Once you narrow down the data set to a manageable level, you need the ability to visualize what you have and how it is related to other information. Additionally, more and more companies are now looking to perform predictive analytics in an attempt to make predictions about future events. Lastly, all of these different tools do not exist as one solution today, so they need to be integrated as one with extraction, transform, and load (“ETL”) capabilities.
Over the past 18 months, we have seen a pickup in M&A activity around the three markets noted above, and we expect to see more transactions announced as companies build a solution to address these Big Data challenges. Unfortunately, this holistic solution does not exist today, leaving us to wonder: “Big Data Solution – Where Art Thou?”