100-Year Outlook on Data
It's hard to believe that just a decade ago, the only way to process and analyze large data sets involved manipulating and querying relational database management systems (RDBMS). Database professionals served as gatekeepers of this information, and data was structured in columns and rows—with subsequent data input having to conform to this schema to be stored. And though the data explosion was still years away, managing structured data was already pushing the limits of available hardware, forcing enterprises to make expensive capital expenditures on supporting technology.
Big Data on the Rise
Fast forward to the present day, where unstructured data is the king of the hill. RDBMSs are still in use, but big data is now the key technology in play—powering most of the SaaS and business apps in use.
The vast majority of data generated now is unstructured, arising out of social media activity, mobile use and the growing myriad of "Internet of things" (IoT) devices that control everything from home lighting to pacemakers. Cloud computing has made the cost of supporting hardware a scalable, on-demand affair, with sophisticated computing and tools now available to everyone—not just enterprises and firms with deep pockets.
Despite all of this, data is still largely underutilized.
In 2013, a scant 22 percent of information in the digital domain was considered useful. Furthermore, less than 5 percent of that useful data was actually analyzed by stakeholders. Looking forward, it's clear that the evolution of data-related technologies will certainly lead to better, more relevant information for driving critical decision-making.
For the data management industry, the continuing challenge is how to improve mining, storage and analysis capabilities for data, which is increasing both in volume and complexity.
Bigger Data on the Horizon
Just as the rise of social and mobile media propelled information technology into the age of big data, the IoT renaissance will lead to a next generation of data technologies with both accurate and meaningful predictive capabilities.