Discover PerformanceHP Software's community for IT leaders // June 2014
Tapping the profit potential of dark data
Structured and semi-structured data lie dormant in your data warehouse—but it’s becoming increasingly easy to monetize them.
Millions of dollars may lurk somewhere inside your organization’s data warehouses and mainframes. You’ve collected, processed, and stored it; you just haven’t monetized it. You might call it "dark data," and if you can transmute it, the return on investment could be enormous.
Dark data encompasses a variety of data types, but essentially it’s semi-structured and unstructured data that you can’t see or use effectively. If Big Data is the new oil, then dark data is shale gas. And just as drilling for gas-rich shale means diving deeper into the earth’s crust, the process of extracting dark data requires another level of identifying, offloading, and deploying if you’re going to hit pay dirt.
"There’s a huge benefit from monetizing dark data," Steve Totman, director of strategy at Syncsort, said in a recent HP Vertica webinar. In fact, he said, there’s "billions of dollars’ worth" of benefits.
Framing the challenge
The problem mining dark data poses is that it usually resides in your mainframe, and critical mainframe data is hidden to the business.
"Data warehouses aren’t the dynamic forces they should have been," Totman said. "Legacy technology has posed significant barriers to accessing your organization’s dark data. To take advantage of it at scale ... the infrastructure you’d need becomes unaffordable."
Utilizing mainframe data has also traditionally been a time-consuming endeavor. You’re forced to choose between speed of decisions and quality of insight—not to mention how expensive it is to process and store mainframe data. Your monthly bill is based on CPU utilization, and the typical annual cost for the "average" $10 billion organization is $15.7 million: reductions in usage translate into instant OpEx savings.
At the same time, semi-structured and unstructured information like social media data can be the key to maximizing your marketing capabilities. In the webinar, HP Vertica VP Chris Selland said that "dark data gives you a much deeper understanding of your customers" when you move it into the light.
To do so, you need three things from your data warehouse:
- Massive scalability
- Real-time speed
- The ability to get to semi-structured and unstructured data
Selland discussed HP Vertica’s Flex Zone offering as a powerful approach to loading, managing, and exploring semi-structured data. After you’ve identified mainframe and heavy ELT workloads in your data warehouse, Flex Zone performs an exploratory analysis that produces analytic reports in real time.
Revealing dark data allows you to monetize and streamline workflows and eliminate legacy waste. The savings are immediate. The earning potential could be massive. And you’ve been sitting on it all along.
For more on accessing—and monetizing—unstructured and semi-structured data, watch the free, on-demand webinar, "Monetizing your dark data" (reg. req’d), and visit Vertica.com for an article and video on Vertica 7’s Flex Zone.
HP Software’s Paul Muller hosts a weekly video digging into the hottest IT issues. Check out the latest episodes.
Welcome to a new reality of split-second decisions and marketing by the numbers.
Looking toward the era when everyone — and everything — is connected.
Introduction to Enterprise 20/20
What will a successful enterprise look like in the future?
Challenges and opportunities for the CIO of the future.
Dev Center 20/20
How will we organize development centers for the apps that will power our enterprises?
IT Operations 20/20
How can you achieve the data center of the future?
What the workforce of 2020 can expect from IT, and what IT can expect from the workforce.
Preparing today for tomorrow’s threats.
Data Center 20/20
The innovation and revenue engine of the enterprise.