Discover PerformanceHP Software's community for IT leaders // September 2014
Field notes: How analytics leaders thrive
Smart companies are developing strategies to deal with a tool proliferation, talent shortages, and making Big Data investments pay off.
When Big Data hit the big time, hardly anyone was prepared—not the enterprise CEO, not the CIO, and not vendors selling data tools. Enterprises are stuck trying to navigate constant change and a proliferation of tools and strategies.
To find out how enterprises are coping, we spoke with Walt Maguire, chief field technologist for HP Vertica. Maguire spends his time on the ground, getting close to the IT leaders who are knitting tools and human processes into a modern Big Data analytics practice. These days, he says, the biggest challenge isn’t getting tools that can do the job—it’s finding the talent. Maguire, whose first role with Vertica was as an engineer helping to build it, says the most successful enterprises are working hard to develop Agile processes and are thinking about their data technology requirements as a portfolio.
Q: How has the Big Data wave impacted the vendor marketplace?
Walt Maguire: We have data coming from devices that simply didn’t exist 20 years ago. It’s also coming from things that existed but weren’t writing data, like dishwashers and refrigerators. When the web first came out, I wasn’t producing a clickstream of information about my demographics and psychographic behaviors.
This explosion of data blindsided entire industries on the vendor side. They had developed an entire ecosystem based on a pricing and revenue model that simply didn’t work once we reached those new volumes. That caused a lot of churn on the tool and vendor side, and has driven tool proliferation.
Q: From an enterprise perspective, how do you move forward with so much uncertainty in the marketplace?
WM: First, you have to be realistic. It’s highly unlikely that you’ll begin a project knowing exactly what the end state needs to be. I often recommend taking your Big Data/analytics/data science team—whatever you’re going to name it—and setting up their development process with an Agile-like methodology. Create sprints with goals, and at the end of each sprint, stop and take stock of where you ended up. Then plan the next sprint, and so on.
But you have to choose technology that you can actually work with in that way. Some things that you can buy will have no instrumentation, like buying a 747 that doesn’t have a cockpit. There are others that don’t scale so well, and there’s a lot of stuff in between. So cautiously picking a toolset that aligns with Agile methodology and the skill sets in your organization is very important to success. Also, it’s beginning to dawn on people that no single tool is going to solve all their problems. They’re beginning to understand that they really need a portfolio.
Q: What’s the most effective design for a data science team?
WM: Today we’re seeing analytics teams or individuals embedded in the business units—as well as infrastructure and data science people—and their job is to advance that capability in line with the business as it moves forward. It’s changing the IT delivery model.
Q: How does adding an embedded analytics team change workflows for the better?
WM: Twenty years ago, if you wanted a report from IT, turnaround time was measured in weeks or months. That’s not a workable model. Analytics is a highly iterative process: I ask one question of the data, and realize from the answer that I need to ask a different question or get different data—in other words, to refine my thinking. Making those cycles go fast is key to advancing the insight. By embedding those people in those units, you typically get much faster answers.
And this is where the technology supports it. Something like Vertica takes those long query times off the table. Now it runs in seconds, so an analyst can be much more productive. This is where the two things line up—the technology and the business process are complementary.
Q: Is culture change a pervasive problem?
WM: Some organizations are strategically avoiding Big Data because it means introducing a lot of technology change and a lot of skill change.
For those who are going after it, the main challenge is acquiring the data science skill set. Obviously, data scientists today are at a premium. Growing that talent internally can be very useful, since a data scientist is a somewhat fuzzy role for most organizations. So you can very often find those people internally.
There used to be maybe 20 people maintaining the data environment and three analysts. In the new world, what we need are three people maintaining the environment and 20 analysts. The smart organizations are recognizing this, and they’re giving people a path to build those skills and transition into those roles.
Q: Does retraining of the old guard work? Do people make the transition easily?
WM: Most do. I think the bigger problem is people running out the door after they’ve retooled. Some markets are highly competitive for those skills. So retooling people is not so hard, because there’s a lot of excitement around it. The question is: How do you retain them once you get them into that role? That’s tougher.
Q: Have you seen any good innovations for that?
WM: A lot of it comes down to positive engagement: finding ways of keeping it fun or otherwise fulfilling. For those companies that do a good job of retaining people, a key element is making sure that you are providing a good experience. Some of that is culture, some of that is conscious design.
Q: How is technology helping with the human side of Big Data analytics?
WM: It used to be performance that created bottlenecks in analytics. The technology has gotten faster and, as a result, it’s now the human decision-making process that is becoming a potential bottleneck. I can’t just randomly throw 10,000 variables at an outcome and assume that it’s fine. I have to apply some judgment.
But what if you had a tool that could update a regression model for you, in seconds? There are some products and services that are starting to show up in the market which do that sort of thing. They’re designed to take the data science process and accelerate it. This way, I can take what used to take a month, and I can assess and determine a likely model in an hour.
Q: Are most organizations thinking about their Big Data in a holistic way, or are most of them still focused on individual problems?
WM: It’s a very mixed bag. Some approach it like a science project. They say: “We know it’s important. We’re not sure yet what we’re going to do with it, so we’re going to go do something, and then we’ll move it along and see if we can find some use for it.” Those projects typically don’t work out well.
At the other extreme, there’s the approach of taking the existing business function and making it somewhat better. The outcome is very quantifiable. I can measure it in dollars. But the overall value to the business isn’t as high. They may have saved some money or made a process somewhat faster, and there’s nothing wrong with that. But that’s not where the real value of Big Data is found.
The group in the middle, though, find a couple of high-value use cases and build out projects to deliver meaningful outcomes for those use cases. The rate of success there is very high. It’s also typically very helpful for the business. It can be like opening up a whole new line of business.
Learn about the latest iteration of the Vertica analytics platform: Dragline. And see how it fits into a holistic Big Data platform with HP’s HAVEn.
HP Software’s Paul Muller hosts a weekly video digging into the hottest IT issues. Check out the latest episodes.
Welcome to a new reality of split-second decisions and marketing by the numbers.
Looking toward the era when everyone — and everything — is connected.
Introduction to Enterprise 20/20
What will a successful enterprise look like in the future?
Challenges and opportunities for the CIO of the future.
Dev Center 20/20
How will we organize development centers for the apps that will power our enterprises?
IT Operations 20/20
How can you achieve the data center of the future?
What the workforce of 2020 can expect from IT, and what IT can expect from the workforce.
Preparing today for tomorrow’s threats.
Data Center 20/20
The innovation and revenue engine of the enterprise.