Hadoop Opportunity ‘Never Been Bigger’ Says Hortonworks CEO

Don’t be fooled by the Hadoop naysayers, Hortonworks CEO Rob Bearden told an audience at the Hadoop Summit yesterday. “The opportunity has never been bigger,” he said. “We may never have an opportunity this big again in our careers.”

Bearden delivered a positive and energetic keynote address at the Hadoop Summit, which drew about 4,000 attendees to the San Jose convention center, Hortonworks‘ biggest show of the year. The technology veteran set an optimistic tone with his comments, but he also addressed the elephant in the room, what Gartner last month coined Hadoop’s “fairly anemic” adoption.

To recap: Gartner analyst Nick Heudecker ruffled the feathers of the Hadoop faithful last month when he reported that Hadoop adoption is coming in below expectations. “Despite considerable hype and reported successes for early adopters, 54 percent of survey respondents report no plans to invest [in Hadoop] at this time, while only 18 percent have plans to invest in Hadoop over the next two years,” wrote Heudecker.

That report came on the heels of Wikibon’s latest market analysis, which found that spending on Hadoop software and subscriptions accounted for $187 million in 2014 (or less than 1 percent of $27.4 billion in overall big data spending). Spending on Hadoop software and subscriptions is expected to grow to $677 million by 2017, when the overall big data market will have grown to $50 billion. That’s still less than the $1.1 billion market capitalization for Hortonworks, which went public in December under the symbol “HDP.”

Hortonworks CEO Rob Bearden previously held executive positions at SpringSource, JBoss, Oracle, i2 and Manhattan Associates

But to Bearden’s way of thinking, Hadoop’s traction in the enterprise and its adoption rate is right where it should be at this point in time. “We’re just starting into an industry that’s transformative,” he said. “If you draw a parallel to where the relational database was five years into its formation, which at this point was 25 to 30 years ago, I think arguably Hadoop is on a much faster adoption and ramp curve.”

Bearden brought in other data points about Hadoop adoption, including those gathered by ETR, a market research firm. ETR founder Thomas DelVecchio shared the stage with Bearden to present some of the findings.

According to ETR, out of the 25 enterprise IT categories that it tracks, Hadoop carries the highest “absolute 2015 net spending intentions.” In other words, companies intend to spend more on Hadoop than on any other major enterprise IT category, such as information security, social media, cloud computing, etc.

While hard spending on Hadoop software and services may not be enormous at the moment, the future rosy to Bearden. “The opportunity that sits in front of us is simply staggering to me,” Bearden said. “I think the thing to recognize that’s so powerful about Hadoop is once it goes in, it’s kind of like putting electricity into the house for the first time. So much becomes possible and so much value gets created.”

Hadoop is rising for a simple reason, Bearden says: Because relational databases can’t handle the onslaught of new data types that have emerged just recently, he said.

Hadoop has a better upside than 25 other enterprise IT categories, according to ETR.

“The opportunity for all of us with Hadoop is being driven by the explosion of data that really started three to four years ago with the new paradigm data sources,” Bearden said. “It’s those new paradigm data sources that have grown and… put pressure on traditional data architecture and transaction systems. They’ve become very siloed, very constrained.”

At the end of the day, Hadoop wins and simply blends into the enterprise IT environment, says Forrester analyst Mike Gualtieri. “What percentage of large enterprise do you think Forester estimates will adopt Hadoop?” the analyst asked the audience during the keynote. Before answering, Gualtieri encouraged the audience to think about how many enterprises have adopted other technologies, like relational database management systems and Web servers.

“It’s 100 percent,” Gualtieri says. “No matter what the adoption rate is now, in the next couple of years we estimate that 100 percent of large companies will be using Hadoop in one form or another. These are exciting times.”

According to Bearden, the scalability advantage that Hadoop holds over existing architectures is creating new opportunities across every industry, he said. “There’s tremendous value in bringing all of this data together under management, especially in a central architecture,” he says. “That kind of modern data architecture…unlocks tremendous value and opens up use cases and business models” that were never possible before.

Hadoop is solving a new problem in a new way that creates new opportunities, but that doesn’t mean the old relational systems get replaced, Bearden says. “None of the existing capabilities go away,” he says. “I think that was always what was so powerful when we stated looking at the Hadoop opportunity….It didn’t have to be a zero-sum game for Hadoop to win.”

This “massive opportunity” that Hadoop represents is only possible because of the open source community, Bearden added.

“I’ve said from day one that Hadoop is too big of a platform and too big of an opportunity for any one company… and that this had to be driven by a community effort,” he said. “Hadoop is about a community. It’s about the users, the committers. It’s about the ecosystem and what you do to drive the tech forward and really try to transform it. I think we should be very proud, especially in the last three or four years of what we’ve accomplished together as a team and as a family.”

The Hadoop Summit runs through Thursday

Related Items:

Gartner: Hadoop Adoption ‘Fairly Anemic’

Does Hadoop Need a Reality Check?

Hortonworks Files for IPO, Will Trade Under ‘HDP’