Big Data Spending to Hit $8M Per Organization, IDG Says

The average organizations will spend about $8 million on big data projects this year, according to IDG Enterprise’s Big Data report for 2014, which found that nearly half of organizations surveyed have already begun big data projects or have plans to begin one. The biggest enterprises are investing the most heavily in big data, but smaller firms are not far behind.

Big data is beginning to make a big mark on organizations around the globe. The challenges that today’s growing data volumes present, and the opportunities that emerging technologies like Hadoop provide, together are changing the face of IT as we know it.

Organizations are struggling under exploding data volumes, according to IDG Enterprise’s recent survey, which was conducted online and involved 751 readers of IDG’s esteemed magazines. The survey carries a margin of error of 3.2 percent, the company says.

According to the survey, the average organization today is measuring an average of 164 TB data. Over the next 12 to 18 months, that number is expected to balloon by 76 percent, to 289 TB. Nearly a third of the biggest enterprises, meanwhile, will be working with a 1 PB dataset by the end of 2014, IDG Enterprise found.

Source: IDG Enterprise Big Data Study, 2014

To deal with this digital surge, organizations are ramping up big data initiatives and bolstering their capabilities to capture, store, sort, share, analyze, and visualize their data. These big data initiatives involve investments in hardware, software, and personnel.

On the hardware front, 49 percent of respondents say they plan to increase investments in servers over the next three years to handle the data crunch, followed by 47 percent increasing spending on storage arrays. The survey found 44 percent of respondents plan to boost spending on cloud infrastructure over that timespan, while 43 percent plan to boost spending on discovery and analytics software. About 30 percent of respondents report using an open source big data framework, such as Hadoop.

People with “data” in their titles are in high demand (as are news publications with “data” in their names). According to IDG Enterprise’s survey, 27 percent of survey respondents plan to hire data scientists over the next 12 to 18 months, followed by data architects (24 percent), data analysts (also 24 percent), and data visualizers (23 percent). Twenty-one percent are looking to hire business analysts and research analysts over that timespan, the survey found.

Source: IDG Enterprise Big Data Study, 2014

What do organizations hope to accomplish with this big data buildup? According to IDG Enterprise’s survey, 59 percent of respondents want to improve the quality of decision-making, while 53 percent want to increase the speed of decision-making. Getting better planning and forecasting, and developing new products/services and revenue streams are goals of 47 percent of respondents. Keeping existing customers or acquiring new ones is a goal of 44 percent of those who took the survey.

2014 will be a formative year for the big data futures of many organizations. The survey found that 49 percent of the respondents are in the process of implementing or are likely to implement a big data solution. At this time, only 12 percent of survey respondents already have a big data project deployed. That may seem like a low number, but it is in line with other surveys, and shows how green the big data market really is.

No worries, says Matthew Yorke, CEO of IDG Enterprise, who reports that three quarters of the survey takers expect big data to be in mainstream use within the next three years. “It is not surprising that 70 percent of enterprise organizations are investing in big data, compared to 56 percent of SMB [small and midsized business] organizations,” he says. “Some of the biggest winners in this, within enterprise organizations, will be marketers who partner with IT to better understand their customer data, activities and drivers.”

Related Items:

Datanami Dishes on ‘Big Data’ Predictions for 2014

The 2013 Big Data Year in Review

The Big Data Market By the Numbers