Big data has provided a gold rush for IT job seekers. Soaring demand for skills like Hadoop and Hbase have pushed up salaries and perks over the last few years. But now there are signs of a slowdown as businesses learn that jumping into big data is not a recipe for instant ROI.
Premium pay for 58 big data-related skills and certifications declined by an average of 4.7 percent during the last nine months of 2014, according to a survey by Foote Partners, which tracks the market value of hundreds of skills across the technology industry.
“The problem that developed in 2014 is too many employers have not been satisfied with the return on their sizable investments in big data initiatives," says David Foote, the firm's principal analyst.
It’s important to note that premium pay is not the same as base salaries, which are still quite high, according to a recent survey by Dice.com, a large tech-focused job board. Indeed, the average base pay for five big data skills is well over $120,000 a year.
Despite the slowdown, Foote expects premium pay for big data skills to recover slowly over the next 24 months. “Big data capabilities are too critical for staying competitive,” he says.
Getting ROI from big data isn't so easy in many enterprises
As spending on big data outpaces overall IT spending, in the United States alone there will be 181,000 deep-analytics roles in 2018 and five times that many positions requiring related skills in data management and interpretation, IDC predicts.
That’s only three years in the future, so why has premium pay shrunk recently? As Foote points out, some companies are disappointed that they haven’t seen more immediate gains from the deployment of big data technologies and specialists.
Cultural and organizational barriers related to data governance, transparency, and sharing data across the enterprise have held back returns, he says. Those barriers don’t exist in less-hierarchical tech companies, which have been faster to see returns on big data. “You can’t simply hire Hadoop experts from Google and Amazon and expect to replicate the big data capabilities found at those companies," says Foote.
Even so, base salaries for some key big data skills are very high, according to Dice.com:
- Flume: $123,458
- Hadoop: $121,313
- Hbase: $126,369
- MapReduce: $127,315
- Pig: $124,563
There are also lots of jobs to be had in big data, Dice reported. As of mid-February, 4,700 jobs were posted on the board in those categories -- the job growth rate in those areas averaged about 57 percent from a year earlier.
But in what may be a leading indicator of a salary and perhaps a hiring slowdown, premium pay for some of the same noncertified skills shrank in the last nine months, according to the Foote data. They include Hbase, Flume, and Hadoop.
Linux and cloud are still hot, and big data may again be hot
Although big data may be cooling a bit, cloud- and Linux-related skills remain very hot.
Dice and the Linux Foundation surveyed more than 1,000 hiring managers in enterprises, small businesses, governments, and staffing agencies. Findings from the report include:
- 97 percent of hiring managers reported they will hire Linux professionals in the next six months.
- 42 percent of hiring managers say experience in OpenStack and CloudStack will have a major impact on their hiring decisions
- 88 percent of hiring managers report it’s “very difficult” or “somewhat difficult” to find Linux talent
That last bullet point suggests that job seekers with Linux skills have leverage to negotiate exceptionally good contracts because the supply of candidates is so tight.
A separate indicator, premium pay for cloud-related skills and certifications, also was strong, indicating that vendors with cloud certifications in Amazon, Salesforce, EMC, Hewlett-Packard, Microsoft, Oracle, and EMC VMware "are succeeding in their efforts to promote their cloud-related products and services with substantial market budgets," according to Foote.
The strong showing for Linux comes as no surprise. But the apparent weakness in big data is a surprise worth watching. Still, it’s likely that the trend spotted by Foote will correct itself before too long. If that’s the case, this is a good time to learn those skills and cash in when the demand returns later.