A few months ago, I reported that "for job seekers the big data gold rush is slowing." But, I added, "the pause could be temporary." It was.
After slipping during the last nine months of 2014, premium pay for big-data-related skills and certifications is once again on the upswing, according to Foote Partners, which tracks the market value of hundreds of skills across the technology industry.
In the first quarter of this year, premium pay for 35 big data-related skills that aren’t certified jumped 3.9 percent, while premiums for big data certifications inched up about 1 percent.
The firm predicts that the rebound will continue over the next year or two. "Big data skills are just too critical for staying competitive," says David Foote, the firm’s principal analyst. "They’ve expanded in popularity from a few industries to nearly every industry and market."
It’s important to note that premium pay is not the same as base salaries, which are still quite high, according to a survey by Dice.com, a large tech-focused job board. Indeed, the average base pay for at least six big data skills is well over $120,000 a year. For example, the average salary for Cloudera Impala, an open source MPP SQL query engine for mining data stored in Apache Hadoop, is a bit less than $140,000 a year, according to Dice.
Meanwhile, unemployment in the tech industry is lower than it’s been since 2008, right before the financial crisis swept away millions of jobs across the economy. According to the U.S. Bureau of Labor Statistics, tech unemployment was a surprisingly low 2.3 percent during the first quarter of this year, but it dropped even lower in April to 1.9 percent.
The most valuable tech skills in big data
With tech unemployment at a near-record low and interest in big data still growing -- despite some disappointments -- it’s not at all surprising that salaries are so high. As of the end of 2014 -- the latest available data from Dice -- here are the 10 big-data-related skills that pay the best and their average salaries.
- MapReduce: $127,315
- Cloudera: $126,816
- HBase: $126,369
- Pig: $124,563
- Flume: $123,186
- Hadoop: $121,313
- Hive: $120,873
- Zookeeper: $118,567
- Data Architect: $118,104
- Solr: $117,394
There is, though, a bit of a caveat: There aren’t that many jobs related to some of the newer and higher-paying big data skills. At the beginning of May, Dice listed 220 Zookeeper-related jobs, 563 for Pig, and 374 for data architect. All three showed strong growth over 12 months, but they started from a very small base.
Older and less glamorous as it may be, Hadoop is far and away the skill in the highest demand with 2,528 listings, an increase of 41 percent in the last 12 months.
Salaries for other IT specialties are on the rise as well. Mobile app developers, for example, saw salaries increase by 10.2 percent and now range from $107,500 to $161,500, according to a survey earlier this year by Robert Half Technology.
Baby Boomer managers don’t get big data
The rebound in premium pay -- a good measure of a skill’s popularity with business -- is puzzling. Foote, one of the first analysts to notice the slowdown in the value of big data skills, says he has yet to solve the puzzle.
But he has been very clear about the reasons some businesses have been disappointed with their expensive big data initiatives. "The problem that developed in 2014 is too many employers have not been satisfied with the return on their sizable investments in big data initiatives," he says.
Cultural and organizational barriers related to data governance, transparency, and sharing data across the enterprise have held back returns, he says.
Foote also points the finger at aging executives who aren’t yet comfortable with data-driven decision-making. "The tendency of Baby Boomer generation execs to go with their gut and experience and resist what advanced analytics tell them" is a major barrier to adoption, he says. "It’s become harder to manage forward if you spend too much time looking backward."
Foote’s report notes that there has been a surprising amount of volatility in premium pay for many certified and noncertified IT skills. If that’s the case, perhaps the recent ups and downs in premium pay for big data skills won’t have a simple explanation, or the changes might simply be blips that appear on the radar screen from time to time.
In any case, there’s little doubt that IT professionals with big data skills will continue to be in high demand and command excellent salaries.