There's no shortage of data science smarts

Powerful new big data tools demand analytics expertise -- but a growing body of shared knowledge and a new generation of self-taught experts will fill the gap

"Smarts" are the ability to adapt to changing conditions, as well as to adapt conditions so that they are more conducive to human life and prosperity. Continuous adaptation is essential to human sustainability in a turbulent world.

This vision depends on advanced analytics, which in turn relies on cultivating a cadre of data scientists to build the big-data-powered applications that drive it all. To the extent that we instrument our businesses and communities with big data, advanced analytics, real-time sensor grids, automated feedback remediation loops, embedded decision automation, and other optimized infrastructure, we can ensure a sustainable human footprint on the planet.

[ Download InfoWorld's Big Data Analytics Deep Dive for a comprehensive, practical overview of the field. | Subscribe to the InfoWorld Daily newsletter, our summary of the key news for technology development and IT management. ]

Data scientists are among the most important developers in the era of big data. They include statistical analysts, data miners, predictive modelers, computational linguists, and other professionals whose job is to find deep insights in large, complex data sets. You can't unlock the full value of big data in your business if you don't bring together your best and brightest data scientists and give them the tools they need to do their job with maximum productivity.

Some people believe that the era of big data will screech to a halt due to a shortage of business-oriented data scientists. However, I think those concerns are exaggerated, for several reasons:

  • As more data discovery, acquisition, preparation, and modeling functions are automated through better tools, today's data scientists will have more time for the core of their jobs: statistical analysis, modeling, and interaction exploration.
  • Data scientists are developing fewer models from scratch. That's because more and more big data projects run on application-embedded analytic models integrated into commercial solutions.
  • Data scientists will increasingly be sourced, as needed, from external professional services firms.
  • More organizations are establishing data science centers of excellence to foster standardization, reuse, collaboration, governance, and automation within and across data science initiatives.
  • Open source communities and tools will greatly expand the pool of knowledgeable, empowered data scientists at your disposal, either as employees or partners.
  • Autodidacts -- including self-taught, uncredentialed, data-passionate business analysts -- will come to play a significant role in many organizations' data science initiatives.

The bottom line is that the core data scientist aptitudes -- curiosity, intellectual agility, statistical fluency, research stamina, scientific rigor, skeptical nature -- are widely distributed throughout workforces everywhere. If your business's survival depends on big data, you need to recruit, train, and incentivize these people. Just as important, you should give them the tools and environment where they can do their best work, per this discussion. Your business success depends on it.

This article, "There's no shortage of data science smarts," was originally published at For the latest business technology news, follow InfoWorld on Twitter.

Copyright © 2013 IDG Communications, Inc.