Intel's Python distribution turbocharges data science

Intel Distribution for Python adds Intel's high-speed math libraries to the existing, highly convenient Anaconda version for data scientists

Intel's Python distribution turbocharges data science
NASA (Creative Commons BY or BY-SA)

The conventional wisdom about Python is that it makes your job easier, not necessarily faster. But if your job includes data science, Intel is working to make it faster as well.

Now available generally, the Intel Distribution for Python repackages the existing Anaconda distribution of Python, created by Continuum Analytics. The speed boost comes from the Intel Math Kernel Library (MKL), a collection of routines that leverage the capabilities of recent Intel processors to provide better performance for common data-science-related tasks like linear algebra or fast Fourier transforms.

Intel offered pre-release access to the distribution in March, but without the branding. It was immediately clear that providing a curated Python distribution with MKL speed built in was a major convenience. This is doubly true given the breadth of packages included with the distribution: NumPy, SciPy Pandas, Matplotlib, IPython/Jupyter, and many other science- and analysis-oriented staples.

Python users often struggle to assemble all these pieces into a working stack; this is why Anaconda does all that heavy lifting ahead of time. The distribution's conda installer helps in managing packages that sometimes require jumping through hoops to install. On top of it all, Intel is adding support for its MPI library to allow those packages to scale more effectively across multiple cores, with which Python still struggles.

Anaconda also brings two other major projects aimed at speeding up Python to Intel's distribution. Cython allows Python modules to be incrementally transformed into C, and Numba can speed up Python functions with a JIT compiler.

Anaconda has been long established as a major Python distribution for data analysts rather than for a general-purpose programming audience. Continuum Analytics began offering an enterprise edition of Anaconda last year along with the free version, and Intel is taking the same tactic with its distribution. The free edition of Intel Distribution for Python comes with forum support only, but when purchased as part of Intel Parallel Studio XE, it includes technical support for one year.