Why so slow? It isn’t just because most Python runtimes are interpreters rather than compilers. It is also due to the fact that the inherent dynamism and the malleability of objects in Python make it difficult to optimize the language for speed, even when it is compiled. That said, Python’s speed may not be as much of an issue as it might seem, and there are ways to alleviate it.
Python performance optimizations
It isn’t always the fate of a slow Python program to be forever slow. Many Python programs are slow because they don’t properly use the functionality in Python or its standard library. Novice Python programmers often write Python as if it were C or Java, and leave performance on the table. Math and statistics operations can be sped up dramatically by using libraries such as NumPy and Pandas.
A common adage of software development is that 90 percent of the activity for a program tends to be in 10 percent of the code, so optimizing that 10 percent can yield major improvements. With Python, you can selectively convert that 10 percent to C or even assembly, using projects like Cython or Numba. The result is often a program that runs within striking distance of a counterpart written entirely in C, but without being cluttered with C’s memory-micromanagement details.
Finally, alternative Python runtimes have speed optimizations that the stock CPython runtime lacks. PyPy, for instance, is a just-in-time (JIT) Python compiler that converts Python to native machine code on the fly. PyPy can provide orders-of-magnitude speedups for many common operations.
Developer time often trumps machine time
Or to put it another way: For many tasks, speed of development beats speed of execution.
A given Python program might take six seconds to execute versus a fraction of a second in another language. But it might take only ten minutes for a developer to put that Python program together, versus an hour or more of development time in another language. The amount of time lost in the execution of the Python program is more than gained back by the time saved in the development process.
Obviously, this is less true when you’re writing software that has high-throughput, high-concurrency demands, such as a trading application or database. But for many real-world applications, in domains ranging from systems management to machine learning, Python will prove to be fast enough.
Plus, the flexibility and pace of development that Python enables may allow for innovation that would be more difficult and time-consuming to achieve in other languages.
When speed of development and programmer comfort are more important than shaving a few seconds off the machine clock, Python may well be the best tool for the job.
Further Python reading
The Python essentials from InfoWorld:
- Python developers profiled: What you use, what you do
- How Python makes programming simple
- How to get started with Python
- What is Cython? Python at the speed of C
- Cython tutorial: How to speed up Python
- 6 essential libraries for every Python developer
- Anaconda, CPython, PyPy, and more: Know your Python distributions
- Virtualenv and venv: Python virtual environments explained
Python for machine learning and deep learning:
- Why you should use Python for machine learning
- PyTorch tutorial: Get started with deep learning in Python
Python for data science:
- Julia vs. Python: Julia language rises for data science
- 5 essential Python tools for data science—now improved
- Get started with Anaconda, the Python distribution for data science
- What’s new in the Anaconda distribution for Python
- Introducing Pandas DataFrame for Python data analysis
