Speed up your Python code in just 5 lines of code using concurrent.futures

In this video we quickly demo how to use concurrent.futures’ ThreadPoolExecutor() context manager to significantly speeding up downloading of our articles.

We hit a 4x speed increase, but tuning its settings you might even gain more. Of course be mindful how you use this to not cause too many requests to a web server.

However, you can use it with any callable (function) and we’ve seen people making other people really happy by the performance boost it provided in just ~5 lines of code!

Code (cleaned it up a bit): https://gist.github.com/bbelderbos/1b13763e1ba91cb5595cef8c1c9821c0
Docs: https://docs.python.org/3/library/concurrent.futures.html
Request headers: https://stackoverflow.com/a/27652558
Our Python tips book: https://pybit.es/tips/