Python 3.2 Beta 1 Introduces New Concurrency Features
Python 3.2 Beta 1 adds a futures library for concurrent programming that is part of a new concurrent top-level package. The main goal is to simplify the parallelizati...
Python 3.2 Beta 1 adds a futures library for concurrent programming that is part of a new concurrent top-level package. The main goal is to simplify the parallelization of code and to make it simple to launch and control asynchronous and concurrent operations. In addition, this new release provides many improvements to existing multithreading features.If you want to translate multicore power into application performance with Python, the new features introduced in Python 3.2 Beta 1 will help you simplify your code. The java.util.concurrent.package package for Java inspired the design of the new concurrent.futures module for Python. This new module allows managing both processes and threads with a uniform high-level interface. This approach is very useful when you want to write code that can use either processes, threads, or a combination of both to parallelize the execution of code.
The abstract concurrent.futures.Executor class defines the interface for the two asynchronously executing callables:
- concurrent.futures.ProcessPoolExecutor: It uses a pool of processes to execute calls asynchronously.
- concurrent.futures.ThreadPoolExecutor: It uses a pool of threads to execute calls asynchronously.
You can control the maximum number of either worker processes or threads for both ProcessPoolExecutor and ThreadPoolExecutor.
There is a new concurrent.futures.Future object that encapsulates an abstraction of the features common to remote procedure calls, processes, and threads. Thus, you can use the Future object to check the status of an asynchronous call, no matter whether this call runs either in a new process or in a new thread. The Future object allows accessing the results of an asynchronous operation and provides support to exception handling, cancellation, and timeouts. It is very easy to wait for certain Future instances by using the new concurrent.futures.wait module function.
Python 3.2 Beta 1 introduces a rewritten GIL (short for Global Interpreter Lock). The new version of the mechanism for serializing execution of concurrently running Python threads really reduces lock contention and you can tune the new absolute duration of thread switching parameter through the sys.setswitchinginterval(). If you have code that uses multiple threads, you can test the code with this beta version and you will have great chances of achieving an improved throughput. However, remember that this release is still in the Beta 1 version.
Now, the acquire() method for regular locks, recursive locks, and the semaphore (threading.Semaphore) accept a timeout argument. If you have code that uses locks or semaphores, you will be happy with this optional parameter that allows establishing timeouts.
If you are interested in learning more about these new features in Python 3.2 Beta 1, you can read Raymond Hettinger's article "What's New In Python 3.2" and Brian Quinlan's PEP "Futures - Execute Computations Asynchronously."
