Channels ▼

Gastón Hillar

Dr. Dobb's Bloggers

Python 3.2 Beta 1 Introduces New Concurrency Features

December 13, 2010

Python 3.2 Beta 1 adds a futures library for concurrent programming that is part of a new concurrent top-level package. The main goal is to simplify the parallelization of code and to make it simple to launch and control asynchronous and concurrent operations. In addition, this new release provides many improvements to existing multithreading features.If you want to translate multicore power into application performance with Python, the new features introduced in Python 3.2 Beta 1 will help you simplify your code. The java.util.concurrent.package package for Java inspired the design of the new concurrent.futures module for Python. This new module allows managing both processes and threads with a uniform high-level interface. This approach is very useful when you want to write code that can use either processes, threads, or a combination of both to parallelize the execution of code.

The abstract concurrent.futures.Executor class defines the interface for the two asynchronously executing callables:

  • concurrent.futures.ProcessPoolExecutor: It uses a pool of processes to execute calls asynchronously.

  • concurrent.futures.ThreadPoolExecutor: It uses a pool of threads to execute calls asynchronously.

You can control the maximum number of either worker processes or threads for both ProcessPoolExecutor and ThreadPoolExecutor.

There is a new concurrent.futures.Future object that encapsulates an abstraction of the features common to remote procedure calls, processes, and threads. Thus, you can use the Future object to check the status of an asynchronous call, no matter whether this call runs either in a new process or in a new thread. The Future object allows accessing the results of an asynchronous operation and provides support to exception handling, cancellation, and timeouts. It is very easy to wait for certain Future instances by using the new concurrent.futures.wait module function.

Python 3.2 Beta 1 introduces a rewritten GIL (short for Global Interpreter Lock). The new version of the mechanism for serializing execution of concurrently running Python threads really reduces lock contention and you can tune the new absolute duration of thread switching parameter through the sys.setswitchinginterval(). If you have code that uses multiple threads, you can test the code with this beta version and you will have great chances of achieving an improved throughput. However, remember that this release is still in the Beta 1 version.

Now, the acquire() method for regular locks, recursive locks, and the semaphore (threading.Semaphore) accept a timeout argument. If you have code that uses locks or semaphores, you will be happy with this optional parameter that allows establishing timeouts.

If you are interested in learning more about these new features in Python 3.2 Beta 1, you can read Raymond Hettinger's article "What's New In Python 3.2" and Brian Quinlan's PEP "Futures - Execute Computations Asynchronously."

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.