Channels ▼
RSS

.NET

Improving Futures and Callbacks in C++ To Avoid Synching by Waiting


Simply put, a task is a proxy for an eventual value. The value might (but doesn't have to) be produced on a different thread. In addition to the value, the task can produce a side effect, such as adding a record to the log file or printing a message on the screen. In that sense, a task can be thought of as a generalization of a function, where the act of invocation is separated from the act of retrieving the result. In C++11, the tasks are called futures. The future is a class template that can be used as follows:

std::future<int> f = std::async([] {
    return do_work();
});
// do useful work here
int result = f.get();

Here, we instantiate the future by calling an std::async function taking a lambda expression. Depending on your implementation, std::async will either launch a new thread or schedule the execution of the function on a thread pool. Because the creation of the future is separated from the retrieval of the result, some useful work can take place between these events, potentially resulting in parallel execution. We get more parallelism by running more futures concurrently, and we can build higher-level constructs on top of futures to do so.

A future in C++ doesn't have to be backed by a concurrently running function created by std::async. Another concept, called the promise, is used to communicate the value to the consumer of the future. For example, we can create a promise, pass it to another thread, and let it set the value on it. In the meantime, we can execute some useful work before retrieving the value from the future:

std::promise<int> p;
std::thread t([&]() {
    p.set_value(do_work());
});
std::future<int> f = p.get_future();
// do useful work
int result = f.get();

Don't Keep Me Waiting!

For many C++ programmers,the introduction of basic concurrency concepts in C++11 is exciting news. We no longer have to deal with non-standard, sometime proprietary, and platform-specific libraries for parallelism. By using the standard C++11 facilities, we can be sure that our code will compile and run on all platforms — if not now, then in the near future, when C++11 compilers become universally available on every platform.

I will disappoint many readers by stating bluntly that futures in C++, in their current form, are ill-suited for building modern connected multicore programs.

First of all, futures and the associated primitives in C++11 form a programming model that is too low-level for our everyday needs. While it is possible to build higher-level constructs such as parallel_for or parallel_sort on top of futures, commercial libraries such as TBB and PPL will always outperform our scrappy implementations. The way to solve this is to bring those industrial-strength implementations into the Standard.

For a software library to be low-level is not necessarily a bad thing, as long as the components of the library are composable. Unfortunately, this is not the case with futures. As Bartosz Milewski opined, "What was omitted from the standard, however, was the ability to compose futures." I would add that the ability to compose futures in a wait-free way is particularly detrimental for connected multicore applications.

Specifically, consider the aforementioned example. When f.get() is called on the GUI thread, the thread blocks and is unable to process other messages sent to it by the event loop. As a result, the program stops responding to the user input — it freezes. Now imagine calling f.get() on the server side. A well-implemented thread pool will detect the blocking operation and spawn another thread that is able to do useful work. However, if many threads decide to block at the same time (imagine a Web service under heavy load), the thread pool will either spawn a large number of threads (which are expensive, as I mentioned earlier) or throttle thread creation, rendering the server temporarily unresponsive — even though the threads are sitting idle in the blocked operations doing no useful work. Things get worse when we need to compose multiple futures — this simply cannot be done without blocking every one of them individually.

Towards a Better Future

Futures are a great way of decomposing a program into concurrent parts, but a poor way of composing these parts into a responsive and scalable program. At Microsoft, we hear consistently from C++ developers that writing concurrent code is hard, and writing code that composes concurrent I/O and CPU-bound operations efficiently is especially hard. As part of our attempt to address this, we extended the PPL with the concept of the task.

The task in PPL is conceptually similar to std::future, with one important addition — continuations. Like std::future, the concurrency::task is a class template that can be associated with a function that returns the payload of the task. Now, instead of calling get on the task to retrieve its value, we can call the then method to set up the continuation, in which we process the value:

task<int> t([]()
{
    return do_work();
});
task<void> t2 = t.then([](int n)
{
    std::cout << n;
});

The task is a very simple object that contains a placeholder for the eventual value and a list of function objects — the continuations to be invoked when the task transitions to the completed state. The then method doesn't block — all it does is add a continuation to that list. When the task completes, all its continuations are scheduled on the thread pool.

The default thread pool can be substituted with a custom scheduler, which is passed as a parameter to the then function. This is useful in several scenarios — for example, an event loop-based scheduler will run the work on a GUI thread, allowing it to access the thread-bound UI controls. A priority scheduler can run the work on a high-priority thread. A locality-aware scheduler will run the work close to a particular CPU core or a NUMA node, and so on.

The return value of the then is again a task, which can be continued, allowing for serial composition of operations. Multiple continuations off the same task can execute concurrently, giving us a way to do parallel composition of operations. Several tasks can be composed via the join or the choice operator, both non-blocking operations that produce a task that can be continued, or composed with others, and so on. For example, the choice operator can be used to produce a task that completes when the first of the argument tasks completes:

task<int> t1([]()
{
    return do_work();
});

task<int> t2([]()
{
    return do_some_other_work();
});

(t1 || t2).then([](int n) {
    cout << "The first value to arrive is " << n;
});

The ideas of tasks and continuations in PPL are influenced by the Task Parallel Library, developed by Microsoft for the .NET framework, as well as the Promises library of CommonJS, and (of course) std::future in C++11.

At Microsoft, we've been quite successful with this model so far. Windows Runtime has adopted PPL tasks as the model of asynchrony in C++, and we've started a new project called Casablanca to build a modern C++ library for cloud-based client-server communication, in which all concurrent operations are exposed as PPL tasks. Today, PPL is a Microsoft library for targeting the Windows platform only, but we feel that the developer community at large can benefit from the ideas we developed in the PPL. That's why we've submitted a proposal to the C++ Standard's Committee to extend the std::future concept with continuations, along the lines of the continuations in the PPL tasks. The proposal is publicly available, and we're currently in the process of building an experimental reference implementation compatible with Visual Studio 2012. We're a small team in Microsoft and not shy about asking for help from the community. If you're willing to volunteer constructive feedback, or test-drive our implementation, I promise that your voice will be heard.


Artur Laksberg is a member of the Visual C++ team at Microsoft, working on concurrency libraries and tools. He can be reached at arturl@microsoft.com.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video