Channels ▼
RSS

JVM Languages

Improving Futures and Callbacks in C++ To Avoid Synching by Waiting


The C++11 standard provides several long-requested concurrency features such as the std::thread, std::future, and others. While those are a welcome addition to the language, in this article, I will show that they are not sufficient for all but the most basic concurrency needs. I will argue that the primitives in C++11 are particularly ill-suited for modern applications that must deal with the concurrency imposed by I/O operations and exploit multicore at the same time.

Fortunately, many of these limitations can be addressed by augmenting C++11 futures with continuations, based on experience with the Parallel Patterns Library (PPL) at Microsoft. The reader is expected to have a working knowledge of C++11 and some experience writing parallel code, but familiarity with the PPL is optional.

Connected and Multicore

We take it for granted that the software we use daily is both connected to the Internet and able to harness multiple cores. It is natural to think of the Internet (or the cloud) and multiple cores not as two distinct capabilities of a program, but as a single elastic compute resource. As Herb Sutter puts it in Welcome To The Parallel Jungle, "The network is just another bus to more compute cores."

However, a developer building a modern connected multicore application faces two distinct challenges:

  • Building a well-performing connected program requires dealing with the latency and unpredictability typical of I/O operations. This is difficult, and getting it right results in a an application that is responsive and scalable, but not necessarily faster.
  • Building a well-performing multicore program is a parallel programming job — a different challenge altogether, often requiring a different tool set and different skills. When this is done right, the program runs faster, although its speed is usually orthogonal to the responsiveness and the scalability.

Taking the view of the cloud as a natural extension of multicore behooves us to find a programming model that, at the very minimum, gives us a way to efficiently compose the I/O operations and the multicore operations.

Concurrency in C++

In the last decade, the software industry has developed a many tools for multicore programming in C++. Libraries such as Intel's TBB (Threading Building Blocks) or Microsoft's PPL (Parallel Patterns Library) are the state of the art. These tools excel at parallel decomposition — partitioning serial code into multiple "chores" that run on multiple cores.

But there is more to being "connected and multicore" than just parallelism. Well-performing concurrent programs must combine the connected components, with their inherent latency and unreliability, with the parallel components. Put another way, if parallelism is about decomposing the program into independent parts, concurrency is about both decomposing and composing the program from the parts that work well individually and together.

I believe that it's in the composition of connected and multicore components where today's C++ libraries are still lacking.

The Dreaded Wait

Most of the concurrency primitives in C++11 are composed via waiting. One can spawn a thread (by creating an instance of std::thread), then wait for it to finish by calling the join method. Likewise, the result of a future object (represented by std::future) can be retrieved by calling the get method — during which the calling thread waits for the result to become available.

Why is this a problem?

Waiting on the GUI thread means that the user of the application is rewarded with the "hourglass" or the "spinning donut" while the thread is waiting for an operation to complete. This is bad enough for a CPU-bound operation, but the length of an I/O-bound call can be truly unpredictable and therefore very long.

Clearly, the GUI thread of the application is a scarce resource, and we want to return it to the message pump as soon as possible — but let's not kid ourselves by thinking that all we need to do is offload the long-running operation to another thread. If we did that, how would we synchronize the two threads without waiting?

The woes of composition-by-waiting are not limited to GUI programs. By default, at creation, a thread reserves 1 MB of stack space on Windows and 8 MB on Linux. This value is configurable, but reducing the stack size may break programs with deep call chains or multiple stack-allocated objects. In other words, not only GUI threads are expensive — all threads are. This can be felt acutely in a multithreaded server application. If many threads decide to block at the same time, waiting can bring the server to its knees very quickly.

Continuations

In C/C++, continuations are commonly known as "callbacks," and they are often used for asynchronous programming. This is not to say that the concept is unique to C++, but because the language has been a laggard in adopting mainstream functional programming features such as the lambda expressions, C++ libraries that use continuations consistently are still rare.

The concept of the continuation was pioneered by Scheme, which introduced the style of programming where instead of returning a value, a function takes an additional parameter — the continuation that is invoked to process the return value of the function. Naturally, the continuation itself is also a function that can take continuations, and so on.

Continuations make the flow of control explicit — instead of invoking a function and waiting for it to complete, a program written in a continuation-passing style specifies explicitly what to do with the return value when it is available.

For concurrent programs, continuations are a boon because they allow us to avoid blocking waits — which, as I stated above, greatly hinder responsiveness and scalability.

JavaScript has made continuations ubiquitous in Web programming. Because JavaScript is single-threaded, waiting for the server to produce the data would freeze the browser. Instead, JavaScript uses a technique known as AJAX, where the act of issuing a request to the server is separated from the act of handling the data retrieved from the server:

http.open("GET", "customer.html");
http.onreadystatechange = function() { 
    if(http.readyState == 4) {
        var serverResponse = http.responseText; 
        // process serverResponse here: ... 
    }
}

More recently, Node.js has been very successful at capturing the mindshare of the developer community thanks to its use of continuations for server-side programming. In Windows Runtime, which powers the Metro-style apps in Windows 8, the concept of continuations is used holistically for all potentially long-running applications. Continuations are, in fact, the only way of working with asynchronous operations.

Tasks, Futures, and Promises

A beloved child has many names, as the saying goes. The concept of the "task" — also known as the future or the promise, depending on the language and library — represents a relatively straightforward idea.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video