Channels ▼
RSS

Tools

OpenMP: A Portable Solution for Threading


Data-Race Conditions

Many data-race conditions could be due to output dependencies, in which multiple threads attempt to update the same memory location, or variable, after threading. In general, the OpenMP C++ and Fortran compilers do honor OpenMP pragmas or directives while encountering them during compilation phase, however, the compiler does not perform or ignores the detection of data-race conditions. Thus, a loop similar to the following example, in which multiple threads are updating the variable x will lead to undesirable results. In such a situation, the code needs to be modified via privatization or synchronized using mechanisms like mutexes. For example, you can simply add the private(x) clause to the parallel for pragma to eliminate the data-race condition on variable x for this loop.


// A data race condition exists for variable x;
// you can eliminate it by adding private(x) clause.

#pragma omp parallel for
for ( k = 0; k < 80; k++ )
{
    x = sin(k*2.0)*100 + 1;
    if ( x > 60 ) x = x % 60 + 1;
    printf ( "x %d = %d\n", k, x );
}

As discussed previously, data-race conditions can sometimes be difficult to spot; that is, more difficult than shown in this example. When using the full thread synchronization tools in the Windows API or in Pthreads, developers are more likely to avoid these issues because data is designed from the start to be managed with threading contention and race conditions in mind. However, when using OpenMP, it is easier to overlook data-race conditions. One tool that helps identify such situations is Intel Thread Checker, which is an add-on to Intel Vtune Performance Analyzer.

Managing Shared and Private Data

In writing multi-threaded programs, understanding which data is shared and which is private becomes extremely important, not only to performance, but also for program correctness. OpenMP makes this distinction apparent to the programmer through a set of clauses such as shared, private, and default, and it is something that you can set manually. With OpenMP, it is the developer's responsibility to indicate to the compiler which pieces of memory should be shared among the threads and which pieces should be kept private. When memory is identified as shared, all threads access the exact same memory location. When memory is identified as private, however, a separate copy of the variable is made for each thread to access in private. When the loop exits, these private copies become undefined.

By default, all the variables in a parallel region are shared, with three exceptions. First, in parallel for loops, the loop index is private. In the next example, the k variable is private. Second, variables that are local to the block of the parallel region are private. And third, any variables listed in the private, firstprivate, lastprivate, or reduction clauses are private. The privatization is done by making a distinct copy of each of these variables for each thread.

Each of the four clauses takes a list of variables, but their semantics are all different. The private clause says that each variable in the list should have a private copy made for each thread. This private copy is initialized with its default value, using its default constructor where appropriate. For example, the default value for variables of type int is 0. In OpenMP, memory can be declared as private in the following three ways.

  • Use the private, firstprivate, lastprivate, or reduction clause to specify variables that need to be private for each thread.
  • Use the threadprivate pragma to specify the global variables that need to be private for each thread.
  • Declare the variable inside the loop — really inside the OpenMP parallel region — without the static keyword. Because static variables are statically allocated in a designated memory area by the compiler and linker, they are not truly private like other variables declared within a function, which are allocated within the stack frame for the function.

The following loop fails to function correctly because the variable x is shared. It needs to be private. Given example below, it fails due to the loop-carried output dependence on the variable x. The x is shared among all threads based on OpenMP default shared rule, so there is a data-race condition on the x while one thread is reading x, another thread might be writing to it:


#pragma omp parallel for
    for ( k = 0; k < 100; k++ ) {
        x = array[k];
        array[k] = do_work(x);
    }

This problem can be fixed in either of the following two ways, which both declare the variable x as private memory.


// This works. The variable x is specified as private.

#pragma omp parallel for private(x)
for ( k = 0; k < 100; k++ )
{
    x = array[i];
    array[k] = do_work(x);
}

// This also works. The variable x is now private.

#pragma omp parallel for
for ( k = 0; k < 100; k++ )
{
    int x; // variables declared within a parallel
           // construct are, by definition, private
    x = array[k];
    array[k] = do_work(x);
}

Every time you use OpenMP to parallelize a loop, you should carefully examine all memory references, including the references made by called functions.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video