Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Debugging and Optimizing Multithreaded OpenMP Programs

In this article we discuss OpenMP technology whose main value is the possibility of improving and optimizing the already created code. OpenMP standard is a set of specifications for paralleling code in an environment with shared memory. OpenMP can be used if this standard is supported by the compiler. Besides, you need absolutely new tools for the debugging stage when errors are detected, localized, and corrected and optimization is performed.

A debugger for sequential code is a tool which a programmer knows very well and uses very often. It allows you to trace changes of variables' values when debugging a program in single-stepping mode with the help of advanced user interface. But with debugging and testing of multithreaded applications everything is different, in particular creation of multithread applications becomes the main area in creating effective applications.

Debugging of a sequential program is based on that the level of predictability of initial and current program's states is determined by the input data. When a programmer debugs multithreaded code he usually faces absolutely unique problems: in different operation systems different planning strategies are used, the computer system's load changes dynamically, processes' priorities can be different etc. Precise reconstruction of the program's state at a certain moment of its execution (a trivial thing for sequential debugging) is very difficult in case of a parallel program because of its undetermined behavior. In other words, behavior of processes launched in the system, in particular their execution and waiting for execution, deadlocks and the like, depends on random events occurring in the system. What to do? It is obvious that we need absolutely different means for debugging parallel code.

As parallel computer systems have become a common thing on consumer market, demand for means of debugging multithreaded applications has increased greatly. We will consider debugging and increase of performance of a multithreaded application based on OpenMP technology. Listing Five presentes the.

Let's consider sequential code of Function function in Listing One as an example. This simple subprogram calculates values of some mathematical function with one argument.

double Function(int N)
  double x, y, s=0;
  for (int i=1; i<=N; i++) {
    x = (double)i/N;
    y = x;
    for (int j=1; j<=N; j++) {
      s += j * y;
      y = y * x;
  return s;

Listing One

When calling this function with N argument equal 15000, we'll get 287305025.528.

This function can be easily paralleled with the help of OpenMP standard's means. Let's add only one string before the first operator for (Listing Two).

double FunctionOpenMP(int N)
  double x, y, s=0;
  #pragma omp parallel for num_threads(2)
  for (int i=1; i<=N; i++) {
    x = (double)i/N;
    y = x;
    for (int j=1; j<=N; j++) {
      s += j * y;
      y = y * x;
  return s;

Listing Two

Unfortunately, the code we've created is incorrect and the result of the function is in general undefined. For example, it can be 298441282.231. Let's consider possible causes.

The main cause of errors in parallel programs is incorrect work with shared resources, i.e. resources common for all launched processes, and in particular -- with shared variables.

This program is successfully compiled in Microsoft Visual Studio 2005 environment and the compiler even doesn't show any warning messages. However it is incorrect. To understand this you should recall that variables in OpenMP programs are divided into shared, existing as single copies and available for all the threads, and private, localized in a concrete process. Besides, there is a rule saying that by default all the variables in parallel regions of OpenMP are shared save for parallel loops' indexes and variables defined inside these parallel regions.

From the example above it is obvious that x, y, and s variables are shared what it absolutely incorrect. s variable should be shared as it is the adder in the given algorithm. But when working with x or y each process calculates their next value and writes it into the corresponding variable (x or y). And in this case the result depends on the sequence of executing the parallel threads. In other words, if the first thread calculates x value, writes it into x variable and after the same operations are performed by the second thread, tries to read the value of x variable it will get the value written into it by the last thread, i.e. the second one. Such errors, when program operation depends on the sequence of executing different code sections, are called race condition or data race ("race" condition or "race" of computing threads; it means that unsynchronized memory accesses occur).

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.