Channels ▼
RSS

Parallel

Introduction to OpenMP




Environment Variables


  • OpenMP provides the following environment variables for controlling the execution of parallel code.

  • All environment variable names are uppercase. The values assigned to them are not case sensitive.

OMP_SCHEDULE

Applies only to DO, PARALLEL DO (Fortran) and for, parallel for (C/C++) directives which have their schedule clause set to RUNTIME. The value of this variable determines how iterations of the loop are scheduled on processors. For example:

setenv OMP_SCHEDULE "guided, 4"
setenv OMP_SCHEDULE "dynamic"

OMP_NUM_THREADS

Sets the maximum number of threads to use during execution. For example:

setenv OMP_NUM_THREADS 8

OMP_DYNAMIC

Enables or disables dynamic adjustment of the number of threads available for execution of parallel regions. Valid values are TRUE or FALSE. For example:

setenv OMP_DYNAMIC TRUE

OMP_NESTED

Enables or disables nested parallelism. Valid values are TRUE or FALSE. For example:

setenv OMP_NESTED TRUE

Implementation notes:

  • Your implementation may or may not support nested parallelism and/or dynamic threads. If nested parallelism is supported, it is often only nominal, in that a nested parallel region may only have one thread.

  • Consult your implementation's documentation for details - or experiment and find out for yourself if you can't find it in the documentation.

OMP_STACKSIZE

New with OpenMP 3.0. Controls the size of the stack for created (non-Master) threads. Examples:

setenv OMP_STACKSIZE 2000500B
setenv OMP_STACKSIZE "3000 k "
setenv OMP_STACKSIZE 10M
setenv OMP_STACKSIZE " 10 M "
setenv OMP_STACKSIZE "20 m "
setenv OMP_STACKSIZE " 1G"
setenv OMP_STACKSIZE 20000

OMP_WAIT_POLICY

New with OpenMP 3.0. Provides a hint to an OpenMP implementation about the desired behavior of waiting threads. A compliant OpenMP implementation may or may not abide by the setting of the environment variable. Valid values are ACTIVE and PASSIVE. ACTIVE specifies that waiting threads should mostly be active, i.e., consume processor cycles, while waiting. PASSIVE specifies that waiting threads should mostly be passive, i.e., not consume processor cycles, while waiting. The details of the ACTIVE and PASSIVE behaviors are implementation defined. Examples:

setenv OMP_WAIT_POLICY ACTIVE
setenv OMP_WAIT_POLICY active
setenv OMP_WAIT_POLICY PASSIVE
setenv OMP_WAIT_POLICY passive

OMP_MAX_ACTIVE_LEVELS

New with OpenMP 3.0. Controls the maximum number of nested active parallel regions. The value of this environment variable must be a non-negative integer. The behavior of the program is implementation defined if the requested value of OMP_MAX_ACTIVE_LEVELS is greater than the maximum number of nested active parallel levels an implementation can support, or if the value is not a non-negative integer. Example:

setenv OMP_MAX_ACTIVE_LEVELS 2

OMP_THREAD_LIMIT

New with OpenMP 3.0. Sets the number of OpenMP threads to use for the whole OpenMP program. The value of this environment variable must be a positive integer. The behavior of the program is implementation defined if the requested value of OMP_THREAD_LIMIT is greater than the number of threads an implementation can support, or if the value is not a positive integer. Example:

setenv OMP_THREAD_LIMIT 8



Memory and Performance Issues

Thread Stack Size:

  • The OpenMP standard does not specify how much stack space a thread should have. Consequently, implementations will differ in the default thread stack size.

  • Default thread stack size can be easy to exhaust. It can also be non-portable between compilers For example, the table below shows some approximate thread stack size limits at LC using the default version compilers (Jan '09).

    Compiler Approx. Stack Limit Approx. Array Size (doubles)
    IBM POWER5 AIX:
    xlc, xlf
    8 MB 707 x 707
    AMD Opteron Linux
    icc, ifort
    8 MB 707 x 707
    AMD Opteron Linux
    pgcc, pgf90
    16 MB 1024 x 1024
    AMD Opteron Linux
    gcc, gfortran
    4 MB 512 x 512
    AMD Opteron Linux
    pathcc
    pathf90
    67 MB
    >1.4GB
    2048 x 2048
    >9500 x 9500

  • Threads that exceed their stack allocation may or may not seg fault. An application may continue to run while data is being corrupted.

  • Statically linked codes may be subject to further stack restrictions.

  • A user's login shell may also restrict stack size

  • Examples for increasing the thread stack size to 12 MB at LC:

    Platform Shell Commands
    csh
    Shell Commands
    ksh
    IBM POWER5 AIX limit stacksize 12288
    setenv XLSMPOPTS "stack=12000000"
    ulimit -s 12288
    export XLSMPOPTS="stack=12000000"
    AMD Opteron Linux limit stacksize 12288
    setenv KMP_STACKSIZE 12000000
    ulimit -s 12288
    export KMP_STACKSIZE=12000000



LLNL Specific Information and Recommendations

LC OpenMP Implementations:

  • OpenMP is fully supported all of LC's IBM and Opteron systems.

  • LC maintains different versions of compilers. For the most recent information, please see: www.llnl.gov/asci/platforms/bluepac/CompsAvails.html

  • All versions should be at least OpenMP 2.0 compliant. None are OpenMP 3.0 compliant as of Jan, 2009.

Compiling:

  • Use the following compiler flags to "turn on" OpenMP compilations:

    Compiler Flag
    IBM -qsmp=omp
    Intel -openmp
    PathScale -mp
    PGI -mp
    GNU -fopenmp

Documentation:




References and More Information

  • Author: Blaise Barney, Livermore Computing.

  • The OpenMP web site, which includes the C/C++ and Fortran Application Program Interface documents.
    www.openmp.org


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video