Channels ▼


Managing Application Thread Use

The Load Governor

The load governor (Figure 2) uses a basic metric—processor idle time—to compute system utilization. Based on the idle time, the governor identifies available cores and returns the number of available cores to the requesting applications. For example, if the idle threshold is set to 5, the governor marks all cores that are idle 5 percent or more of their time as available (95 percent or less utilized) at the time of the snapshot. The processor statistics (system time, idle time, and so on) are collected from /proc/stats periodically (with Linux); this period can be adjusted as a parameter to the governor.

The goal of the governor's algorithm is not to prevent oversubscription, but rather to allow applications to finish as soon as possible. The reflection of the algorithm on the applications can be described as "slow start fast finish" with the exception of the first multithreaded application that contacts the governor, because the first application is guaranteed to start with as many threads as the number of cores. (The assumption here is that when the first application starts, the utilization of all the cores is less than the defined threshold.)

Figure 2: Block diagram showing the internals of the load governor.

The Thread Count Library

The second part in this framework is the library providing the API calls to communicate to the governor, and API calls to adjust the number of threads based on the number of available cores received as a response. The applications increment their thread count by the number of available cores returned, with the ceiling being the number of physically available cores.

The modifications to the original applications are minimal:

  1. Include two header files. One for the thread count and governor communication, and one for the timing calculations. The header file for the timing calculations is not necessary but is used during the simulation.
  2. Link with library.
  3. Modify the code so that just before entering a parallel region, thread count is updated.

Also, the applications were instrumented to run many iterations rather than just one. And before entering a parallel region, each application sets its thread count, which was updated by this sample OpenMP ( code segment:

init_thread_count ( );       // initialize thread count
while ( iter < ITERATION ) 
   set_thread_count();       // update thread count
   PARALLEL_REGION        //original benchmark
   #pragma omp parallel for
   for (...) 
      WORK ( );


To simulate multiple multithreaded applications running in parallel, I used benchmarks parallelized via OpenMP. I chose OpenMP because it is easy to prototype with and easy to implement the proposed framework. Some of the benchmarks were taken from the OpenMP source code repository (SCR) project ( and others were written from scratch. All the benchmarks are CPU-bound and scale well.

The simulation is performed in two different scenarios:

  • In the uniform run, two or more instances of the same benchmark were executed.
  • In the mix run, one or more instances of all benchmarks were run together simultaneously.

Again, the first benchmark that is started gets the maximum thread count, which is equal to the number of physically available processors.

The baseline measurement for both uniform and mix scenarios is taken when any running benchmark sets the number of threads as the number of processors available on the system, rather than changing their thread count during runtime (no load governor is running). In other words, if n different benchmarks or n instances of any benchmarks are executed simultaneously, there is a total of (n * the number of cores) threads.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.