Channels ▼

Mark Nelson

Dr. Dobb's Bloggers

C++11 Hash Containers and Debug Mode

November 29, 2011

The First Results

I ran my test program in Visual C++ Release mode, using all the standard settings for a console application. For purposes of comparison, I ran the same program using g++ 4.6.1 on the same computer, booted up under Linux. For the set of 1,000,000 tokens, the results are shown below:

TaskVC++ 10 Releaseg++ 4.6.1 -O3
Fill unordered_map<string>0.41s.11s
Fill unordered_map<string const *>0.39s0.14s
Destroy unordered_map<string>3.17s0.01s
Destroy unordered_map<string const *>3.24s0.004s
Fill map<string>0.83s.53s
Fill map<string const *>0.88s0.66s
Destroy map<string>.14s0.01s
Destroy map<string const *>.07s0.002s

There are a few interesting points to take away from these tests:

  • Microsoft's compiler is taking an exceptionally long time to destroy hashed containers — one order of magnitude greater than it took to create it, and two orders of magnitude greater than it takes g++ to do the same task.
  • It doesn't look like constructing and destroying the strings is a big factor. Both compilers have roughly the same performance with both std::string and std::string *. Microsoft's behavior is counterintuitive, as it takes longer to construct and destroy containers using the pointer.
  • The GNU compiler appears to be able to run through this exercise notably faster.

The time it takes to destroy the table is a concern — having a C++ program hang for over 3 seconds to destroy a modestly large data structure is a serious concern — particularly when the same task completes in a few milliseconds with g++.

The Pathological Results

These concerns are nothing compared to what I see when running in debug mode. Setting my Visual Studio project to debug mode, then running the same test, yields the results shown here:

TaskVC++ 10 Debug
Fill unordered_map<string>17.41s
Fill unordered_map<string const *>17.08s
Destroy unordered_map<string>505.36s
Destroy unordered_map<string const *>505.99s
Fill map<string>13.29s
Fill map<string const *>13.15s
Destroy map<string>0.94s
Destroy map<string const *>0.18s

Those numbers are hard to believe. Destroying a hash table takes one millisecond when using g++. In VC++ 10, it takes almost 10 minutes!

Worse, we suddenly see that hashed containers are slower than the containers built on red-black trees. Again, this just doesn't make sense.

The big problem with these numbers is that it means the debug mode of the compiler is effectively unusable for a lot of tasks. Regardless of how much testing it does, when it is this slow, it is just not useful.

A Workaround

I didn't invest the time to try debugging Microsoft's library, so I don't really know where the time is being spent. I did try a few workarounds to speed things up, and I found one technique that helps a lot. Before including any Microsoft header files, try entering this single line in your source:

#define ITERATOR_DEBUG_LEVEL 0

With this definition in place, the delete times return to ball park of the times seen when running in release mode. Of course, you give up some debugging. I believe that an explanation of what this macro does might be found here.

In the final analysis, I think Microsoft has some serious work to do here. The performance of its hashed containers, and to a lesser extent, the pre-C++11 associative containers, needs serious examination. If the library is going to run this much slower than the competition, I need a good explanation why.

Source

HashTest.cpp

Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 


Video