Channels ▼

Top 10 Challenges in Parallel Computing

In the spirit of year-end, top 10 lists, here is one for parallel computing. A version appeared earlier this year in a paper with Tim Mattson, who's talked about this in other forums and gets full credit for its collation; it reflects thinking by some of the folks here, and is worth posting for discussion. Note the list is for parallel computing, so encompasses hardware, tools, and design considerations as well as programming. In priority order:

  • Finding concurrency in a program -- how to help programmers "think parallel"?
  • Scheduling tasks at the right granularity onto the processors of a parallel machine.
  • The data locality problem: associating data with tasks and doing it in a way that our target audience will be able to use correctly.
  • Scalability support in hardware: bandwidth and latencies to memory plus interconnects between processing elements.
  • Scalability support in software: libraries, scalable algorithms, and adaptive runtimes to map high level software onto platform details.
  • Synchronization constructs (and protocols) that enable programmers write programs free from deadlock and race conditions.
  • Tools, APIs and methodologies to support the debugging process.
  • Error recovery and support for fault tolerance.
  • Support for good software engineering practices: composability, incremental parallelism, and code reuse.
  • Support for portable performance. What are the right models (or abstractions) so programmers can write code once and expect it to execute well on the important parallel platforms?

Reactions, please! Some things to consider:

  • We make no mention of power consumption -- is that reasonable? How would the power challenge be phrased, and where would it slot into the list?
  • Data locality (#3) is intimately tied to scalability (#4). As systems become heterogeneous, with an assortment of different latencies among elements, how can we reconcile this tuning to portability (#10)?
  • Are we underestimating the importance of composability, so innocently tucked into "good software engineering" (#9)? And by the way, how would you teach that concept to university students, and in which course?

Lots more -- let's pick it up in 2009. Happy New Year to all!

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.