Top 10 Challenges in Parallel Computing

In the spirit of year-end, top 10 lists, here is one for parallel computing. A version appeared earlier this year in a paper with Tim Mattson, who's talked about this in other forums and gets full credit for its collation; it reflects thinking by some of the folks here, and is worth posting for discussion. Note the list is for parallel computing, so encompasses hardware, tools, and design considerations as well as programming. In priority order:

• Finding concurrency in a program -- how to help programmers "think parallel"?
• Scheduling tasks at the right granularity onto the processors of a parallel machine.
• The data locality problem: associating data with tasks and doing it in a way that our target audience will be able to use correctly.
• Scalability support in hardware: bandwidth and latencies to memory plus interconnects between processing elements.
• Scalability support in software: libraries, scalable algorithms, and adaptive runtimes to map high level software onto platform details.
• Synchronization constructs (and protocols) that enable programmers write programs free from deadlock and race conditions.
• Tools, APIs and methodologies to support the debugging process.
• Error recovery and support for fault tolerance.
• Support for good software engineering practices: composability, incremental parallelism, and code reuse.
• Support for portable performance. What are the right models (or abstractions) so programmers can write code once and expect it to execute well on the important parallel platforms?

Reactions, please! Some things to consider:

• We make no mention of power consumption -- is that reasonable? How would the power challenge be phrased, and where would it slot into the list?
• Data locality (#3) is intimately tied to scalability (#4). As systems become heterogeneous, with an assortment of different latencies among elements, how can we reconcile this tuning to portability (#10)?
• Are we underestimating the importance of composability, so innocently tucked into "good software engineering" (#9)? And by the way, how would you teach that concept to university students, and in which course?

Lots more -- let's pick it up in 2009. Happy New Year to all!