Wayne Pfeiffer is a Distinguished Scientist at San Diego Supercomputer Center where he does research in supercomputer performance analysis. He has been at the Center since helping found SDSC in 1985 and has held various senior management positions. He has a background in computational physics and a Ph.D. in engineering science from the California Institute of Technology.
Q: What is petascale computing, and why is it important?
A: Petascale computing refers to computers that can achieve a petaflop/s, that’s 10^15 floating-point operations per second, or one thousand teraflop/s. Some time in 2008 we will see one or more supercomputers with peak speeds of at least one petaflop/s, and by 2010 there should be a system that can sustain a petaflop/s on useful scientific and engineering applications.
Since the beginning of the computer age just over 60 years ago, the speed of the world's most powerful computers has increased exponentially, doubling about every year and a half in line with Moore's Law. That’s an increase of roughly a factor of 10 every five years, or 12 powers of 10 over the past 60 years!
With these dramatic advances in computer capability, computer simulations have transformed science and engineering, opening the way for discoveries in areas where analytic theory and traditional experiments can’t be used. So profound is this transformation that simulation is now widely considered the third branch of scientific discovery, complementing theory and experiment.
A next-generation supercomputer that can sustain one or more petaflop/s will let researchers cross new thresholds in solving some of the most challenging problems in science and engineering, such as designing more effective drugs, anticipating the impacts of climate change, and understanding the evolution of the universe.
Q: What are the main challenges in building and using petascale computers?
A: The biggest challenge is scalability, that is, getting an application to solve a problem efficiently on an enormous number of processor cores. Regardless of the processor technology, hundreds of thousands of cores will be needed to sustain a petaflop/s. We’ll need better interconnects, new parallel programming models, and major application development efforts to achieve good scaling in the petascale regime.
Power efficiency is also increasingly important, which is why processor clock speeds haven’t risen much in recent years. Because power grows faster than linearly with clock speed, it turns out that aggregating large numbers of relatively slow processors can significantly reduce the overall power requirement and operating cost for a given peak speed. This approach has been used with great success in IBM's Blue Gene architecture.
Another significant challenge is reliability. As computer size grows, the components need to be more reliable or have more redundancy and fault tolerance. In general, simpler processors with lower clock speeds tend to be more reliable.
Q: Software will be a key component in petascale computing. How do you see parallel programming evolving?
A: Parallel programming on high-performance computers has traditionally involved some variant of message passing, typically using MPI (Message Passing Interface). This has proved workable for the modest number of programmers involved, although it’s far from ideal.
The adoption of multi-core processors will profoundly change parallel programming. Multi-core processors are becoming widespread and no longer limited just to high-performance computers. Because many more programmers will have to deal with multi-core parallelism, there will be tremendous pressure to develop a programming model that’s easier to use than message passing, perhaps based on transactional memory.
Once a multi-core programming model becomes standard, it will initially be used within the shared memory of a single processor, or maybe several processors. Outside that shared memory unit, a more traditional message-passing model will apply. But because of the added complexity, the resulting hybrid programming model isn’t likely to remain attractive over the long run. So we’ll need more research and development on ways to present the programmer with a single parallel model. Hopefully this will lead to a natural extension of the multi-core model to entire computers made up of very many cores.
Q: When will we see an exascale computer?
A: Well, as Neils Bohr supposedly said, 'Prediction is very difficult, especially about the future.'
An exaflop/s is 10^18 flop/s, which is a thousand times faster than a petaflop/s. If computer speed kept advancing at its historical rate, then it would take about 15 years to go from a petaflop/s to an exaflop/s. But I think it may turn out to take quite a bit longer.
Moore's Law has held up remarkably well over many decades, and may apply for several more years. After that, microprocessor feature sizes will become so small that the technological advances needed to achieve further increases in microprocessor speed will likely be subject to the economic law of diminishing returns. This will mean the end of Moore's Law, or at least a significant lengthening of the doubling time.
Q: Tell us about your background.
A: My formal training was in math, physics, and engineering science. For 18 years I did fission and fusion energy research and development using the most powerful computers available. Then I helped found SDSC 22 years ago, and I’ve been here ever since. I have long been fascinated by what limits computer performance, and much of my work over the years has been directed toward improving our understanding of these limits and how to move beyond them.
Q: What do you enjoy doing outside of work?
A: As many of my colleagues know, I make it a point to run every day no matter where I am. I also enjoy many other outdoor activities, including biking, swimming, skiing, and hiking. Indoors I like to read, play the piano, and listen to classical music.