A Few Minutes with James Reinders
James Reinders, author of Intel Threading Building Blocks: Outfitting C++ for Multi-core Processors, recently took a few minutes to talk about going parallel.
Q: James, are software developers ready for parallelization, or are we lacking the fundamental building blocks -- parallel algorithms, compilers, tools, and the like -- needed to effectively build concurrent applications?
A: We are definitely ready but there is plenty of room for improvement. Parallelism is new to many people, but there is a lot more expertise than most people realize -- our challenges are as much about education as about invention. I think this compares favorably with prior challenges that probably were more invention than education.
In many ways, we are more ready for parallelism than other big changes the software industry has faced. Many software developers have been exploiting parallelism for decades on much more expensive machines. We've learned a lot, and now we need to apply it in new ways. Compare this with visual user interfaces, and networking, and I think the community of experts when X11 or Windows 3.1 hit us, or the Internet exploded, were actually smaller and the prior lessons less clear. The challenge with parallelism may seem larger, but it is important to note the base of knowledge may be much larger too.
We live in a dynamic industry, and it is not slowing down. We're as ready for parallelism as the challenges which preceded it -- and we are just as unprepared too. We'll learn. Buy me a beer next time we meet and I'll tell you what Prof. Norman R. Scott told me years ago about our industry and the pace of change.
Q: I'll hold you to that beer. Until then, how do you respond to this question from a reader: "I'm an experienced developer and I keep hearing about 'parallelization' and 'concurrency' but don't know much about it. Where do I start?"
A: Well, I'm the sort of developer that needs to jump in and get my feet wet, then I listen to people tell me what I should have been thinking about.
As a C++ or C developer, I would jump in with Intel Threading Building Blocks (TBB). This is freely available at www.threadingbuildingblocks.org. If you use Java mostly, I'd still suggest you write code C++ code with TBB. If you are a Fortran programmer, or steadfastly C and anti-C++, then try OpenMP. Virtually all compilers for C, C++, and Fortran now support OpenMP.
The reason for starting with TBB or OpenMP for jumping in is simple: These are both abstractions which remove you from the hardware and focus you on parallelism. You are more likely to think through the parallel algorithm aspects of your problem, rather than the mechanics of implementing threads and dealing with mapping onto processors. That's why you should not jump in using pthreads, Windows threads, Boost threads, or Java threads.
Once you jump in, you'll want to learn about scaling, parallel programming errors (deadlock and race conditions), and how to write maintainable code.
Jump in first. With TBB, try a parallel_for, then a pipeline, and then a pipeline with some stages having a parallel_for. Try a numerical problem, or an image processing problem if you don't have something else you are eager to try.
Q: Threading Building Blocks? I know you've written an award-winning book on them, but what are they and why are they important?
A: Threading Building Blocks is a open source project we started which is the most popular abstraction for parallel programming in C++. It is most known for its algorithm templates, but contains many other smaller things a parallel programmer needs to write a portable parallel program in C++ -- such as concurrent data structures, memory allocation and portable locks. We believe in TBB so much, that we support it as a commercial product in addition to being an open source project. The code is the same, but we are willing to back it with Intel's name and support as needed by customers -- many people see that as a benefit.
Q: As a point of reference, what available tools -- open source and commercial -- have similar goals to Threading Building Blocks?
A: None that I'm aware of. TBB is unique in its portability, abstraction, adoption, and strength of implementation. Things like Boost threads address portability but not abstraction so programming is still too low level. Some community preview libraries from Microsoft are not products, limited to just algorithms, and are Windows-only. Nothing else addresses C++ like TBB, and that's why TBB is so popular with C++ developers.