Distributing Work across Cores using .NET
The easiest way in C# using .NET 3.5 to distribute work across cores is to use the ThreadPool static class. I found however, it wasn't quite as effective as it could be so I rolled my own.
One of the goals of Heron is to make it easy to express parallel operations without having to worry about threads. I am doing this because I find multi-threaded code too darn hard. The irony is that to implement these operators I had to resort to rolling my own work distribution library using threads. Blech!
I originally was using ThreadPool, but I found that it did not do a great job of distributing work across the cores. So I rolled my own parallelization library in C#. You can find the source code online here: http://stackoverflow.com/questions/2215911/how-can-i-most-effectively-take-advantage-of-multiple-cores-for-short-computation.
Interestingly a number of people have recommended to me the Task Parallel Library which is part of .NET 4.0. Effectively it is a more powerful version of my little parallelization library. There are a few reasons that this was not a viable option for me: VS 2010 is still only in Beta and I wanted people to be able to run my code on VS 2008. I'm also a control freak, and I liked the idea of being able to manage work distribution myself.
So far the early benchmarks are showing that Heron does a good job of leveraging the second core of my machine when performing large "map" or "reduce" operations. I have a lot more tests and benchmarks to do before I feel comfortable making a new release of Heron, but things are looking promising! Maybe we can have a language sometime soon that actually does a good job of leveraging multiple cores.