Parallel Correctness: Caveat Emptor!
While task-based programming models provide a wonderful abstraction for parallelism, ensuring that parallel code is correct remains an exercise left to the developer. For example, consider the following code that initializes each element of an array to be one greater than the previous element:
for (int i = 1; i < a.Length; i++)
a[i] = a[i-1] + 1;
In the above case, if element a[0] was initialized to the value 0, the last element of the array will hold the value arraylength -1 after the loop finishes. Note the loop-carried dependency in this code, which means the value of a given iteration of the loop depends on a value created by a previous iteration. Now let's parallelize this loop:
Parallel.For(1, a.Length, (i) => {
a[i] = a[i-1] + 1;});
In this case we're likely to get a different (and wrong) answer for the value of the last element of the array. Why? Parallel.For() causes the work of this loop to be split into several chunks that will be executed in simultaneously-running iterations. Only the iteration starting at element 1 will have the benefit of the initialized value of a[0]. All of the other simultaneous iterations will execute using whatever value a[i-1] happened to hold prior to Parallel.For loop.
It's the developer's responsibility to ensure that they've carefully reviewed code for these kinds of potential parallel pitfalls.
Additional Classes and Data Structures
Both TPL and PPL provide a set of addition helper classes and data structures to make writing parallel applications simpler. These range from cooperative locking and synchronization constructs to parallel versions of common containers and algorithms to classes for lazy variable initialization.
Concurrency Runtime
An underlying runtime is responsible for efficiently executing program tasks based on available system resources. For managed code, this runtime logic is built into the .NET ThreadPool, whereas native code uses a runtime that is included as a part of the Visual C++ runtime. Both runtimes attempt to efficiently execute tasks by maximizing utilization of available cores using concepts such as hill climbing (adding threads to the pool as the workload increases) and work-stealing (taking work from neighboring worker threads when the current worker's queue runs dry. The native code Concurrency Runtime supports additional programmer tweaking, such as the ability to write custom schedulers and to use multiple schedulers in the same application.
Parallel Debugging
The Visual Studio 2010 raises the concept of tasks to be first-class citizens in the debugging experience. The debugger now includes two views accessible from the Debug|Windows menu: Parallel Tasks and Parallel Stacks. These views work equally well for native and managed code and are accessible while program execution is paused, such as after a breakpoint has been hit.
The Parallel Tasks view provides insight into the currently scheduled and running tasks. These could be explicitly created tasks or tasks created under the hood by calling a function such as parallel_invoke() or Parallel.ForEach(). Figure 2 shows the Parallel Tasks view in action.
Using this view, developers can gain insight into details about a task, such as the current thread it is running on (threads are indeed used in the runtime to execute tasks) and the current location of execution in code. This view also provides the execution disposition of the task, such as whether it is running, scheduled but not running, or even deadlocked with another task.
The Parallel Stacks view in Figure 3 gives an all-up view of program execution flow.
Rather than use the Visual Studio Call Stacks view to pick through call stacks one thread at a time as in previous versions, this view provides representation of the call stacks for all of the parallel contexts stitched together and displayed as a visual graph. The Parallel Stacks view can view call stacks by thread or by task, allowing you to view execution flow in a way that makes most sense for your program. This view can also be used to set the active thread in the debugger and navigate back to source code.
