For more information on the Concurrency Runtime Framework, see Concurrency Runtime: The Resource Manager.

Visual C++ 2010 comes with new features and enhancements to simplify more native programming. The Concurrency Runtime (CRT), for instance, is a framework that simplifies parallel programming and helps you write robust, scalable, and responsive parallel applications. The CRT raises the level of abstraction so that you do not have to manage the infrastructure details that are related to concurrency. The Concurrency Runtime also enables you to specify scheduling policies that meet the quality of service demands of your applications.

Figure 1 presents the architecture of Concurrency Runtime Framework.

[Click image to view at full size]

Figure 1

In this article, I discuss the Task Scheduler layer and examine how it works internally. To do so, I use CppDepend, an analysis tool that makes it easier for you to manage complex C\C++ (native, mixed, and COM) code base.

The Task Scheduler

The Task Scheduler schedules and coordinates tasks at runtime. A task is a unit of work that performs a specific job. The Task Scheduler manages the details that are related to efficiently scheduling tasks on computers that have multiple computing resources.

Windows provides a preemptive kernel-mode scheduler -- a round-robin, priority-based mechanism that gives every task exclusive access to a computing resource for a given time period, then switches to another task. Although this mechanism provides "fairness" (every thread makes forward progress), it comes at some cost of efficiency. For example, many compute-intensive algorithms do not require fairness. Instead, it is important that related tasks finish in the least overall time. Cooperative scheduling enables an application to more efficiently schedule work.

Cooperative scheduling is a mechanism that gives every task exclusive access to a computing resource until the task finishes or until the task yields its access to the resource.

The user-mode cooperative scheduler enables application code to make its own scheduling decisions. Because cooperative scheduling enables many scheduling decisions to be made by the application, it reduces much of the overhead that is associated with kernel-mode synchronization.

The Concurrency Runtime (CRT) uses cooperative scheduling together with the preemptive scheduler of the operating system to achieve maximum usage of processing resources. In this article, I examine the Task Scheduler design and lift its hood to see how it works internally. For information on the CRT Resource Manager, see Concurrency Runtime: The Resource Manager. Again, I use CppDepend to analyze the CRT source code.

Scheduler Design

The CRT provides the interface Scheduler to implement a specific scheduler adapted to application needs. Let's examine classes that implement this interface:

[Click image to view at full size]

Figure 2

The CRT provides two implementations of the scheduler -- ThreadScheduler and UMSThreadScheduler. As illustrated in the dependency graph in Figure 2, the SchedulerBase contains all common behavior of these two classes.

Is the Scheduler flexible? A good indicator of flexibility is to search for all abstract classes used by the Scheduler.

[Click image to view at full size]

Figure 3

As shown in the dependency graph in Figure 3, the Scheduler uses many abstract classes. It enforces low coupling, and makes the scheduler more flexible, so adapting it to other needs is easy. To explain the role of each abstract class used by the Scheduler, I'll discuss its responsibilities.

There are three major responsibilities assigned to the Task Scheduler:

Getting resources (processors, cores, memory). When the scheduler is created, it asks for resources from the runtime Resource Manager (as explained in CRT Concurrency Runtime: Resource Manager). The Scheduler communicate with Resource Manager using IResourceManager, ISchedulerProxy, and IScheduler interfaces. Resources given by Resource Manager use scheduler policy to allocate resources to the Scheduler.

The policy as shown in Figure 4 is assigned when the Scheduler is created.

[Click image to view at full size]

Figure 4

The CRT creates a default Scheduler if no Scheduler exists by invoking the GetDefaultScheduler method, and a default policy is used. The Task Scheduler enables applications to use one or more Scheduler instances to schedule work, and an application can invoke Scheduler::Create to add another Scheduler that uses a specific policy.

The Concurrency::PolicyElementKey enumeration defines the policy keys that are associated with the Task Scheduler.

[Click image to view at full size]

Figure 5

For more information on policy keys, see this article.

The following collaborations between the Scheduler and Resource Manager shows the role of each interface concerned by the allocation.

Ask for resource allocation:

[Click image to view at full size]

Figure 6

Getting resources from Resource Manager:

[Click image to view at full size]

Figure 7