Channels ▼
RSS

Open Source

Multithreaded Asynchronous I/O & I/O Completion Ports

Source Code Accompanies This Article. Download It Now.


When developing server applications, it is important to consider scalability, which usually boils down to two issues. First, work must be distributed across threads or processes to take advantage of today's multiprocessor hosts. Second, I/O operations must be scheduled efficiently to maximize responsiveness and throughput. In this article, I examine I/O completion ports—an elegant innovation available on Windows that helps you accomplish both of these goals.

I/O completion ports provide a mechanism that facilitates efficient handling of multiple asynchronous I/O requests in a program. The basic steps for using them are:

  1. Create a new I/O completion port object.
  2. Associate one or more file descriptors with the port.
  3. Issue asynchronous read/write operations on the file descriptor(s).
  4. Retrieve completion notifications from the port and handle accordingly.

Multiple threads may monitor a single I/O completion port and retrieve completion events—the operating system effectively manages the thread pool, ensuring that the completion events are distributed efficiently across threads in the pool.

A new I/O completion port is created with the CreateIoCompletionPort API. The same function, when called in a slightly different way, is used to associate file descriptors with an existing completion port. The prototype for the function looks like this:


HANDLE CreateIoCompletionPort(
   HANDLE FileHandle,
   HANDLEExistingCompletionPort,
   ULONG_PTR  CompletionKey,
   DWORD NumberOfConcurrentThreads
   );

When creating a new port object, the caller simply passes INVALID_HANDLE_VALUE for the first parameter, NULL for the second and third parameters, and either zero or a positive number for the ConcurrentThreads parameter. The last parameter specifies the maximum number of threads Windows schedules to concurrently process I/O completion events. Passing zero tells the operating system to allow at least as many threads as processors, which is a reasonable default. For a discussion of why you might want to schedule more threads than available processors, see Programming Server-Side Applications for Windows 2000 by Jeffrey Richter and Jason D. Clark.

Associating File Descriptors with a Port

Once a port has been created, file descriptors opened with the FILE_FLAG_OVERLAPPED (or WSA_FLAG_OVERLAPPED for sockets) may be associated with the port via another call to the same function. To associate an open file descriptor (or socket) with an I/O completion port, the caller passes the descriptor as the first parameter, the handle of the existing completion port as the second parameter, and a value to be used as the "completion key" for the third parameter. The completion key value is passed back when removing completed I/O requests from the port. The fourth parameter is ignored when associating files to completion ports; a good idea is to set this to zero.

Initiating Asynchronous I/O Requests: OVERLAPPED Explained

Once a descriptor is associated with a port, (and you may associate many file descriptors with a single I/O Completion Port), an asynchronous I/O operation on any of the descriptor(s) results in a completion event being posted to the port by the operating system. The same Windows APIs that let callers perform standard synchronous I/O have a provision for issuing asynchronous I/O requests. This is accomplished by passing a valid OVERLAPPED pointer to one of the standard functions. For example, take a look at ReadFile:


BOOL ReadFile(
  HANDLE    File,
  LPVOID    pBuffer,
  DWORD     NumberOfBytesToRead,
  LPDWORD   pNumberOfBytesRead,
  LPOVERLAPPED  pOverlapped
  );

For typical (synchronous) I/O operations, you've always passed NULL for the last parameter, but when doing asynchronous I/O, you need to pass the address of an OVERLAPPED structure in order to specify certain parameters as well as to receive the results of the operation. Asynchronous calls to ReadFile are likely to return FALSE, but GetLastError returns ERROR_IO_PENDING, indicating to the caller that the operation is expected to complete in the future.

A common mistake when using OVERLAPPED structures is to pass the address of an OVERLAPPED structure declared on the stack:


OVERLAPPED Gone;
// Set up 'Gone'..
ReadFile ( hFile, pBuf, Count,
          &NumRead, &Gone );

This just won't work because ReadFile returns immediately, and when the function containing the call to ReadFile exits, the stack will be unwound and the data pointed to by &Gone will become invalid. Thus, you should ensure that your program manages its OVERLAPPED structures (and any buffers you're using) carefully. The example employs a fairly common strategy that involves having a C++ class representing a connection derive from OVERLAPPED—which may offend some C++ purists, but is a practical solution to the problem. The connections are allocated on the heap, and when I/O operations are initiated, the connections' pointer is passed as the pointer to OVERLAPPED.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video