The design model for the RIO extensions uses pre-registered data buffers and completion queues to increase performance. The pre-registered data buffers are locked into memory and stay locked until the data buffers are released by the app. The data buffers can be used over and over again. It is assumed that a server app would maintain a pool of these pre-registered data buffers sized appropriately. RIO uses completion queues (RIO_CQ structures) and request queues (RIO_RQ structures) that are managed by the application. The completion queues are used to indicate when an I/O operation has been completed. A RIO request queue represents a RIO socket descriptor with associated completion queues, configuration data, and a Windows socket.
Using RIO
All RIO functions are extensions to Winsock. An application must call the WSAIoctl function with a special GUID to retrieve a set of pointers to the RIO functions at runtime.
The RIO API is small and divided into two types of functions:
- Buffer, queue, and notification operations: Used for registering and deregistering buffers, creating, resizing, and closing completion queues, creating, resizing, and closing request queues, and notification.
- Send and receive operations: Used to send or receive network data on a connected RIO TCP socket or a bound RIO UDP socket.
For an application using RIO, the normal Winsock functions are used for connect, accept, and listen operations. This does restrict an application using RIO from sending or receiving data as part of a connect or accept operation. Sending or receiving data on a connect or accept call is uncommon, so this limitation doesn't usually pose a hardship. RIO extension functions are used for send and receive operations.
An application creates a standard Windows socket using the socket, WSASocket, accept, AcceptEx, or WSAAccept functions. The RIOCreateRequestQueue function creates a RIO socket descriptor associated with the standard Windows socket. The RIOCreateRequestQueue function also assigns completion queues for send and receive operations to the RIO socket descriptor and sets other configuration data on the RIO socket descriptor.
Two options are available for completion queue notification:
- Event handles
- I/O completion ports
RIO does not provide any access synchronization to the completion queues and request queues. If multiple threads are needed by an app to access the same queue, then the developer must implement any needed synchronization or locking.
Fast Loopback
Another enhancement for performance was also made to Winsock on Windows 8/Windows Server 2012. A new SIO_LOOPBACK_FAST_PATH IOCTL was added to lower the latency and improve performance of TCP over loopback. On Windows, packets using the IPv4 or IPv6 loopback addresses use the network layer. This IOCTL changes the behavior so that TCP packets over loopback use the transport layer instead. This shortens the code path a loopback packet must traverse in the OS, resulting in lower latency and higher performance. This IOCTL can be used with the RIO extensions when using loopback.
To use this enhancement, both sockets using loopback (the client and server) must set this IOCTL. The socket that plans to issue the connect must set this IOCTL before making the connect call. The listening socket must apply this IOCTL before accepting an incoming socket. A limitation of this enhancement is that it can't be used with Windows Filtering Platform (WFP) filters. Another minor limitation is that only a limited number of socket options can be used when this IOCTL is set.
Socket Connected
We've taken a brief look at two new socket programming APIs added with Windows 8. Upcoming articles will dig into details on how to develop apps with these new socket APIs and improve your connected performance.
Further Information
These links provide additional information and explanation.
Windows Runtime Sockets: Connecting with sockets
Winsock RIO Extensions and fast loopback: What's New for Windows Sockets
SIO_LOOPBACK_FAST_PATH control code
Steven Baker works for Microsoft as a senior programming writer on networking. Previously, he worked for many years at the Oregon Department of Energy. Earlier, he wrote articles for several print computer magazines including the networking column for UNIX Review.
