|
Deciding how to handle blocking I/O is essential when designing programs that use the Winsock interface. Our authors present a method for removing the blocking network I/O function calls without having to rewrite the application from scratch.
June 01, 1996
URL:http://www.drdobbs.com/windows/removing-blocking-network-io-from-window/184409900
George, a senior software development engineer at Farallon Computing, can be reached at [email protected]. Derek is a software development engineer at Farallon and can be contacted at [email protected].
Deciding whether to use blocking or nonblocking network I/O is an essential step in the design of Windows programs that use BSD-style sockets, such as those provided by the Winsock interface. A socket function that takes an indeterminate amount of time to complete is said to "block."
Sockets were designed for UNIX, and if needed, the operating system can preempt a blocked task and begin running another program. But 16-bit Windows 3.x can't preempt a task, so calling a blocking function puts all other programs on hold until the call returns. In 16-bit Windows, it is better to use nonblocking socket function calls that return immediately. When these nonblocking functions complete, a notification message is posted to the application's message queue.
Unfortunately, in the rush to bring Internet applications to Windows, many UNIX programs were directly ported. Insufficient consideration was given to the inherent differences between Windows and UNIX. Blocking I/O crept into other programs as a result of poor design decisions. Besides causing performance degradation and locking out other applications, blocking I/O in 16-bit Windows can lead to serious reentrancy issues. Sprinkling PeekMessage loops throughout blocking code won't adequately fix most problems arising from the use of blocking calls. To keep messages flowing, modeless dialogs and keyboard accelerators require special handling in every place you insert PeekMessage loops. In addition, your application cannot handle the special processing that other applications require. Fixing the problems that arise from blocking network I/O is nontrivial.
Porting to Win32 is one solution. Since Windows NT and Windows 95 can preempt a suspended task, blocking I/O no longer locks out other applications. But only true 32-bit programs benefit from preemptive multitasking; 16-bit programs running in Windows 95 cannot be preempted. Also, to really take advantage of Win32's network enhancements, multiple threads should be used in all but the simplest programs. In a multithreaded application, if one of your program's threads is preempted, another can proceed. If you only recompile your 16-bit program for Win32 it will run in a single thread, and the entire application will be suspended when a network I/O function call blocks. Using multiple threads is great if your program only needs to run in Windows NT or Windows 95 (and you are willing to spend the time to redesign it for this much-different paradigm), but if your application also has to run in Windows 3.x, you have only one real choice--remove the blocking network I/O function calls. In this article, we'll present a method for doing this without rewriting the application from scratch.
Although our focus is on reading from and writing to TCP/IP sockets, the concepts apply to other transports that provide blocking and nonblocking versions of network I/O to a reliable stream. To use sockets in Windows, it is necessary to make calls to a driver that implements the TCP/IP protocols. The most commonly used API for 16-bit Windows is Winsock 1.1. The advantage of writing to the Winsock specification is that your program will be able to dynamically link with any WINSOCK.DLL provided by a TCP/IP vendor.
To communicate reliably between two processes, a connection-oriented stream provided by a TCP/IP socket is established between the processes. Then data is sent across the connection as a stream of bytes. The traditional BSD socket operations send() and recv() are used to send data to and receive data from a host. Whether or not calls to these functions block is determined by the mode in which the socket was created. Winsock extends the BSD standard to include a special set of nonblocking functions that begin with the prefix WSAAsync. By creating a nonblocking socket and using the WSAAsynchSelect() function call, an application will receive messages informing it when it can send or receive data.
Because most of the overhead of using Winsock has been abstracted in the Microsoft Foundation Classes, we've used the CAsyncSocket and CSocket classes in our example program. CSocket derives from CAsyncSocket and simply pumps messages until the operation completes or an error occurs. We've derived a TAsyncSocket class from CAsyncSocket and a TSyncSocket class from CSocket. All the code is written in C++.
The TAsyncSocket class abstracts the message-based nature of Winsock into an interface that notifies the caller when a particular I/O operation completes. To perform a nonblocking read, you call the TAsyncSocket::Read member function and tell it how many bytes to read, where to store the data, which function to call when the operation completes, and two reference constant (refcon) values. A refcon is a user-defined value that is passed back to the completion routine when the I/O operation finishes (so appropriate actions can be taken after the call completes). The completion function must have three parameters: the two refcon values and the result value for the I/O operation. The TAsyncSocket::Write operation functions in a similar manner.
As implemented, the TAsyncSocket class permits only one outstanding read and one outstanding write, but could easily be extended to allow multiple read/write requests.
Our sample application, Watch Me Draw, is designed to provide an example of how blocking I/O can be used in a client/server scenario. Once a connection is established, the side that initiated the connection becomes the "drawing" side. The other end of the connection is the "watching" side. The user on the drawing side can draw points and lines on the dialog. Those graphic objects are then transmitted to the watching side, which duplicates the drawing commands on its dialog. To keep the application as simple as possible, there is no code to handle repainting the dialog or to compensate for different dialog sizes in various video modes. In addition, error handling is minimal.
The protocol consists of three possible wire messages sent from the drawing side to the watching side: done, pixel, and line (see Figure 1). The done message signifies that the drawing side is closing the connection and, thus, the watching side should close its end. The pixel message tells the watching side that a PIXEL_DATA structure will follow. Finally, the line message tells the watching side that a LINE_DATA structure will follow.
The drawing side responds to the user's mouse button clicks and performs the appropriate drawing on its dialog box. Once a transmit structure is filled out, the appropriate wire message is sent over the connection, followed by the structure containing the data. Listing One presents three synchronous functions that transmit the information.
Once a connection is established, the watching side issues a synchronous read for a wire message. Once received, if it is a message for a pixel or line, another read is issued to fill in the appropriate structure. The pixel or line is drawn on the watching side's dialog box to duplicate the activity on the drawing side. Then, another read for a wire message is performed and the process begins again. The series of synchronous reads terminates when the done message is received. Listing Two contains the basic structure of the watching side's implementation.
Because nonblocking network I/O calls return immediately, code that depends on the results of a blocking call will not work in a nonblocking paradigm. In Example 1, Recv() is a blocking call. Since the if statement depends on a result obtained by the blocking call (namely, the value of i), making the Recv() call nonblocking will break the code.
If recursion and loops are ignored, we can think about the execution paths through functions as trees with the first statement executed at the root and the last statement executed at the leaf. Figure 2 illustrates the execution path tree of Example 1.
When analyzing a blocking program to convert, look for blocking calls that occur at nonleaf positions in the execution path of a function. In our example, the Recv() call is at the root of the tree, not at a leaf position. To convert blocking programs to nonblocking programs you must finesse your code to assure that all nonblocking calls appear as leaves in your execution path trees, and that the paths through your functions containing nonblocking calls contain no loops. You must eliminate loops in the old program that contained blocking calls.
After you've changed your socket initialization so that your network I/O operations won't block, you have to solve three main problems. The first is to guarantee that all nonblocking calls appear at leaf positions in any function's execution path. In Figure 3(a), the function func1 contains a single blocking call potentially preceded and followed by other groups of statements (represented as Processing Code Blocks 1 and 2). To convert func1, the blocking call becomes a nonblocking call that passes func2 (the callback function that will be called when the operation completes); see Figure 3(b). The code in Processing Code Block 2 has been moved to the body of func2. Thus, if Processing Block 2 depends on a result obtained by the nonblocking call, that result will be available after the call completes.
But how does func2 get access to local variables or parameters from func1? This is the second issue that needs to be resolved during the conversion. Figure 4(a) adds a few details to our generic func1: parameters x and y, and local variables ch and Arr. Since we are splitting up code that once was in a single function (and had access to the parameters and local variables of that function), we need to either pass the necessary variables to the nonblocking calls (so that they can be sent in as parameters to the callback function that calls the rest of the code) or provide another method for this data to be accessed. Passing a potentially lengthy list of arguments to the nonblocking functions makes the new nonblocking code unnecessarily complex and increases the chances you'll overlook something. A better solution is to define a local variable structure for each nonblocking function call containing parameters and local variables needed for code that will be executed by the nonblocking operation's callback function. Figure 4(b) shows the local variable struct for func1.
Figures 5(a) and 5(b) show how the local variable structure is used in the final nonblocking code. The last problem, removing loops that contain blocking calls in the original program, also is solved here. Again func1 is shown with a blocking call in a For loop. The nonblocking version of this code contains two new functions, func2 and func3. In the new func1, the local variable i has been encapsulated into a structure accessible to all three functions. In func1, it is simply initialized and func2 is called. In func2, the termination condition of the loops is tested and the initial code is executed (Processing Block 1) before the nonblocking call is made. When that call completes, func3 is called. This increments the index variable, executes the code that followed the original blocking call (Processing Block 2), and calls func2 again, simulating the loop in the original code.
We've used this general approach to remove the blocking calls from Watch Me Draw. Programs that implement the converted asynchronous transmit and receive portions of the application are available electronically; see "Availability," page 3.
Although removing blocking calls from shipping code may seem a daunting task at first, we've found that the investment more than pays for itself in increased reliability, robustness, and performance. Also, we've found that adhering to a method such as this is much better than simply diving into the code and haphazardly attempting to remove the blocking calls. As in all reengineering efforts, a small amount of planning will save headaches down the road.
char i;
Recv(s, &i, sizeof(char), NULL);
if (i != '0')
{
ErrorHandler();
}
else
{
ComputeResult(i);
}
done (char) pixel (char) x (int) y (int) color (COLORREF) line (char) x1 (int) y1 (int) x2 (int) y2 (int) color (COLORREF) nPenWidth (int)
(a)
void func1(void)
{
// Processing Code Block 1
BlockingCall();
// Processing Code Block 2
}
(b)
void func1(void)
{
// Processing Code Block 1
NonBlockingCall(, func2);
}
void func2(void)
{
// Processing Code Block 2
}
(a)
void func1(int x, float y)
{
char ch;
int Arr[10];
// Processing Block 1
BlockingCall(x, y, ch, Arr);
// Processing Block 2
}
(b)
typedef struct func1
{
int param_x;
float param_y;
char local_ch;
char local_Arr[10];
} FUNC1STRUCT;
(a)
void func1(void)
{
int i;
for (i = 0; i < 10; i++)
{
// Processing Block 1
BlockingCall();
// Processing Block 2
}
}
(b)
void func1(void)
{
func1struct.local_i = 0;
func2();
}
void func2(void)
{
if (func1struct.local_i < 10)
{
// Processing Block 1
NonBlockingCall(, func3);
}
}
void func3(void)
{
// Processing Block 2
func1struct.local_i++;
func2();
}
BOOL TMainDialog::TransmitDoneMessage()
{
HEADER_DATA toTransmit;
toTransmit.message = kchDoneMessage;
if(m_CommunicationSocket.Send(&toTransmit,
sizeof(toTransmit)) != sizeof(toTransmit))
{
return(FALSE);
}
return(TRUE);
} // TMainDialog::TransmitDoneMessage
BOOL TMainDialog::TransmitPixelMessage(LPPIXEL_DATA lpData)
{
HEADER_DATA toTransmit;
toTransmit.message = kchPixelMessage;
if(m_CommunicationSocket.Send(&toTransmit,
sizeof(toTransmit)) != sizeof(toTransmit))
{
return(FALSE);
}
if(m_CommunicationSocket.Send(lpData,
sizeof(PIXEL_DATA)) != sizeof(PIXEL_DATA))
{
return(FALSE);
}
return(TRUE);
} // TMainDialog::TransmitPixelMessage
BOOL TMainDialog::TransmitLineMessage(LPLINE_DATA lpData)
{
HEADER_DATA toTransmit;
toTransmit.message = kchLineMessage;
if(m_CommunicationSocket.Send(&toTransmit,
sizeof(toTransmit)) != sizeof(toTransmit))
{
return(FALSE);
}
if(m_CommunicationSocket.Send(lpData,
sizeof(LINE_DATA)) != sizeof(LINE_DATA))
{
return(FALSE);
}
return(TRUE);
} // TMainDialog::TransmitLineMessage
------------------------ receive code ------------------------
void TMainDialog::ReadSocketData()
{
HEADER_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,
sizeof(toRead)) != sizeof(toRead))
{
// ... handle error ...
return;
}
while(toRead.message != kchDoneMessage)
{
if(!ProcessMessage(toRead.message))
{
// ... handle error ...
return;
}
if(m_CommunicationSocket.Receive(&toRead,
sizeof(toRead)) != sizeof(toRead))
{
// ... handle error ...
return;
}
}
// ... normal close ...
} // TMainDialog::ReadSocketData
BOOL TMainDialog::ProcessMessage(char chMessage)
{
BOOL bConnectionOK = FALSE;
switch(chMessage)
{
case(kchPixelMessage):
bConnectionOK = ProcessPixelMessage();
break;
case(kchLineMessage):
bConnectionOK = ProcessLineMessage();
break;
}
return(bConnectionOK);
} // TMainDialog::ProcessMessage
BOOL TMainDialog::ProcessPixelMessage()
{
PIXEL_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,sizeof(toRead)) != sizeof(toRead))
{
return(FALSE);
}
// ... draw pixel ...
return(TRUE);
} // TMainDialog::ProcessPixelMessage
BOOL TMainDialog::ProcessLineMessage()
{
LINE_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,sizeof(toRead)) != sizeof(toRead))
{
return(FALSE);
// ... draw line ...
return(TRUE);
} // TMainDialog::ProcessLineMessage
void TMainDialog::ReadSocketData()
{
HEADER_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,sizeof(toRead)) != sizeof(toRead))
{
// ... handle error ...
return;
}
while(toRead.message != kchDoneMessage)
{
if(!ProcessMessage(toRead.message))
{
// ... handle error ...
return;
}
if(m_CommunicationSocket.Receive(&toRead,
sizeof(toRead)) != sizeof(toRead))
{
// ... handle error ...
return;
}
}
// ... normal close ...
} // TMainDialog::ReadSocketData
BOOL TMainDialog::ProcessMessage(char chMessage)
{
BOOL bConnectionOK = FALSE;
switch(chMessage)
{
case(kchPixelMessage):
bConnectionOK = ProcessPixelMessage();
break;
case(kchLineMessage):
bConnectionOK = ProcessLineMessage();
break;
}
return(bConnectionOK);
} // TMainDialog::ProcessMessage
BOOL TMainDialog::ProcessPixelMessage()
{
PIXEL_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,sizeof(toRead)) != sizeof(toRead))
{
return(FALSE);
}
// ... draw pixel ...
return(TRUE);
} // TMainDialog::ProcessPixelMessage
BOOL TMainDialog::ProcessLineMessage()
{
LINE_DATA toRead;
if(m_CommunicationSocket.Receive(&toRead,sizeof(toRead)) != sizeof(toRead))
{
return(FALSE);
}
// ... draw line ...
return(TRUE);
} // TMainDialog::ProcessLineMessage
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.