Kris moved from Belgium to New Zealand in September 1995. Currently, he manages the service department of CMC, a Wellington-based Apple Centre. He can be reached at [email protected].
After switching back and forth between Macintosh and UNIX systems, it occurred to me that the filter-and-pipe principle--as implemented on UNIX, MS-DOS, and OS/2--could exist on the Macintosh too. Consequently, I implemented pipes for the Macintosh, analogous to UNIX pipes.
UNIX filters are relatively easy to create and anyone with a basic knowledge of C can write one. I wanted my Macintosh filters to be just as easy to create, so I developed a C program shell for a Macintosh filter. This shell makes it easy for anyone with a basic understanding of C to develop new filters.
My Macintosh pipes are visual and are created by arranging icons in a Finder window. Because a Macintosh pipe is an arrangement of icons, it is more persistent than a UNIX pipe--once created, you can use the pipe multiple times without retyping. My pipes are pure Mac applications: No extensions are made to the basic Macintosh OS, although you do need System 7.
The Basic Behavior
You use a filter by dropping a text file on the filter's icon. The filter reads the dropped file, presents a standard file save dialog, and writes the processed results to the output file. The resulting file is a text file that can be opened from any suitable application. The sample filters include converters between Macintosh and MS-DOS file formats and a sort filter that sorts the lines in a text file.
When you arrange two or more filter icons visually on a horizontal row in a folder, you are constructing a pipe. (Use the command key to make icons "stick" on grid positions while moving icons.) If you drop a document on the left-most icon of this pipe, data is sent from left to right through the consecutive filters. The data hops from icon to icon, until it reaches the right-most filter of the pipe, which saves the result in a file. After processing the dropped document, all filters in the pipe automatically quit.
You can change the behavior of some filters by changing their names. For example, if you rename the sort filter to "sort column 3-6", it will sort the incoming data on character columns 3 to 6 instead of sorting on complete lines. These application-name options are analogous to UNIX command-line options.
When you double click the left-most icon of a horizontal row of one or more filter icons, all filters in the pipe start up and remain idle. I call this a "server pipe." If one or more documents are dropped on the left-most filter icon of the active pipe, they are each processed as previously described. After processing, the complete pipe remains idle and does not quit. By choosing File-Quit in the menu of the left-most filter, all filters in the pipe will quit. Currently, there is no need for a server pipe, but they could be used to exchange data among multiple Macintoshes.
Developing Your Own Filters
The program shell handles all the details of dealing with the Macintosh operating system. You only need to develop a routine called ProcessChar(int c), which is called once for each character in the file. When there is no more data, ProcessChar is called an additional time with the pseudocharacter cEOF (equal to -1). Whenever it has data to send further down the pipe, it can call PutChar(int c) or PutStr(char *s). The easiest way to develop a new filter is to copy and modify the template file copy.c; see Listing One.
The ProcessChar function in simple filters accepts a character, modifies it, and immediately passes it along. More complex filters--like sort--receive and store the complete file before processing it and sending it further down the pipe.
Handling Options
When a filter starts, it parses its own file name. If you want to handle parameters in the application name, you need to expand a few functions to handle the details. The sort filter in Listing Two is a good example of this type of processing.
The AcceptAppNam(char *appNam) function is called with the first word of the filter's name. This allows a filter to change function depending on its name. For example, if the icon is called "Sort Descending Field 2-3," AcceptAppNam("sort") is called.
The ParamNeedsValue(char *param) function is called for each word that follows the program name. It returns True if that word is followed by an argument. In the previous example, ParamNeedsValue("descending") would return False and ParamNeedsValue("field") would return True. If False is returned, the engine passes the parameter to AcceptParameter(char *param), which can set appropriate global state variables. Otherwise, it grabs the next word and treats it as an argument to this parameter.
The argument parsing is geared to handling numeric ranges, such as "column 2-4,7". The engine parses the word following the parameter and calls AcceptParameterFromTo(char *param, char *from, char *to) for each range, and AcceptParameterValue(char *param, char *value) for each lone value. In this example, there would be calls to AcceptParameterFromTo("column","2","4") and AcceptParameterValue("column","7"). You can redefine these functions to control how parameters are handled. Use atoi() if you need to convert strings into integers.
You may need to use global variables in your filter module, either to store state information from the application name parameters, or to store the data you receive before processing it. For example, the sort filter uses global and dynamic data storage extensively.
Overview of the Engine
Each filter consists of two C source files: pipe.c contains the engine, and another file contains ProcessChar() and other filter-specific functions. Filter-specific files for sort and copy are shown in Listings One and Two, respectively. The remaining source code is available electronically; see "Availability," page 3.
At startup, the routine InitPipe is called. The filter scans through all files in the folder where it is located, looking for application icons that are at the same horizontal level and to its left. If it finds more than one, it selects the one that is closest. It then prepares to handle Apple Events.
If an application was found, the filter opens a PPC (Program-to-Program communication) port using the Macintosh PPC Toolbox. A PPC port is identified by two strings, the type and name. Here, the name is chosen to be a random string, and the type is the name of the application whose icon it found to its right. After opening the port, the application is ready to be launched.
The filter now tries to open a PPC connection established by a previous filter. The PPC connection's type is the same as this filter's name. This PPC call, like most PPC calls in pipe.c, is executed asynchronously so the filter will not stall if one of the calls is unsuccessful.
After InitPipe, GetParameters is called. The filter then parses its own file name, and calls the corresponding routines in the filter module.
After the necessary preparations, the filter enters its main event loop, where it regularly checks for incoming PPC connections (from an application to its right) or for Apple Events (from documents dropped on the filter).
Internally, there are data structures that implement two state machines: one for the incoming connection (gInConnection) and one for the outgoing connection (gOutConnection). Depending on the events that occur, these machines cycle through a set of states allowing them to process incoming and outgoing data.
If a file is being processed, and there is not another filter application to the right, the filter brings up a standard file-save dialog. Before doing so, the filter checks to see if it is running in the foreground or in the background. If it is running in the background, it uses the Notification Manager to notify the user and waits until it is put in the foreground.
Example Filters
I've included a variety of simple filters in the source (available electronically). The copy filter is primarily a template for developing new filters. The Toupper and Tolower filters convert all letters in a file to uppercase or lowercase.
The Mac2dos and Dos2mac filters convert text files between Macintosh and DOS format. They change the end-of-line characters and also convert most upper ASCII characters, including accented letters. Because many DOS files are not recognized as text files, I included the file type bina in the bundle resource for the filters in pipe.r, so even untyped DOS files should be accepted by the Dos2mac filter.
The Datascope filter does no processing, but simply shows the data that passes through it in a window. This allows you to peek at the data as it is being processed. The Unique filter removes consecutive duplicate lines from the input.
The Sort filter, which sorts its input by line, is the most complex of those I've described here. Sort supports several parameters. By adding the "descending" (or "desc" or "d") parameter, Sort will sort in descending order instead of the default ascending order. To sort on a specific range of characters, add a "column" option such as "column 3-10" to sort on character positions 3 to 10. If the input consists of tabbed text, you can use the "field" option. "Sort field 2-3" sorts on fields 2 and 3. Tabbed text files are created by most spreadsheet and database programs when you export to text format. Because Sort stores the complete file in memory while sorting, it needs a fairly large application size, unlike most of the other filters, which can live with 100K or less. Sort does not recognize comma-separated parameters, but it could easily be extended to do so.
Compiling the Examples
The example filters can be compiled with either Symantec C++ 7.0 or MPW. An MPW makefile pipe.make and an MPW script makeall are included for making the included example filters. MPW is handy for compiling a set of filters with a single command; with the Symantec IDE you have to compile each filter separately.
Because the console emulator of Symantec C++ suited my needs better than the corresponding MPW SIOW (Simple Input Output Window) library for showing data being processed, the Datascope filter must be compiled using Symantec C++ 7.0. It needs a resource file datascope..rsrc, which is just a compiled form of the resource source file pipe.r. If you modify pipe.r, you can use the pipe.make makefile to recreate datascope..rsrc. To remove some MPW-specific lines from the source, I added a line #define __THINKC__1 to the Edit-Options-Think C-Prefix settings in the project datascope..
While compiling with MPW, you will see a few warnings about parameters being unused. These warnings are harmless.
Future Plans
I plan to extend this filter package in several ways. One improvement will be to rewrite pipe.c to remove a few old Toolbox calls and compile PowerPC-native filters. I would also like to improve the handling of parameters. In particular, the 31-character limit on file names is too short for some applications. It would be nice to store parameters in a separate file and just specify the file name in the application name, such as "sort @settings" to instruct Sort to read the settings text file for more parameters.
I also have many ideas for new filters:
- Monitor will monitor a folder for dropped documents. Each document that is dropped in the folder will be sent through the pipe. This allows you to create server pipes, so you can just drop data in a folder to process it.
- Serial will send the data it receives to one of the serial ports, where you can connect a printer, a plotter, or another peripheral. This will allow you to create serial spoolers.
- Transmit and Receive will be a pair of filters that allow you to send data over the network from one Macintosh to another for further processing. A system could be set up where a single transmitter could send to multiple receivers, distributing a workload over several Macintoshes.
- Transmit TCP and Receive TCP. These will do the same as the aforementioned items, but to and from a UNIX machine. They will allow you to connect UNIX pipes to Macintosh pipes and vice versa.
Listing One
/* copy.c -- The generic version of processchar that should be used as
* a base for new pipes. Contains the routine ProcessChar(int chr).
* ProcessChar should call PutChar(chr) after or while processing incoming
* characters. At the end of the file, there is an extra call to ProcessChar
* with chr == cEOF to signal EOF. chr an unsigned char in the range 0 to 255
* when calling ProcessChar with a valid character.
* There is no harm done by calling PutChar(chr) with chr == cEOF.
*/
#ifndef _processchar_____LINEEND____
#include "processchar.h"
#endif
/****************************************************************************
* Empty filter. Use as a starter for new filters
*/
void ProcessChar(int chr)
{
if (chr != cEOF)
{
PutChar(chr);
}
}
/* ************************************************************************ */
/* This routine will be called at startup with the name of the application
* You can set some of your global variables to influence ProcessChar.
* All characters are converted to lowercase before the call.
*/
void AcceptAppNam(char *appNam)
{
}
/* ************************************************************************ */
/* This routine is called for each command-line parameter encountered. You
* should test the parameters and return TRUE if the parameter needs a value,
* FALSE if not. All characters are converted to lowercase before the call.
* Example: sort ascending column 3-5
* ParamNeedsValue("ascending") should return FALSE
* ParamNeedsValue("column") should return TRUE
* You can use if (strcmp(param,"name") == 0) for comparisons...
*/
int ParamNeedsValue(char *param)
{
return(FALSE);
}
/* ************************************************************************ */
/* Called for each value-less command-line parameter encountered.
* All characters are converted to lowercase before the call.
* Example: sort ascending column 3-5
* AcceptParameter("ascending") is called
* You can use if (strcmp(param,"name") == 0) for comparisons...
*/
void AcceptParameter(char *param)
{
}
/* ************************************************************************* */
/* Called for a single-valued command-line parameter. All characters are
* converted to lowercase before the call.
* Example: sort ascending field 3,4
* AcceptParameterValue("field","3") and AcceptParameterValue("field","4")
* are called.
* You can use if (strcmp(param,"name") == 0) for comparisons...
* Use atoi() for converting numeric strings into integers.
*/
void AcceptParameterValue(char *param,char *value)
{
AcceptParameterFromTo(param,value,value);
}
/* ************************************************************************* */
/* This routine is called for a two-valued command-line parameter.
* All characters are converted to lowercase before the call.
* Example: sort ascending field 3-4,5-7
* AcceptParameterFromTo("field","3","4") and
* AcceptParameterFromTo("field","5","7") are called
* You can use if (strcmp(param,"name") == 0) for comparisons...
* Use atoi() for converting numeric strings into integers.
*/
void AcceptParameterFromTo(char *param,char *valueFrom,char *valueTo)
{
}
Listing Two
/* sort.c -- Store input file in unbalanced binary tree. Sort criterium can
* either be complete lines, range of character columns, or range of tabbed
* text fields.
*/
#ifndef _processchar_____LINEEND____
#include "processchar.h"
#endif
/* *************************************************************************
* SORT filter
*/
typedef struct SLine
{
char *line;
int from;
int to;
struct SLine *pSmaller;
struct SLine *pGreater;
} TLine;
typedef TLine *TpLine;
/* Some globals */
TpLine gpTreeLine = NULL;
char gLine[cMaxStrLen+1];
char *gpLine = gLine;
int gLen = 0;
int gFromPos = 0;
int gToPos = 0;
int gFieldCount = 1;
/* Parameters */
int gAscending = TRUE;
int gCharFromPos = 1;
int gCharToPos = cMaxStrLen;
int gFromField = 1;
int gToField = cMaxStrLen;
void AddLine(char *line, int len, int from, int to)
{
TpLine pLine;
TpLine pPrvLine;
int direction;
pLine = gpTreeLine;
pPrvLine = NULL;
direction = 0;
while (pLine != NULL)
{
char *pStr1;
char *pStr2;
char c1, c2;
pPrvLine = pLine;
pStr1 = pLine->line + pLine->from - 1;
c1 = *(pLine->line + pLine->to);
*(pLine->line + pLine->to) = '\0';
pStr2 = line + from - 1;
c2 = *(line + to);
*(line + to) = '\0';
direction = strcmp(pStr1,pStr2);
*(pLine->line + pLine->to) = c1;
*(line + to) = c2;
if (direction > 0)
pLine = pLine->pSmaller;
else
pLine = pLine->pGreater;
}
pLine = (TpLine) NewPtr(sizeof(TLine));
if (pLine != NULL)
{
pLine->pSmaller = NULL;
pLine->pGreater = NULL;
pLine->from = from;
pLine->to = to;
pLine->line = (TpChr) NewPtr(len+1);
if (pLine->line != NULL)
strcpy(pLine->line,line);
else
{
DisposePtr((Ptr) pLine);
pLine = NULL;
}
}
if (pLine != NULL)
{
if (pPrvLine == NULL)
gpTreeLine = pLine;
else
{
if (direction > 0)
pPrvLine->pSmaller = pLine;
else
pPrvLine->pGreater = pLine;
}
}
}
void DumpTree(TpLine pLine)
{
if (pLine != NULL)
{
if (gAscending)
DumpTree(pLine->pSmaller);
else
DumpTree(pLine->pGreater);
PutStr(pLine->line);
PutChar('\n');
if (gAscending)
DumpTree(pLine->pGreater);
else
DumpTree(pLine->pSmaller);
DisposePtr((Ptr) pLine->line);
DisposePtr((Ptr) pLine);
}
}
void ProcessChar(int chr)
{
if (chr == '\n' || (chr == cEOF && gLen > 0))
{
*gpLine = '\0';
if (gFromPos == 0)
{
gToPos = 0;
gFromPos = 1;
}
AddLine(gLine,gLen,gFromPos,gToPos);
gpLine = gLine;
gLen = 0;
gFromPos = 0;
gToPos = 0;
gFieldCount = 1;
}
else if (chr != cEOF)
{
if (gLen < cMaxStrLen)
*(gpLine++) = chr;
gLen++;
if (gFromPos == 0)
{
if (gFromField <= gFieldCount && gCharFromPos <= gLen)
gFromPos = gLen;
}
if (gFieldCount <= gToField && gLen <= gCharToPos)
gToPos = gLen;
if (chr == '\t')
gFieldCount++;
}
if (chr == cEOF)
{
DumpTree(gpTreeLine);
gpTreeLine = NULL;
}
}
/* ************************************************************************* */
/* This routine will be called at startup with the name of the application
* You can set some of your global variables to influence ProcessChar.
* All characters are converted to lowercase before the call.
*/
void AcceptAppNam(TpChr appNam)
{
}
/* ************************************************************************* */
/* Called for each command line parameter encountered. You should
* test parameters and return TRUE if parameter needs a value, FALSE if not.
* All characters are converted to lowercase before the call.
* Example: sort ascending column 3-5
* ParamNeedsValue("ascending") should return FALSE
* ParamNeedsValue("column") should return TRUE
*/
int ParamNeedsValue(char *param)
{
return(strcmp(param,"column") == 0 || strcmp(param,"field") == 0
|| strcmp(param,"col") == 0 || strcmp(param,"fld") == 0
|| strcmp(param,"c") == 0 || strcmp(param,"f") == 0);
}
/* ************************************************************************* */
/* Called for each value-less command-line parameter encountered.
* All characters are converted to lowercase before the call.
* Example: sort ascending column 3-5
* AcceptParameter("ascending") is called
*/
void AcceptParameter(char *param)
{
if (strcmp(param,"ascending") == 0
|| strcmp(param,"asc") == 0
|| strcmp(param,"a") == 0)
gAscending = TRUE;
else if (strcmp(param,"descending") == 0
|| strcmp(param,"desc") == 0
|| strcmp(param,"d") == 0)
gAscending = FALSE;
}
/* ************************************************************************* */
/* This routine is called for a single valued command line parameter.
* All characters are converted to lowercase before the call.
* Example: sort ascending field 3
* AcceptParameterValue("field","3") is called
*/
void AcceptParameterValue(char *param,char *value)
{
AcceptParameterFromTo(param,value,value);
}
/* ************************************************************************* */
/* This routine is called for a two-valued command-line parameter.
* All characters are converted to lowercase before the call.
* Example: sort ascending field 3-4
* AcceptParameterFromTo("field","3","4") is called
*/
void AcceptParameterFromTo(char *param,char *valueFrom, char *valueTo)
{
if (strcmp(param,"field") == 0
|| strcmp(param,"fld") == 0
|| strcmp(param,"f") == 0)
{
gFromField = atoi(valueFrom);
gToField = atoi(valueTo);
}
else if (strcmp(param,"column") == 0
|| strcmp(param,"col") == 0
|| strcmp(param,"c") == 0)
{
gCharFromPos = atoi(valueFrom);
gCharToPos = atoi(valueTo);
}
}DOS SERIAL NETWORK
