Networking CD-ROMs Using Cached CD Images

When it comes to storage space, the more the merrier

John H. McCoy

John is a member of the computer science staff at Sam Houston State University in Huntsville, Texas. He can be contacted at [email protected].

In our article entitled "Network Access to CD-ROMs," (DDJ, August 1993), Wuhsiung Lu and I described a client/server system for networking CD-ROMs in a DOS/Windows environment. That system worked fine, except when 25 or 30 users simultaneously accessed the same drive. Caching the CD-ROM to the server's hard disk eliminated that problem. Making the cache permanent even eliminated the need for a dedicated drive for the CD-ROM.

The space needed for this permanent cache depends on the CD-ROM volume size and how much it can be compressed. The system I used for adding caching to the server program, for instance, used a 508-MB hard disk. By running PC-DOS 7 with Stacker, I was able to cache both a 635-MB tax-reference CD and the 27-MB PC-DOS 7 installation CD with room to spare. In short, storing cache images can be cheaper than installing more CD drives.

On the client side, MSCDEX allocates buffer space and maps a drive letter to every drive supported by the driver, even if you aren't going to use them all. As the number of CDs increased, this became unmanageable, leading me to write a MSCDEX substitute called SHSUCDX that allows selective drive mapping. The latest versions of the client/server programs are available electronically at ftp://ftp.shsu.edu/pub/cdrom/shsucd12.zip. In this article, I'll describe a cache driver that reads the cached CD images and makes them appear as CD drives. The images can be located either on the client machine or a network file server if the client accesses the network via a real-mode redirector rather than a shell. (I've used the system with the Windows NT DOS client and NetWare VLMs, but it won't work with the NetWare shell NETX.) For these networks, using the cache driver allows you to network CDs without having to use client/server software.

There are DOS programs (the NCJRS database, for example) that bypass Microsoft's CD-ROM extensions and access the driver directly. These programs work fine with the cache driver. Most programs, however, use the extensions and require the use of SHSUCDX, the MSCDEX substitute I'm including with this article. Note that if you attempt to use MSCDEX, your system will crash. MSCDEX trashes itself on a superfluous Qualify File Name request that DOS broadcasts on the INT 2Fh chain when the cache driver attempts to read the cache file.

How the Driver Works

When it installs, MSCDEX maps the CDs to drive letters, sets the attribute bits to indicate that the drive is a network drive (and another one that says it really isn't), and hooks the INT 2Fh multiplexer interrupt as if it were a network redirector. Figure 1, which assumes that MSCDEX has been loaded after the network redirector, illustrates this process.

DOS applications normally access a CD by making a DOS call (INT 21h). DOS, in turn, recognizes from the attributes that the CD is a network drive and makes an appropriate function 11h call to INT 2Fh. MSCDEX checks to see if the call involves one of its drives; if it does, it handles the request by making a call to the CD driver. If the call does not involve one of MSCDEX's drives, MCSDEX passes it down the INT 2Fh chain.

DOS applications can also call MSCDEX directly with function 15h calls to INT 2Fh. These calls bypass the DOS file system to provide access services that are unique to CDs. Some programs use DOS and MSCDEX to locate the CD driver, then make subsequent calls directly to the driver, bypassing both DOS and MSCDEX.

Adding the Cache Driver

When a cache image is used instead of a physical CD, the driver reads sectors from the cache image file; see Figure 2. If the application has made an INT 2Fh call or called the driver directly, then DOS has been bypassed and the driver is free to make an INT 21h call to read the image file. However, if the image file is a network file, DOS will have to make an INT 2Fh call to the network redirector. SHSUCDX sees the call and passes it on to the redirector.

MSCDEX cannot be used even if the image file is on a local disk. When called by the cache driver, DOS sends what appear to be superfluous broadcasts down the INT 2Fh chain, even if a network file is not involved. MSCDEX will crash when it sees a second call on the INT 2Fh chain while processing a prior call-regardless of whether the call is intended for MSCDEX.

Most DOS applications call DOS to access the CD instead of making an INT 2Fh call or calling the driver directly. In this case, when the cache driver needs to read the cache file, it can't make an INT 21h call because it is already in DOS. An undocumented INT 2Fh call is used instead.

Undocumented DOS Calls

Only three INT 2Fh, function 12h calls-open, read, and close-are used. None of these are officially documented. Ralf Brown's interrupt listings indicate that function 12h calls can only be made while in DOS and that they are known to be used by DOS to access COUNTRY.SYS. Andrew Schulman's Undocumented DOS, second edition (Addison-Wesley, 1994) and Geoff Chappel's DOS Internals (Addison-Wesley, 1994) provide some help, along with a lot of caveats. Without this help I wouldn't have known where to start.

Before making a function 12h call, the DOS swappable data area (SDA) must be saved, a switch to the DOS internal stack must be made, and the appropriate registers must be loaded. An INT 21h, function 5D06h call, also undocumented, is needed to locate the DOS stack, get the limits of the SDA, and locate the IN DOS flag. The current PSP and DTA don't seem to be significant in INT 2Fh, function 12h calls. It is not necessary to change them. In particular, you can't open a file during driver install and then use the handle from the driver's PSP to access that file in a subsequent INT 2Fh call. The simple approach I employ here is to test the IN DOS flag, then make an open call followed by a read and a close call using either INT 21h or INT 2Fh, as appropriate, for every image-file access.

Program Details

There are three programs involved. CDCACHER, which is available electronically in both source and executable form, builds the cache file. It is a straightforward process to read the CD blocks and write them to a file. Only single-session, ISO 9660 or High Sierra format CDs are supported. The first 16 blocks on the CD comprise the system area, followed by the primary volume descriptor (PVD) in block 16. I elected to ignore the system area by dropping block zero entirely and zeroing out the remaining blocks. The "cooked" data (2048 bytes) from block 1 through the end of the volume space is then copied to its corresponding record in a direct-access file; for example, block 16 becomes record 16 in the image file.

CDCACHER assumes that the driver name for the CD to be cached is "MSCD001." This can be overridden by including the driver name on the command line or by entering a new name in the dialog box after starting the program.

Caching a full CD takes time-at eight to ten MBytes/min, it takes over an hour. It doesn't run any faster in an OS/2 DOS window (unless you have a faster system), but you can get on with other things while it's copying. CDCACHER won't run in an NT DOS session. I've been told that there is no CD device-driver interface.

SHSUCDHD (also available electronically) is the cache driver. When executed, it installs as SHSU-CDH, using up to five cache images specified on the command line. The filenames are stored in canonical form before discarding the initialization code and TSRing. SHSUCDHD/U unloads the driver.

The resident code follows the standard driver form: A header that specifies the device type and name, a strategy, and an interrupt section. The interrupt section is basically an if statement that handles the I/O commands that Microsoft requires, flagging the others as unknown.

The CD Read Long command actually reads the cache image file. The IN DOS flag is checked first. If IN DOS is set, the DOS SDA is saved before switching to the DOS stack and opening the file with an INT 2Fh, 1226h call. The filename supplied to DOS is already in canonical form but, as mentioned previously, DOS will broadcast a Qualify File Name request on the INT 2Fh chain anyway, before opening the file. An INT 2Fh, 1229h call reads the required block(s) into the data buffer specified in the request header passed to the driver when it was called. Another INT 2Fh call closes the cache file before the driver switches back to its own stack and restores the SDA.

If the IN DOS flag is not set, the SDA does not have to be saved/restored, the driver does not need to switch to the DOS stack, and INT 21h (rather than INT 2Fh) calls are used to open, read, and close the cache file.

The read procedure is also used by the IOCTL ReturnVolumeSize command to read part of the primary volume descriptor into a temporary buffer where the volume size can be extracted.

SHSUCDX (the executable is available electronically) is a replacement for MSCDEX that must be used with SHSUCDHD. It maps one or all of the drive images to drive letters (the last drive must be set in config.sys to provide enough drive letters). If the maximum of five image files were specified when SHSUCDHD was loaded, these will appear as drive units 0 through 4. All five units can be mapped to drive letters beginning with the first available letter by entering SHSUCDX /D:SHSU-CDH. Entering SHSUCDX /D:SHSU-CDH,F,1,2 will map two units: unit 1 to drive F, and unit 2 to drive G. Other combinations and multiple drivers can be specified to map up to ten drives. SHSUCDX /U unloads it.

Recompiling from the Source Code

SHSUCDHD is written entirely in assembler and compiled and linked with MASM 6.0b. CDCACHER, on the other hand, is written in C and compiled with Quick C 2.0. It uses Mix Software's C/Windows ToolChest for the user interface. You will need the C/ToolChest (or you can write your own interface) to recompile and link it.

The source code for SHSUCDX is not included here, but it is available from the URL given earlier. It is written partly in C and partly in assembler. From the SHSUCD11.ZIP file you'll need SHSUCDXASM, CDROM.INC, UNDOC.INC, CMDS.C, CDROM.H, and REDIR.H. Again, the Quick C 2.0 and MASM 6.0b compilers were used for compiling and linking.

Figure 1: How MSCDEX normally works.

Figure 2: Reading sectors from the cache image file.

Listing One

/* CDCACHER 4.0 -- CDCACHER makes a hard disk cache image from a CD. The CD ROM
 *   Server program, SHSUSERV, can then read from this cache or the pseudo 
 *   CD ROM driver program, SHSUCDH, can make it appear as a CD drive. The 
 *   cache image can be located on a local HD or on a network file server.
 *   A DOS program.  CDCACHER is a complete re-write in C (was ADA) and uses 
 *   the C/Windows Toolchest from MIX Software. It reads and writes multiple
 *   CD-ROM blocks which makes it significantly faster than the previous 
 *   versions. To re-compile you must have the Toolchest.
 *   A copyright-reserved, free use program.  Use at your own risk.
 *   (c)John H. McCoy, 1995 Sam Houston St. Univ., TX 77341-2206
 */

#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys\types.h>
#include <sys\stat.h>
#include <dos.h>
#include <string.h>
#include <malloc.h>
#include <signal.h>

#include "winlib.h"

#pragma pack(1)

#define cdRdLong 0x80
#define canonicalize 0x60
#define PriVolDescSector 16
#define ISO_ID "CD001"
#define HSF_ID "CDROM"
#define CD_ISO 'I'
#define CD_HSF 'H'
#define CD_Unknown 'U'

#define OCTETS(from,to) (to - from +1)

typedef unsigned char BYTE;
typedef unsigned int  WORD;
typedef unsigned long DWORD;

struct ISO_CD {
  BYTE     Fill      [OCTETS(1,1)];
  char     cdID      [OCTETS(2,6)];
  BYTE     Fill2     [OCTETS(7,40)];
  char     volLabel  [OCTETS(41,72)];
  BYTE     Fill3     [OCTETS(73,80)];
  DWORD    volSize;
  BYTE     Fill4     [OCTETS(89,2048)];
} far *iso;
struct HSF_CD {
  BYTE     Fill      [OCTETS(1,9)];
  char     cdID      [OCTETS(10,14)];
  BYTE     Fill2     [OCTETS(15,48)];
  char     volLabel  [OCTETS(49,80)];
  BYTE     Fill3     [OCTETS(81,88)];
  DWORD    volSize;
  BYTE     Fill4     [OCTETS(97,2048)];
} far * hsf;

struct RH{
  BYTE   hdrLength;
  BYTE   drvUnit;
  BYTE   drvCmd;
  WORD   drvStatus;
  BYTE   x[9];
  BYTE   far *pdta;
  WORD   sectors;
  DWORD  startSector;
  BYTE   mode;
  };

int GetValidDriver(void);
int GetDriverAddress(char * cdDriver);
void CanonCacheName(char *CacheName);
int GetCDLabel(void);
int GetCacheName(void);
void CacheCD(BYTE Unit);
void CallDev(struct RH far *);
int  CDReadLong(BYTE Unit, WORD SectorCount, DWORD StartSector);
int  ReadPVD(BYTE Unit);
int  PauseContinue(void);
void SetSigFlag(void);
void (far *DevStrategy)();
void (far *DevInterrupt)();

char   *copyrite = " CDCACHE v4.0, (c)1995, John H. McCoy, [email protected] ";
char   CD = CD_Unknown;
char   *cdDriver = "MSCD001 ";
char   far dta[30*2048];
char   far *pdta=dta;
char   CacheName[64]={".\\CD.IMG"};
BYTE   LastUnit;
int    SigFlag = 0;

int main (int argc, char **argv){
    Window    wd;
    int       col=5, row=1, width=68, height=1;
    union REGS      regs;
    struct SREGS    sregs;
    BYTE            Unit;
    int             rc;
    int             handle;

    width= strlen( copyrite);
    col = (80-width)/2;
    wd = w_open(col, row, width, height);
    w_print(wd,copyrite);

    if (argc > 1){
      if ((((argv[1][0] == '-') ||(argv[1][0] == '/'))&& (argv[1][1] == '?'))
         || (argv[1][0] == '?')){
            printf ("CACHCECD [DriverName] \n");
            goto done;
      }
      else cdDriver = argv[1];
    }
    iso = (struct ISO_CD *) pdta;
    hsf = (struct HSF_CD *) pdta;
    do {
       if (GetValidDriver() == -1){
          printf("No valid CD Driver.  Aborted.\n");
          goto done;
          }
       rc = GetCDLabel();
       }
    while (rc < 0);
    Unit = rc;
    if (GetCacheName() == -1){
          printf("User selected Quit.\n");
          goto done;
          }
    CanonCacheName(CacheName);
    handle = open(CacheName, O_BINARY);
    if (handle != -1){
       printf("%s already exists.\n",CacheName);
       printf("Delete file and re-run or re-run with different cache name.\n");
       close(handle);
       }
    else
       CacheCD(Unit);
 done:
    w_close(wd);
}
void SetSigFlag(void){
  ++SigFlag;
}
int ValidateCacheName(Field fd, int lastkey){
    Window    wd;
    int       col=5, row=4, width=68, height=3;
    char tempCache[64];

   if (lastkey == _ESC) return FD_OK;
   f_getstring(fd,tempCache);
   CanonCacheName(tempCache);

   if (open(tempCache, O_BINARY|O_RDONLY, S_IWRITE)!=-1){
     close(tempCache);
     wd = w_open(col, row, width, height);
     w_umessage(wd,"Cache File Already Exists");
     w_lmessage(wd,"Press a key to Continue");
     w_print(wd,tempCache);
     k_getkey();
     w_close(wd);
     return FD_ERR;
     }
   else
     return FD_OK;
}
int GetCacheName(){
    Window    wd;
    Field     fd;
    char      *fdData
          ="________________________________________________________________";
    int       col=5, row=4, width=68, height=3;
    int       field_col=1, field_row=1;
    int       xkey;

    wd = w_open(col, row, width, height);
    w_umessage(wd,"Cache File Name");
    w_lmessage(wd,"ESC to Quit");
    fd = f_create("", fdData);
    f_startclear(fd,ENABLE);
    f_return(fd,ENABLE);
    f_setbuffer(fd,CacheName);
    f_validate(fd,ValidateCacheName,DISABLE);
    xkey = f_process(wd, field_col, field_row, fd);
    f_getstring(fd,CacheName);
    f_free(fd);
    w_close(wd);
    if (xkey == _ESC) return -1;
    else return 0;
}
void CanonCacheName(char *CacheName){
   union REGS      regs;
   struct SREGS    sregs;
   char *pCacheName;

   pCacheName = CacheName;
   regs.h.ah = canonicalize;
   regs.x.di = FP_OFF(pCacheName);
   regs.x.si = FP_OFF(pCacheName);
   sregs.es = FP_SEG(pCacheName);
   sregs.ds = FP_SEG(pCacheName);
   int86x (0x21,®s,®s,&sregs);
}
void CacheCD(BYTE Unit){
   DWORD     i,n, volSize;
   int       handle;
   union REGS      regs;
   struct SREGS    sregs;
   Window    wd;
   int       col=5, row=4, width=68, height=1;

   (void)ReadPVD(Unit);
   if(CD==CD_ISO)
      volSize = iso->volSize;
   else if (CD==CD_HSF)
      volSize = hsf->volSize;
   else{
      printf("Unkown CD Format.  Can't cache it.\n");
      return;
   }
   for (i=0;i<sizeof dta;i++){dta[i]=0;}

   wd = w_open(col, row, width, height);
   w_umessage(wd,"Creating Cache File ");
   w_lmessage(wd,"Control Break to interrupt");
   handle = open(CacheName, O_BINARY|O_CREAT|O_WRONLY,S_IWRITE);
   n = 15;
   (void)write(handle,pdta, n*2048);
   SigFlag = 0;
   signal(SIGINT, SetSigFlag);
   i = n+1;
   while (i < volSize){
     n = volSize - i;
   w_printf(wd,"Writing block %ld of %ld to %s.\n",i,volSize,CacheName);
     if (n>=30){
        (void)CDReadLong(Unit,30,i);
        (void)write(handle,pdta, 61440);
        i += 30;
        }
     else {
        (void)CDReadLong(Unit,n,i);
        (void)write(handle,pdta, n*2048);
        i += n;
        }
     if (SigFlag != 0){
       if (PauseContinue() == -1) break;
       else{
          SigFlag=0;
          signal(SIGINT, SetSigFlag);
       }
     }
   }
   w_close(wd);
   close (handle);
}
int PauseContinue(void){
   int       xkey;
   Window    wd;
   int       col=15, row=10, width=48, height=1;
   wd = w_open(col, row, width, height);
   w_umessage(wd,"Control Break ");
   w_print(wd,"  Press ESC to Quit, any other key to continue.");
   xkey=k_getkey();
   w_close(wd);
   if (xkey==_ESC)
      return(-1);
   else
      return (0);
}
void CallDev(struct RH far * ptr){
    _asm  push     bx
    _asm  push     es
    _asm  mov      bx, [bp+8]
    _asm  mov      es, bx
    _asm  mov      bx, [bp+6]
    _asm  call     DevStrategy
    _asm  call     DevInterrupt
    _asm  pop      es
    _asm  pop      bx
}
int CDReadLong(BYTE Unit, WORD SectorCount, DWORD StartSector){
   int i;
   struct RH rh;
   rh.hdrLength = sizeof (rh);
   for (i=0;i<9;i++){rh.x[i]=0;}
   rh.drvStatus = 0;
   rh.mode = 0;
   rh.drvCmd = cdRdLong;
   rh.pdta = pdta;
   rh.sectors = SectorCount;
   rh.startSector = StartSector;
   rh.drvUnit = Unit;
/*   printf("Reading unit %d \n",Unit);*/
   CallDev(&rh);
}
int ReadPVD(BYTE Unit){
   (void)CDReadLong(Unit,1,PriVolDescSector);
   if(strncmp(iso->cdID,ISO_ID,sizeof(iso->cdID))==0)
      CD=CD_ISO;
   else if (strncmp(hsf->cdID,HSF_ID,sizeof(hsf->cdID))==0)
      CD=CD_HSF;
   else
      CD=CD_Unknown;
}
int GetCDLabel(void){
    Window    wd;
    Field     fd;
    char      cdLabel[15];
    int       col=22, row=4, width=21, height=15;
    int       i,xkey;
    Control   lb;
    BYTE      Unit;
    strncpy(cdLabel,"    ",4);
    wd = w_open(col, row, width, height);
    w_umessage(wd,cdDriver);
    w_lmessage(wd,"ESC for New Driver");
    lb = c_add_listbox(wd," Drive  Label ",2,1,17,13,LISTBOX_SORT);
    for (Unit=0; Unit<=LastUnit; Unit++){
      (void)ReadPVD(Unit);
      if (CD=='I')
         strncpy(&cdLabel[4],iso->volLabel,11);
      else if (CD=='H')
         strncpy(&cdLabel[4],hsf->volLabel,11);
      else
         strncpy(&cdLabel[4],"Unknown Fmt",11);
      i=Unit;
      i/=10;
      cdLabel[1] = '0'+i;
      i=Unit;
      i%=10;
      cdLabel[2] = '0'+i;
      c_add_item(lb,cdLabel,-1,ENABLE);
   }
    xkey=c_dialog(wd);
    if (xkey == _ESC){
       w_close(wd);
       return -1;
    }
    else{
       Unit = c_read(lb,C_STATE);
       w_close(wd);
       return Unit;
    }
}
int ValidateDriverName(Field fd, int lastkey){
   char tempDriver[12];
   if (lastkey == _ESC) return FD_OK;
   f_getstring(fd,tempDriver);
   if (GetDriverAddress(tempDriver)==-1)
     return FD_ERR;
   else
     return FD_OK;
}
int GetValidDriver(){
    Window    wd;
    Field     fd;
    char      *devName="________";
    int       col=5, row=4, width=15, height=4;
    int       field_col=3, field_row=1;
    int       xkey; /* don't declare lastkey if using f_validate  */

    wd = w_open(col, row, width, height);
    w_umessage(wd,"CD DRIVER");
    w_lmessage(wd,"ESC to Quit");
    fd = f_create("", devName);
    f_validate(fd,ValidateDriverName,DISABLE);
    f_startclear(fd,ENABLE);
    f_return(fd,ENABLE);
    f_setbuffer(fd,cdDriver);
    xkey = f_process(wd, field_col, field_row, fd);
    f_getstring(fd,cdDriver);
    f_free(fd);
    w_close(wd);
    if (xkey == _ESC) return -1;
    else return 0;
}
int GetDriverAddress(char *cdDriver){
    int   handle;
    union REGS      regs;
    struct SREGS    sregs;
    struct DTA {
       char         cmd;
       char   far * pdevHdr;
     } dta;
    struct DTA far *pdta;

   handle = open( cdDriver, O_BINARY);
   if (handle==-1) return(-1);
   dta.cmd = 0;
   pdta = &dta;
   regs.h.ah = 0x44;
   regs.h.al = 0x02;
   regs.x.bx = handle;
   regs.x.cx = 5;
   sregs.ds = FP_SEG(pdta);
   regs.x.dx = FP_OFF(pdta);
   int86x (0x21,®s,®s,&sregs);
   close(handle);
   if (regs.x.cflag) return(-1);

   FP_SEG(DevStrategy) = FP_SEG(dta.pdevHdr);
   FP_OFF(DevStrategy) = *(int far *)(dta.pdevHdr+6);
   FP_SEG(DevInterrupt) = FP_SEG(dta.pdevHdr);
   FP_OFF(DevInterrupt) = *(int far *)(dta.pdevHdr+8);
   LastUnit = *(dta.pdevHdr+21) - 1;
   return (0);
}