Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

Database

Debugging Memory Allocation Errors


AUG90: DEBUGGING MEMORY ALLOCATION ERRORS

Replacing standard C functions and checking the status of the heap

This article contains the following executables: SPENCER.LST SPENCER.ALL

Larry is president of Cornerstone Systems Group, a firm that specializes in C and C++ consulting and training. Larry is also the secretary of the Connecticut chapter of the Independent Computer Consultants Assoc. He can be reached at 10 N. Main St., West Hartford, CT 06107; 203-236-9209.


It's 10 a.m., and you are working on your new real estate analysis program, HIGHRISE, that's scheduled to be finished tomorrow. The high-powered analysts will soon be sitting in their high-backed, leather chairs, clicking on your high-rise icon, eager to find out which property will drive profits still higher. However, you're feeling pretty low. HIGHRISE works great, but once in a while it locks up the computer. Usually, this happens after about a half hour of operation. Also, what about the time HIGHRISE forecasted a profit of $17,546,321.97 on a $1,000 investment? You never could reproduce the problem, so it was probably a fluke, right? "Please, let it have been a hardware error!"

Of course, if you know enough C to be interested in this article, you know that there is a 99 percent chance that the problem is related to memory allocation. Maybe you freed a pointer that you never allocated. Much like trying to walk down stairs that have not been built yet. Maybe you allocated memory but never freed it. A slower, agonizing death by asphyxiation.

You are probably thinking, "If I know enough C to be interested in this article, I just don't make that kind of mistake." Don't be so sure. About half the commercial database and screen-handling libraries I have worked with fail to free all the memory they allocate, even after their shutdown functions have been called! If the pros can make this mistake, so can we all.

The only way to find HIGHRISE's problem is to identify each memory allocation, make sure there is a corresponding free(), then find each free() and make sure there is an allocation for it. But HIGHRISE has now grown to half a meg of source. Even with a sophisticated debugger, you face a formidable task.

I'll show you an easier way. In the first part of this article, I'll present some functions that can replace the standard functions malloc(), free(), and so on. Each replacement function tracks your program's activity and reports it in a debugging file. If all is well, it calls the standard library function to do the real work. Using these replacement functions, I've tracked down some very cunning bugs, and you will, too. In the second part, I'll show how to obtain the heap-status of the heap without any special programming. A few well-placed calls to the functions may help you home in on the section of code that is giving you trouble. This technique described in the second part turned up the errors in those commercial libraries mentioned earlier.

Replacement Functions

Look at the program in Listing One (page 178). This program is a disaster. One block of memory is allocated but never freed. If this happens enough times in a program, it runs slower and slower, until it finally runs out of memory and hangs. Another block in Listing One is freed without ever having been allocated. In DOS, that kind of mistake can even clobber your file allocation table. (I did once!) Either type of error can be almost impossible to track down in a large, complex program.

Now look at Listing Two (page 178). I have replaced the calls to the standard functions with calls to the replacement functions that I will now describe.

First, notice the calls to memMalloc(). memMalloc() works just like malloc(), but there is a second argument. This argument is a "tag" that you can use to identify the call in the debugging file. This is the pattern for all the replacement functions: You use them just like the standard functions, but they take a tag as an extra argument.

The calls to memFree() replace calls to free(). Again, a tag is added to the argument list. Listing Three (page 178) shows the replacement routines. We'll begin with memMalloc(). This function begins by calling malloc(). memTrack_alloc() is then used to track the allocation.

memFree() calls memTrack_free(), which tracks the attempt to free, and returns TRUE if the memory may in fact be freed or FALSE if not. Only after determining that the memory may be freed does memFree() free it. The memTrack routines are obviously the key to the whole scheme, so let's discuss them next.

memTrack_alloc() and memTrack_free() keep track of pointers allocated and freed. The implementation shown here keeps a journal of activity in a file, but you could easily write an implementation that keeps records in an array or a binary tree. Although the file method is slower, it has the advantage of surviving the crash that you are trying to debug!

As you can see in memTrack_alloc(), the file consists of records of three fields each. The first field is either an A (for allocated) or F (for freed). The second is the address of the allocated item. The third is the tag you supplied in your call to the routine.

memTrack_alloc() simply appends a record of the allocation that it is tracking. Part of this record is the allocated address, in %p (pointer) format. This formats the address as 0000:0000. The %p format is not available with some compilers, so you may want to use a long integer, or whatever is appropriate on your machine.

memTrack_free() searches through the file, looking for confirmation that the memory you are about to free was, in fact, allocated. If it finds an "allocated" record with the same memory address, it over-writes the A (allocated) with an F (freed). If it does not find such a record, it writes an error message to the file.

These routines call the function memTrack_fp(), which returns a FILE pointer for the debugging file. The name of this file must be in the environment variable MEMTRACK. In DOS, this is accomplished with something like: SET MEMTRACK=C:\MYPROG.MEM

You may want to use a RAM drive if your program is at least giving you a normal exit. (If the program locks up your system, you will have no way to get to the RAM drive!)

If MEMTRACK is not set in the environment, nothing will hang -- you just won't get a debugging file. This provides an easy way to turn off memory tracking without recompiling all your code.

Finally, memTrack_msg() appends a message to the debugging file. If, for some reason, the debugging file could not be opened, the message goes to standard error.

Speaking of the debugging file, check Figure 1 to see what it looks like now. Recall that the memory for "tag 1" was allocated but never freed. In the debugging file, this results in a line that starts with an "A" (for allocated). The memory for "tag 2" was allocated and freed properly. memTrack_free() has over-written an "A" that used to be on line 2 with an "F." This is how you want all your debugging lines to be. Finally, there is a real problem with the line for tag 4: We tried to free a pointer that we never allocated.

Figure 1: The debugging file after memTrack_msg()

  A 2524:000C tag 1
  F 2524: 021A tag 2

  Tried to free 113A:000B (tag 4).  Not   allocated.

Of course, all this housekeeping takes extra processing time so we want to be able to use the standard functions alone in final production. The #include file mem.h (Listing Four, page 180) takes care of this for us. Notice the statement #ifdef MEMTRACK.

If you have #defined MEMTRACK, the header file emerges from the preprocessor as a set of function prototypes. If you have not (#else), the calls to the replacement routines are #defined to be calls to the functions in the standard library. Thus, you will incur no extra overhead in your tested, production version. Even the tag strings will not take up static space in your executable program; they vanish at preprocessing time, when the #define is processed! You can cause MEMTRACK to be #defined by coding #define MEMTRACK in each of your C programs, ahead of your #include mem.h. Alternatively, you can use a command-line switch on your compiler. With the Microsoft compiler, for example, type cl/c/DMEMTRACK program.c.

I prefer the command-line method because it is easier to undo. I just recompile without the switch. The insource method does, however, have one advantage: You can tell whether a given module was compiled with MEMTRACK simply by looking at the source.

If you're careful, you can even #define MEMTRACK selectively. That is, you can compile one subset of programs with /DMEMTRACK and another without. For this to work, each subset must be self-contained as far as memory allocation is concerned -- neither one should free memory allocated by the other.

So far, we have replaced only malloc() and free(). However, there are several other routines in the standard library that allocate memory. We must replace all of them, or we will not have a complete record of our activity. For example, if we do not replace realloc(), our debugging file will have no record of its allocation. memFree() will then raise a spurious objection when we try to free the memory. The other functions in Listing Three complete the job by replacing calloc(), realloc(), and, don't forget, strdup().

To debug a program with the replacement functions, then, there are five steps:

    1. #include <mem.h>

    2. Use the mem routines instead of the standard ones

    3. Compile with MEMTRACK #defined, either in the source- or on the compiler's command line

    4. Link with the mem routines

    5. Set the environment variable MEMTRACK equal to the name of your debugging file.

To turn off debugging you have two choices:

  • Recompile everything without MEMTRACK #defined, and link without the replacement routines. This is the preferred method for a commercial release.
  • Take MEMTRACK out of your environment. The replacement routines will still be a part of your code, but the memTrack routines will not do anything.

Heap Status

Sometimes the situation is so awful, and your program so large, that it is impractical to begin with the approach presented earlier. Or you may think the situation is perfect, and you just want to be sure. The routines that follow give you a summary of the heap's status with one function call. These routines are built on some that Microsoft provides with its C compiler. In the hope that you use Microsoft, or that your compiler provides similar routines, I now refer you to Listing Five (page 180).

You will see that I have made all the same mistakes I made in Listing One. But this time, I have interspersed calls to the heapPrt() function. This function tells me how many allocations I have made, and how many bytes they total. By default, heapPrt() writes to standard output, with puts(). Depending on what else your program does to the screen, this may not be what you want. I have, therefore, coded heapPrt() so you can designate your own message function. In Listing Five, we use my_own_msg_func().

The output at the end of the listing tells the story: We have 10 bytes yet to be freed when the program is done. This is never good, if only from a stylistic point of view. Now, some programmers like to let the operating system free the memory when the program exits. If you are one of those programmers, I tell you that this is like letting your wife or mother pick up your dirty laundry. Sure she'll do it, but sooner or later it's going to bite you. Maybe your program will become a subroutine and strangle your application after enough calls. Or maybe you will have a real memory problem that will be difficult to find amidst all the sloppiness.

Whatever your relationship with your mother, I think you can appreciate the usefulness of heapPrt(). It takes a global view, while the mem ... () routines in the first part of this article are more focused. heapPrt() is particularly useful when you suspect that there is unfreed memory in someone else's code and you can't replace all his malloc()s with the routines in this article.

In a complicated program, the best approach is to call heapPrt() in places where you can predict the status of the heap. Often, you will predict that two successive calls to heapPrt() will produce the same output, because the code in between is supposed to free all the memory it allocates. If your expectation is not met, you can zero in on the problem with more heapPrt() calls, or use the mem ... () routines.

The code for heapPrt() and an associated function, heapUsed(), is in Listing Six (page 180).

A Freebie

At the beginning of this article, I said that 99 percent of slow deaths are caused by memory allocation mistakes. What about the other one percent? Those are the programs that open files but never close them. Eventually, you run out of file handles. That problem is analogous to the memory allocation problem, and I have written a suite of functions to handle it. They are included in the file you can download from this magazine's bulletin-board service. You also may obtain the whole package free of charge by writing me at Cornerstone Systems Group, 10 N. Main St., West Hartford, CT 06107. Please include a formatted, IBM PC-compatible disk.

Summary

I have presented two sets of functions. The first can serve as a bookkeeping layer between your programs and the standard memory management functions. The bookkeeping can be turned off by a quick change to your environment. The bookkeeping layer can be removed completely by recompiling. The second set of functions will give you a snapshot of the heap at any point. Either set of functions can save hours of time in very difficult situations.

_DEBUGGING MEMORY ALLOCATION ERRORS_ by Lawerence D. Spencer

[LISTING ONE]

<a name="01b8_000a">

/* bad.c -- Mistakes in memory allocation */

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>

main()
{
   char *allocated_but_never_freed;
   char *this_one_is_ok;
   char *freed_but_never_allocated;

   allocated_but_never_freed = malloc(10);
   this_one_is_ok            = malloc(20);

   free(this_one_is_ok);
   free(freed_but_never_allocated);

   return(0);
}



<a name="01b8_000b"><a name="01b8_000b">
<a name="01b8_000c">
[LISTING TWO]
<a name="01b8_000c">

/* bad.c -- Mistakes in memory allocation */

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>

#include <mem.h>

main()
{
   char *allocated_but_never_freed;
   char *this_one_is_ok;
   char *freed_but_never_allocated;

   allocated_but_never_freed = memMalloc(10,"tag 1");
   this_one_is_ok            = memMalloc(20,"tag 2");

   memFree(this_one_is_ok,           "tag 3");
   memFree(freed_but_never_allocated,"tag 4");

   return(0);
}




<a name="01b8_000d"><a name="01b8_000d">
<a name="01b8_000e">
[LISTING THREE]
<a name="01b8_000e">

/* memMalloc() -- Same as malloc(), but registers activity using memTrack().
* Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/

#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>

#include <mem.h>

void *memMalloc(size_t bytes, char *tag)
{
   void *allocated;
   allocated = malloc(bytes);
   memTrack_alloc(allocated, tag);
   return(allocated);
}

/* memFree() -- Same as free(), but registers activity using memTrack().
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/

#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>

#include <mem.h>

void memFree(void *to_free, char *tag)
{
   if (memTrack_free(to_free, tag))
      {
      free(to_free);
      }
}
/* MEMTRACK.C -- Module to track memory allocations and frees that occur
* in the other mem...() routines. Global routines:
*     memTrack_alloc() -- Records allocations.
*     memTrack_free()  -- Records attempts to free.
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/

#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>

#include <mem.h>

static FILE  *memTrack_fp(void);
static void   memTrack_msg(char *msg);

#define  ALLOC   'A'
#define  FREE    'F'

/* Track an allocation. Write it in the debugging file in the format
*   A 0000:0000 tag */
void memTrack_alloc(void *allocated, char *tag)
{
   FILE  *fp;

   if (fp = memTrack_fp())
      {
      fseek(fp,0L,SEEK_END);
      fprintf(fp,"%c %p %s\n",ALLOC, allocated, tag);
      fclose(fp);
      }
}

/*  Track freeing of pointer. Return FALSE if was not allocated, but tracking
 *  file exists. Return TRUE otherwise. */
int memTrack_free(void *to_free, char *tag)
{
   int   rc = 1;
   FILE  *fp;
   void  *addr_in_file = 0;
#define  MAX_LTH  200
   char  line[MAX_LTH];
   char  found = 0;
   char  dummy;
   int   ii;
   long  loc;
   if (fp = memTrack_fp())
      {
      rewind(fp);
      for ( loc=0L; fgets(line,MAX_LTH,fp); loc = ftell(fp) )
         {
         if (line[0] != ALLOC)         /* Is the line an 'Allocated' line?  */
            continue;                  /*   If not, back to top of loop.    */
         ii = sscanf(line,"%c %p",&dummy, &addr_in_file);
         if (ii==0 || ii==EOF)
            continue;
                                       /* Is addr in file the one we want?  */
         if ( (char *)addr_in_file - (char *)to_free == 0 )
            {
            found = 1;
            fseek(fp,loc,SEEK_SET);    /* Back to start of line    */
            fputc(FREE,fp);            /* Over-write the ALLOC tag */
            break;
            }
         }
      fclose(fp);
      if (!found)
         {
         char  msg[80];
         sprintf(msg,"Tried to free %p (%s).  Not allocated.",to_free,tag);
         memTrack_msg(msg);
         }
      }
   return(rc);
}

/* Return FILE pointer for tracking file.  */
static FILE  *memTrack_fp()
{
   static char  *ep = NULL;    /* Points to environment var that names file */
   FILE  *fp = NULL;           /* File pointer to return                    */

   if (ep == NULL              /* First time through, just create blank file */
   &&   (ep = getenv("MEMTRACK"))
   &&   (fp = fopen(ep,"w")) )
      {
      fclose(fp);
      fp = 0;
      }
   if (ep)                     /* If we have a file name, proceed.          */
      {                        /*   Otherwise, do nothing.                  */
      fp = fopen(ep,"r+");     /* Open debugging file for append access.    */
      if (!fp)
         {
         fprintf(stderr,"\a\nCannot open %s\n\a",ep);
         }
      }
   return(fp);
}

/* Write a message to the debugging file. */
static void memTrack_msg(char *msg)
{
   FILE  *fp;

   if (fp = memTrack_fp())
      {
      fseek(fp,0L,SEEK_END);
      fprintf(fp,"\n%s\n",msg);
      fclose(fp);
      }
   else
      {
      fprintf(stderr,"%s\n",msg);
      }
}

/* memCalloc() -- Same as calloc(), but registers activity using memTrack().
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/
#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>
#include <mem.h>

void *memCalloc(size_t num_elems, size_t bytes_per_elem, char *tag)
{
   void *allocated;
   allocated = calloc(num_elems, bytes_per_elem);
   memTrack_alloc(allocated, tag);
   return(allocated);
}

/* memRealloc() - Same as realloc(), but registers activity with memTrack().
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/
#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>
#include <mem.h>

void *memRealloc(void *allocated, size_t bytes, char *tag)
{
   memTrack_free(allocated, tag);
   allocated = realloc(allocated, bytes);
   if (allocated)
      {
      memTrack_alloc(allocated, tag);
      }
   return(allocated);
}

/* memStrdup() -- Same as strdup(), but registers activity using memTrack().
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/
#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>
#include <string.h>
#include <mem.h>

void *memStrdup(void *string, char *tag)
{
   void *allocated;
   allocated = strdup(string);
   memTrack_alloc(allocated, tag);
   return(allocated);
}




<a name="01b8_000f"><a name="01b8_000f">
<a name="01b8_0010">
[LISTING FOUR]
<a name="01b8_0010">


/* MEM.H  -- ** Copyright (c) 1990, Cornerstone Systems Group, Inc. */

#ifdef MEMTRACK

void *memCalloc(size_t num_elems, size_t bytes_per_elem, char *tag);
void  memFree(void *vp, char *tag);
void *memMalloc(size_t bytes, char *tag);
void *memRealloc(void *oldloc, size_t newbytes, char *tag);
void *memStrdup(void *string, char *tag);
     /* The next two functions are only called by the other mem functions   */
void memTrack_alloc(void *vp, char *tag);
int  memTrack_free(void *vp, char *tag);
#else
#define  memCalloc(NUM,BYTES_EACH,TAG)       calloc(NUM,BYTES_EACH)
#define  memFree(POINTER,TAG)                free(POINTER)
#define  memMalloc(BYTES,TAG)                malloc(BYTES)
#define  memRealloc(OLD_POINTER,BYTES,TAG)   realloc(OLD_POINTER,BYTES)
#define  memStrdup(STRING, TAG)              strdup(STRING)
#endif



<a name="01b8_0011"><a name="01b8_0011">
<a name="01b8_0012">
[LISTING FIVE]
<a name="01b8_0012">

/* DEMOHEAP.C - Demonstrate use of heap...() functions.
*  Copyright (c) 1990 - Cornerstone Systems Group, Inc.
*/

#include <stdio.h>
#include <malloc.h>
#include <heap.h>

static void my_own_msg_func(char *msg);

main()
{
   char *allocated_but_never_freed;
   char *this_one_is_ok;
   char *freed_but_never_allocated;
   heapPrt_set_msg_func(my_own_msg_func);
   allocated_but_never_freed = malloc(10);
   heapPrt("after first malloc()");
   this_one_is_ok            = malloc(20);
   heapPrt("after second malloc()");
   free(this_one_is_ok);
   heapPrt("after first free()");
   free(freed_but_never_allocated);
   heapPrt("after second free()");
   return(0);
}

/* heapPrt() makes its report with puts() by default.  This will not be
*  appropriate for some applications, so we will demonstrate the use of an
*  alternative message function.  This one writes to stderr.
*  The alternative function should take one argument (a char *).  Its
*  return value is ignored, so it might as well be void.
*/
static void my_own_msg_func(char *msg)
{
   fprintf(stderr,"My own message function: %s\n",msg);
}
OUTPUT:
My own message function:     1 allocations,     10 bytes, after first malloc()
My own message function:     2 allocations,     30 bytes, after second malloc()
My own message function:     1 allocations,     10 bytes, after first free()
My own message function:     1 allocations,     10 bytes, after second free()



<a name="01b8_0013"><a name="01b8_0013">
<a name="01b8_0014">
[LISTING SIX]
<a name="01b8_0014">

/* heap.h - Header file for use with heap...() functions.
*   Copyright (c) 1990 - Cornerstone Systems Group, Inc.
*/

void heapPrt(char *tag);
void heapPrt_set_msg_func(void (*new_msg_func)() );
void heapUsed(unsigned int *numused, long *totbytes);

/* HEAPUSED.C -- Tell how much of heap has been used. For use with MS C 5.x
*  Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/
#include <malloc.h>
#include <heap.h>

void heapUsed(
unsigned int   *numused,
long           *totbytes)
{
   struct _heapinfo hinfo;
   int    status;
   *numused  = 0;
   *totbytes = 0L;
   hinfo._pentry = (char *)0;
   while ( (status=_heapwalk(&hinfo)) == _HEAPOK)
      {
      if (hinfo._useflag == _USEDENTRY)
         {
         ++ (*numused);
         *totbytes += hinfo._size;
         }
      }
}

/* HEAPPRT.C -- Print summary information about heap. For use with MS C 5.x
*   This module contains two functions:
*     heapPrt() prints the summary information.
*     heapPrt_set_msg_func() allows you to specify a function for heapPrt()
*        to use, other than printf().
*   Copyright (c) 1990, Cornerstone Systems Group, Inc.
*/
#include <stdio.h>
#include <malloc.h>
#include <heap.h>
static void (*heapPrt_msg_func)() = 0;

/*------------------------------------------------------------------------*/
void heapPrt(
char  *tag)        /* Description of where you are in processing            */
{
   unsigned int   numused;          /* Number of allocations used           */
   long           totbytes;         /* Total bytes allocated                */
   char           msg[80];          /* Message to display                   */
   heapUsed(&numused, &totbytes);
   if (!heapPrt_msg_func)
      heapPrt_msg_func = puts;
   sprintf(msg, "%5u allocations, %6ld bytes, %s",numused,totbytes,tag);
   heapPrt_msg_func(msg);
}
/*------------------------------------------------------------------------*/
void heapPrt_set_msg_func(
void (*new_msg_func)())
{
   heapPrt_msg_func = new_msg_func;
}
/*------------------------------------------------------------------------*/










Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.