Channels ▼


Domain Usage Tracking for Windows NT

Source Code Accompanies This Article. Download It Now.

Dec98: Domain Usage Tracking for Windows NT

Paul is a technology coordinator for the College of Applied Human Sciences at Colorado State University, where he develops network management and control applications. He can be contacted at [email protected]

The College of Applied Human Sciences at Colorado State University operates four general-purpose computer labs (located in different buildings) for approximately 4000 undergraduate and graduate students. But how heavily, how often, and by how many students are they used? These are the kind of questions we ask every year as we plan upgrades and schedules. To answer questions such as these, I present in this article an application I developed that lets administrators track usage by workstation or user on Windows NT domains.

The usage-tracking system has three pieces:

  • Srvuse.exe, written in C, which reads the event log from specified servers, matches logon/logoff records, sums usage by workstation/user and day/hour, and stores the data in comma-delimited text files.
  • Dbload.exe, also written in C, which reads the text files, and loads the data into an ODBC Data Source (usage.mdb, in this case).
  • Usage.mdb, an Access database, which holds the data and routines for printing the usage reports and graphs.

Srvuse and dbload are generic and independent of domain organization. Usage.mdb is not. The SQL statements that drive the usage reports are tied to a specific workstation naming convention. However, this is the smallest piece of the system, and easily customized. Here, I will concentrate on the heart of the system -- srvuse.exe. Source code and executables for the usage-tracking system are available electronically; see "Resource Center," page 3.

Srvuse Overview

Windows NT provides a simple way to track events (such as logging on or off the network) by utilizing built-in event logs:

  • The System log is an event log for system error messages.
  • The Application log is for programs such as antivirus tools, network backup, database servers, and the like.
  • The Security log holds all of the messages relating to system security: logon/logoff information, object accesses, permission modifications, and so on. It contains all the data about who logged on when, and when he/she logged off.

Each log entry has the same format. First is the header information: length, record number, time the record was generated, time the record was written, event ID, event type, number of strings in the string data section, event category, string data section offset, length of a user security ID, user security ID offset, length of the data section, and the data section offset. After the header information, the name of the source of the record is followed by the name of the computer that generated the record. Both are null-terminated strings. If the source of the record logged the name of the computer, a user security ID is next.

Finally, we come to the record's actual data. First is a section for string data. Each string is a null-terminated array of characters. Following the string data is the binary data. The binary data is specific to the record source, and of value to either the source or its developer. The record ends with enough pad bytes to ensure the DWORD alignment, followed by a repeat of the record length. The length is repeated to make it easier to move backward through an array of these structures. Microsoft calls this structure EVENTLOGRECORD.

When logon/logoff auditing is enabled, each successful logon or logoff generates an event in the Security Event Log of the validating domain controller. A successful logon has an event ID of 528. The logoff event ID is 538. While both of these records have an identical format (EVENTLOGRECORD), the string data section contains different information. For the logon record, it holds the user name, domain, session ID, logon type, logon process, authentication package used to validate the logon, and name of the workstation where the user logged on. The logoff record holds much the same information: the user name, domain, session ID, and logon type. Based on empirical analysis, the session ID appears to be unique not only within a single domain controller, but also within the entire domain. Therefore, the session ID makes the perfect field to match a logon record to its corresponding logoff record. Five steps are required to condense the event log data from a single domain controller into a usable format:

  • 1. Read the Security Event log and extract all records with an ID of 528 or 538. The read_sec_log function performs this task.
  • 2. Take all logon and logoff records and match them together with the Session ID field. This step is done by match_on_off.
  • 3. Count the number of unique workstations, users, and dates involved, by using count_unique.
  • 4. Using sum_by_day, total the number of seconds each unique user or workstation was in use for each unique day.
  • 5. Flag which hours of the day each user or workstation was logged on, by using sum_by_hour.


Since each step has to be performed on each domain controller, I decided to make srvuse multithreaded. This way, the data gathering process would run on all servers simultaneously, and, hopefully, take less time. I decided to run each step in parallel, rather than the entire process. This allowed me to debug one function at a time, minimizing the number of possible interactions. While multithreading is not complicated, it does require a slightly different design outlook. My biggest problem was passing data from one step of the process to the next, because each thread function can have only one parameter, an unsigned long integer.

I defined a structure called control that holds the server name, server's UNC name, names of the three temporary files, names of the four output files, seven counter variables, and two array pointers. In main(), an array of these control structures is allocated and most of the data initialized. A global pointer lets each thread access the array, and the offset of the appropriate control structure is passed to each function.

Having solved my argument-passing problem, I encountered another complication. There are three functions for creating threads: CreateThread, _beginthread, and _beginthreadex. The first is the Win32 API function for thread creation. The next two come from Microsoft's C run-time library (RTL). I originally used CreateThread, which offers all of the options for thread creation: security attributes, stack size, thread parameters, and the ability to create a suspended thread. However, CreateThread has a major drawback: If you use any functions from the C RTL, your application suffers memory leaks because ExitThread does not reclaim the memory used by these functions. This means no calloc, free, malloc, or memcpy. Although these functions have Win32 equivalents, it also means no strcpy, strcmp, strchr, and the like -- and these are harder to do without, especially if you're doing any sort of character parsing or manipulation. I also needed to use the C RTL functions to manipulate the event record's time/date stamp. The answer to this problem lay in the Microsoft C run-time function _beginthreadex, which has an argument list almost identical to that of CreateThread. Furthermore, all of the Win32 synchronization options work with the handle returned by _beginthreadex. Most importantly, _endthreadex does release the memory used by C RTL functions. _beginthreadex requires the __stdcall calling convention, so I had to use the /Gz compiler switch.

My final multithreading problem was converting the time/date stamp of an event log record into a meaningful time and date. The time/date stamp is the number of seconds that have elapsed since January 1, 1970. This stamp is also Coordinated Universal Time (also known as UTC, Greenwich Mean Time, or GMT). Therefore, to do anything with it, you must convert this long integer to real month, day, year, hour, minute, and second values. The C RTL function localtime, used to perform the conversion, uses a single static workspace buffer. This means consecutive calls will destroy prior results. More importantly, it means two or more overlapping calls will return bizarre results. The only solution was to limit access to the localtime function to one thread at a time. The Win32 API has a full set of thread and process synchronization primitives. I chose the Critical Section. I defined a global CRITICAL_SECTION variable, timestampcvt, and initialized it in main with InitializeCriticalSection. Then, in match_on_off, just prior to the timestamp conversion, I call EnterCriticalSection. Until the LeaveCriticalSection is called, all other threads will block, waiting for access to the conversion section of code. I call localtime to convert the logon time, store the results, and repeat the process for the logoff time. Finally, I call LeaveCriticalSection, allowing the other threads access to localtime.

When the program was finished, I tested the run time for four servers sequentially (four separate invocations of srvuse), and four servers simultaneously. While this was not a formal test, the simultaneous run time was approximately one third the time of the sequential process. The extra work was worth it.

Other Win32 and C RTL Functions used in Srvuse

Synchronization was not limited to accessing localtime. At each stage, I also had to make sure that all threads were finished before beginning the next. Early on, I queried the status of each thread in a while loop; this was not efficient. The WaitForMultipleObjects function puts the parent thread in a low overhead wait loop, preventing the next step's threads from being created until all of the current threads are finished.

Of course Win32 has a complete API for event log processing. I only needed four: OpenEventLog, GetNumberOfEventLogRecords, ReadEventLog, and CloseEventLog. OpenEventLog takes the computer name and the name of the event log to open (System, Security, or Application) and returns a handle that will be used in all future calls. GetNumberOfEventLogRecords takes this handle and returns the number of records in the log. ReadEventLog gets data from the event log. The flags EVENTLOG_SEQUENTIAL_READ and EVENTLOG_BACKWARDS_READ return the records in the same order as the Event Viewer application. CloseEventLog closes the event log handle.

Putting it all Together

Processing begins in main, where the server names are pulled from the command line, the array of control structures is allocated and initialized, and threads are created. Each step is represented by one function, and one set of threads.

rd_sec_log (Listing One, at the end of this article) opens the Security event log for a specific server, and reads through the records, looking for logon (event ID 528) and logoff (event ID 538) records. When one is found, the appropriate strings are extracted from the string data section, and either an onrec or an offrec structure is written out to one of two temporary files (one for the logon records, and one for the logoff records).

After rd_sec_log, match_on_off pairs each logon record with the matching logoff record. These combined records are stored in onoffrec structures and written to another temporary file. The logon/logoff records are paired to the unique logon ID. If a logon record has no corresponding logoff record, or vice versa, then no onoffrec structure is created. This will happen if srvuse is executed while users are logged on. I avoid this by executing srvuse when the labs are closed and all workstations are logged off. match_on_off is where the Critical Section maintains single-thread access to localtime. All of the processing takes place in RAM. At the end of the matching, the temporary files with the logon and logoff records are deleted.

At this point, the data has been extracted. We know who logged in at which workstation, and when they logged off. To be useful, the data must be divided by user and workstation. By grouping the logons by workstation, we track lab usage by day and hour. Grouping by user lets us track the usage based on class rosters and schedules.

count_unique extracts the unique workstation names, user names, and dates, and builds arrays of workstation-per-day and user-per-day totals. Once the unique workstation names, user names, and dates have been identified, a record for each workstation/day and each user/day is created. This generates some extraneous data because it creates a record for each workstation or user for each day, including workstations that were not used and users who did not logon on that particular day.

The sum_by_day function totals the seconds each workstation or user was logged in for each specific day. It steps through each of the logon/logoff pairs, and adds the time for each session to the appropriate workstation/day or user/day record. When this summing is completed, it creates two text files. The first is server.wrk, and the second is server.usr where server is replaced with the name of the domain controller being processed. The files are comma-delimited data files with the user or workstation name, date, and the total seconds the workstation or user was logged in that day.

The sum_by_hour function processes the logon/logoff records and sets a flag for each hour in which the workstation or user was logged in. It goes through the records created by count_unique, and for each combo record sets the appropriate hour flag in the hourly usage record. If the workstation or user was logged in during an hour, that hour gets flagged 1; otherwise it gets a 0. A user that logged in at 08:35:26 am and logged out at 10:26:44 am gets 8 am, 9 am, and 10 am flagged in the hourly usage record. Just like sum_by_day, sum_by_hour creates two text files as output -- server.uhr and server.whr for user/hour and workstation/hour, respectively. Each record has 26 fields: workstation or user name, date, and one field for each hour of the day.

Finally, we have four text files for each server. Srvuse has finished execution, and it's time to update the database. Dbload opens each text file, and inserts the records into the database using SQL INSERT statements and an ODBC data source. While I did create dbload for this project, its purpose is sufficiently focused to allow it to be treated as a black box in this discussion. Complete source code is available electronically. The final piece of the system comes from the Access database itself. Currently, there are two reports:

  • Lab Usage - Aggregate, which displays total usage per day by lab.
  • Lab Usage - Hourly Summaries, which shows which workstations were in use during which hours for each day.

An Access form, Report Center, controls which reports are displayed, and the beginning and ending dates. By separating the data collection from the reporting, I can give administrators access to the reports via the database, and I do not have to worry about keeping updates synchronized.

Everything is Working Fine, Right?

After the system had been in use for about two months, I began to notice the daily totals changing. The change was always an increase, and seemed to begin about a week after the data for the day was gathered. Since the database is keyed to not allow duplicate entries for each workstation/day or user/day total, I thought it was a bug in srvuse. After several days of searching and testing, I discovered the cause. We had configured the event logs on each server to use five MB and to overwrite old events as necessary. As time goes on, older records are overwritten in the event log, a new daily total for a particular user or workstation will appear. Since this entry is not identical to the original, the record is inserted into the database with no errors. Consequently, after a period of time, the totals begin to increase and continue to do so until all of the entries for that day have been overwritten in the event log. Needless to say, this caused me some anxiety, since we were counting on this system to give us accurate usage information. However, I was saved by my supervisor. As it turned out, he was viewing the usage totals each day, and transcribing the four lab totals into an Excel spreadsheet. His spreadsheet reflected the actual usage, and was not affected by the upward drift. A database modification will correct the problem, leaving srvuse unchanged.


This system went live in April 1997. At the end of May, for the first time, we were able to do a real analysis of lab usage during the last three weeks of the semester. In July, we loaned one of our labs to the university, and were able to give them information on how often and when it was used. At the end of July, two of the four labs were wiped out by a flash flood that struck our campus. Naturally, with only two labs for faculty to teach in and students to use, we wondered if we had enough capacity to continue operation. By using the Aggregate Usage report, we have been able to monitor the usage and make sure that our lab capacity is sufficient.

Trying to determine when and how your network is used remains one of the most critical network administration tasks. It's further complicated by a lack of simple, low-cost applications for monitoring and data collection. Windows NT and the Win32 API offer most of the pieces necessary to do the job yourself.


I would like to thank the College of Applied Human Sciences at Colorado State University for supporting this development, and my supervisor, Tom Mazzarisi, for asking the questions in the first place.


Listing One

/********************************************************************* rd_sec_log return values:
*     1 = Success
*   247 = Error locking event log record buffer 
*   248 = Error allocating event log record buffer
*   249 = Error creating .on file
*   250 = Error creating .off file
*   251 = Error allocating ofr buffer
*   252 = Error allocating onr buffer
*   253 = Error getting number of event log records
*   254 = Error opening security event log
* Compile: cl /MT /nologo /W3 /D "WIN32" /D "CONSOLE"  
*             /Zp1 /Gz /c rd_sec_log.c
#include <stdio.h>
#include <stdlib.h>
#include <process.h>    /* _beginthreadex & _endthreadex */

#include <windows.h>
#include <winnt.h>
#include <winbase.h>    /* Eventlog,Thread,Sych & Memory Mgmt API */

#include "srvuse.h"     /* Application specific */

/*** Global so it's visible to ALL threads, functions, etc. ***/
extern struct control   *ctrl;     /* Array of control structures */
unsigned long int rd_sec_log(unsigned long int *srv) { 
  unsigned char     *tmp0;         /* Convert session id to 2 longs */
  unsigned char     *tmp1;         /* Convert session id to 2 longs */
  unsigned char     *tmp2;         /* Convert session id to 2 longs */
  unsigned char     *evtbuf=NULL;  /* Current event log record */
  unsigned char     *strbeg;       /* Start eventlog entry string */
  unsigned int       rtc=1;        /* Function return code */
  unsigned int       ctr;          /* Scratch FOR loop counter */
  unsigned int       recctr;       /* Event log record counter */
  unsigned int       evtread=0;    /* Bytes read from log */
  unsigned int       evtreq=0;     /* Bytes required next entry */
  unsigned long int  evtrecs;      /* Number of event log records */
  unsigned long int  evtreadflag=0;/* Event log read flags */
  unsigned long int  stroff;       /* String offset event log rec*/
  unsigned long int  bytes_writ=0; /* Bytes written with file write */
  struct onrec      *onr;          /* Logon record buffer */
  struct offrec     *ofr;          /* Logoff record buffer */
  BOOL               tfrtc;        /* True/False return code */
  HANDLE             evtlog;       /* Eventlog handle */
  HANDLE             tmpon;        /* Temporary file for onrecs */
  HANDLE             tmpoff;       /* Temporary file for offrecs */
  HGLOBAL            bufhnd;       /* Buffer memory handle */

  /*** Open server's Security Event Log. If there is an error,  
   ***  return a status code of 254. ***/
  if(evtlog==NULL) {
    goto RD_SEC_LOG_EXIT;
  /*** Retrieve the number of event log records.  If there is an error,
   ***   return a status code of 253. ***/
  if(tfrtc==FALSE) {              
    goto RD_SEC_LOG_EXIT;         
  /* Initialize lgrecs field in the proper control structure */

  /*** Allocate a buffer for one logon record.  If there is an error,
   ***   return a status code of 252 ***/
  onr=(struct onrec *)GlobalAlloc(GPTR,sizeof(struct onrec));
  if(onr==NULL) {                
    goto RD_SEC_LOG_EXIT;        
  /*** Allocate a buffer for one logoff record.  If there is an error,
   ***   return a status code of 251 ***/
  ofr=(struct offrec *)GlobalAlloc(GPTR,sizeof(struct offrec));
  if(ofr==NULL) {                
    goto RD_SEC_LOG_EXIT;        
  /*** Open a temporary file to hold all of the logoff records.  If 
   ***   there is an error, return a status code of 250 ***/
  if(tmpoff==INVALID_HANDLE_VALUE) {
    goto RD_SEC_LOG_EXIT;           
  /*** Open a temporary file to hold all of the logon records.  If 
   ***   there is an error, return a status code of 249 ***/
    goto RD_SEC_LOG_EXIT;          
  /*** Allocate a buffer for one event log record.  If there is an 
   ***   error, return a status code of 248 ***/
  if(bufhnd==NULL) {           
    goto RD_SEC_LOG_EXIT;      
  /* Lock the event record buffer in memory */
  /*** Lock the event log record in memory.  If there is an error, 
   ***   return a status code of 247 ***/
  evtbuf=(unsigned char *)GlobalLock(bufhnd); 
  if(evtbuf==NULL) {             
    goto RD_SEC_LOG_EXIT;        
  /*** Process every record in the Security log ***/
  for(recctr=0; recctr<evtrecs; recctr++) {
    /*** Set the event log read flags to access log in same order as 
     ***   event viewer application ***/
    /*** Read one event log record ***/
    /*** If there was an error, and it was caused by buffer being too small,
     *** re-allocate the buffer to the needed size, and retry the read ***/
    if(tfrtc==FALSE && GetLastError()==ERROR_INSUFFICIENT_BUFFER) {
     bufhnd=GlobalReAlloc(bufhnd,evtreq*sizeof(unsigned char),GMEM_ZEROINIT);
     evtbuf=(unsigned char *)GlobalLock(bufhnd);
    /*** If there was no error, and record is a logon record process data,
     *** save it to the onrec buffer, and write the buffer out to disk  ***/
    if(((EVENTLOGRECORD *)evtbuf)->EventID==528) {
      stroff=((EVENTLOGRECORD *)evtbuf)->StringOffset;
      strbeg=(((unsigned char *)evtbuf)+stroff);
      onr->ontime=((EVENTLOGRECORD *)evtbuf)->TimeGenerated;
      for(ctr=0; ctr<7; ctr++) {   /* Logon record has 7 strings */
        if(ctr==0)                 /* User name is first string */
        if(ctr==1)                 /* Domain name is second string */
        /*** With the session id, since it gets saved as 2 longs, convert it 
         *** using strtol, after finding the separating comma ***/
        if(ctr==2) {               /* Session ID is third string */
          tmp0=strbeg;             /* Point to the ( */
          tmp0++;                  /* And then 1 past it */
          tmp1=strchr(strbeg,','); /* Point to dividing comma */
          tmp1++;                  /* And then 1 past it */
        if(ctr==6)                     /* Workstation is 7th string */
          if(strbeg[0]=='\\')          /* If the name is UNC name */
            strcpy(onr->wrk,strbeg+2); /* Leave off the \\ */
          else                         /* Otherwise */
            strcpy(onr->wrk,strbeg);   /* Just copy it */
        strbeg=strchr(strbeg,'\0');    /* Point to end this string */
        strbeg++;                      /* Point to beg next string */
      tfrtc=WriteFile(tmpon,onr,sizeof(struct onrec), &bytes_writ,NULL);
      /*** Zero the record after write ***/
      ZeroMemory(onr,sizeof(struct onrec));
   /***Logoff record processing matches logon processing, except ofrec buffer
   ***is used, and there is less information in string portion of record ***/
    if(((EVENTLOGRECORD *)evtbuf)->EventID==538) { 
      stroff=((EVENTLOGRECORD *)evtbuf)->StringOffset;
      strbeg=(((unsigned char *)evtbuf)+stroff);
      ofr->offtime=((EVENTLOGRECORD *)evtbuf)->TimeGenerated;
      for(ctr=0; ctr<4; ctr++) {  /* Logoff record has 4 strings */
        if(ctr==2) {              /* Session ID is third string */
          tmp0=strbeg;            /* Point to the ( */
          tmp0++;                 /* And then 1 past it */
          tmp1=strchr(strbeg,',');/* Point to dividing comma */
          tmp1++;                 /* And then 1 past it */
        strbeg=strchr(strbeg,'\0');/* Point to end this string */
        strbeg++;                  /* Point to beg next string */
      tfrtc=WriteFile(tmpoff,ofr,sizeof(struct offrec), &bytes_writ,NULL);
      ZeroMemory(ofr,sizeof(struct offrec));
    /*** Unlock event log record buffer, resize it, and relock it ***/
    evtbuf=(unsigned char *)GlobalLock(bufhnd);
  }                                    /* End of processing loop */
  * Using a switch case statement for error processing, allows the
  * function to properly terminate no matter where the error occurred
  switch(rtc) {
    case   1 :
    case 247 : evtbuf=GlobalFree(bufhnd);
    case 248 : tfrtc=CloseHandle(tmpon); 
    case 249 : tfrtc=CloseHandle(tmpoff);
    case 250 : ofr=GlobalFree(ofr);      
    case 251 : onr=GlobalFree(onr);      
    case 252 :
    case 253 : tfrtc=CloseEventLog(evtlog);
    case 254 : _endthreadex(rtc);          

Back to Article


Copyright © 1998, Dr. Dobb's Journal

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.