Channels ▼
RSS

Web Development

Enhancing Terminal Output in Perl


Jul03: Enhancing Terminal Output in Perl

Enhancing Terminal Output in Perl

The Perl Journal July 2003

By Shay Harding

Shay has worked with transaction processing systems at CCBill LLC, for the last five years and can be contacted at sharding@ccbill.com.


This article discusses how to make terminal output easier to read and monitor. If you are like me and spend most of the day at a UNIX command prompt, you'll probably benefit from using Term::Report and Term::StatusBar, two modules I created for making terminal output easier to track. They can be used separately, but are best if used together. These modules have few dependencies, which are loaded as required, so there is no need to install anything new.

Typically, programs send their output to STDOUT or STDERR. If there are too many lines sent to the terminal, they end up scrolling and may be irretrievable depending on your terminal's buffer size. You can send the output to a file, but then how do you monitor the program to make sure everything is going well? You could tail -f <file>, but then you run into the same problem of output scrolling off of the screen.

The most important components of useful terminal output are:

1. The ability to see that the program is processing.

2. The ability to see what the program is doing.

3. The ability to track the program's progress.

With these criteria met, there is no doubt about whether the program has hung or what progress it has made in processing the data.

Standard Terminal Output

A simple example of terminal output seen in many scripts is shown in Listing 1. This first example will not utilize Term::Report or Term::StatusBar. This will be modified later in order to show how the Term modules work and how they can be useful.

This listing prints the output to the terminal. If there were thousands of items to iterate over, the output would end up scrolling off the screen. Another problem is that there is no way to gauge progress and tell how long it might take to finish. The data sent to the terminal ends up being cluttered and not very useful. In processing larger data sets, it would be difficult to determine the program's output by using this method.

Cleaning up the Output

The previous output can be organized with the help of Term::Report. With minor alterations, the amount of output can be reduced, thus improving readability; see Listing 2.

The use of Term::Report usually doesn't get any more complicated than the aforementioned example. But even this simple implementation of Term::Report makes the output more organized and easier to follow. It is readily apparent that the program is working and what data it is processing. Two important criteria for useful terminal output have been met.

If there are thousands of items to iterate over, the constructor can be changed as shown below:

my $report = Term::Report->new(startRow => 1, numFormat => 1);
</PRE>


This would format numbers using <i>Number::Format</i> (i.e., 1000 becomes 1,000). This makes it even easier to read the output quickly.</p>



The last criterion, the ability to track the program's progress, is accomplished by using <i>Term::StatusBar</i>. Rather than create a separate object, use <i>Term::Report</i>'s ability to wrap the <i>Term::StatusBar</i> module. (See <A NAME="rl3"><A HREF="#l3">Listing 3</A>.)</p>



Notice that <i>Term::Report</i> has been used to create a status bar. The status bar needs to know how many items there are to process. Then it's just a matter of calling <i>StatusBar->update()</i> with each iteration of data processing.</p>



When updating the inventory in the aforementioned example, the status bar is reset rather than creating a new object. To tell the status bar to empty rather than fill, pass <i>reverse => 1</i> to the <i>reset()</i> method. This is a recent addition to <i>Term::StatusBar</i>. Calling the <i>printBarReport() </i>method outputs our statistics summary. This prints a horizontal bar chart based on the final values and scale of the status bar.</p>



With these minor changes, all three criteria for useful terminal output are satisfied. Use the <i>subText</i> and <i>subTextAlign</i> methods of <i>Term::StatusBar</i> to enhance the output further. These place information just under the status bar to show what the program is currently processing.</p>

<PRE>
my $report = Term::Report->new(
   startRow => 4,
   numFormat => 1,
   statusBar => [
      label => 'Widget Analysis: ',
      subText => 'Locating widgets',
      subTextAlign => 'center'
   ],
);
...
if (!($_%int((rand(10)+rand(10)+1)))){
   $report->finePrint('discarded', 0, ++$discard);
   $status->subText("Discarding bad widget");
}
else{
   $status->subText("Locating widgets");
}
Another new addition to Term::StatusBar is the showTime parameter. When turned on, an estimated time to completion is placed at the top of the status bar. Notice in the code below, the value of startRow has changed. This is to allow space for the estimated completion time. In a future version, this sort of manual adjustment probably will not be necessary.

my $report = Term::Report->new(
   startRow => 5,
   numFormat => 1,
   statusBar => [
      label => 'Widget Analysis: ',
      subText => 'Locating widgets',
      subTextAlign => 'center',
      showTime => 1
   ],);
If the module is unable to figure out the estimated time, then "00:00:00" will be displayed. When using the reverse method, there is no estimated time tracked. This will be possible in future releases of the module.

Caveats and Possible Enhancements

Term::StatusBar has some limitations. It requires knowing up front how many items there are to process. This is so it can properly set its scale and appropriately update progress. Problems might arise in processing an extremely large file. Many computers do not possess enough memory to load an entire file in order to determine how many lines it contains.

Even if memory is not a factor, it would take a while to read these kinds of files twice: once for Term::StatusBar and once to process the data. We can determine a file's size by using the file test operator -s. This is only useful if the program reads a byte at a time. Usually, a file is read line-by-line, and each line may not be the same length in bytes. One strategy would be to sample lines in the file some number of times. The sysopen, sysseek, and sysread functions can be used to avoid using a lot of memory. However, this would only give a good guess and could be completely wrong depending on the unevenness of the line lengths of the file.

Another, and possibly more accurate way would be to use the file's size as a basis. Then, when calling Term::StatusBar->update(), pass it the length of the line just processed. This would allow accurate tracking of progress as the file was processed. This would also allow the processing of many types of files in the same fashion.

Another possible addition in the near future is a way to "serialize" the output to a file and reinstate it to the terminal at a later point in time. This would allow a process to run in the background and be monitored periodically.

While Term::Report and Term::StatusBar may not be perfect, they can help improve readability of terminal output. These modules give you the ability to monitor a program's progress and quickly determine its processing status. They may not work in every situation, but the goal is to make them usable in many situations.

TPJ

Listing 1

#!/usr/bin/perl

$|++;
use Time::HiRes qw(usleep);

my ($items, $discard) = (100,0);

## Monitor inventory (L=Locating; D=Discarding)
for (1..$items){
   if (!($_%int((rand(10)+rand(10)+1)))){
      $discard++;
      print " D ";
   }
   else {
      print "L";
   }

   usleep(50000);
}
print "\n";

## Update inventory
for (1..($items-$discard)){
   print "U";
}

print "\n\n\n    Summary for widgets: \n\n".
      "       Total:        $items\n".
      "       Good Widgets: ".($items-$discard)."\n".
      "       Bad Widgets:  $discard\n\n";

Back to Article

Listing 2

#/usr/bin/perl

$|++;
use Time::HiRes qw(usleep);
use Term::Report;

my $report = Term::Report->new(startRow => 1);
my ($items, $discard) = (100,0);

$report->savePoint('total', "Total widgets: ", 1);
$report->savePoint('discarded', "\n  Widgets discarded: ", 1);

## Monitor inventory 
for (1..$items){
   $report->finePrint('total', 0, $_);

   if (!($_%int((rand(10)+rand(10)+1)))){
      $report->finePrint('discarded', 0, ++$discard);
   }

   usleep(50000);
}

## Update inventory
$report->savePoint('inventory', "\n\nInventorying widgets... ", 1);

for (1..($items-$discard)){
   $report->finePrint('inventory', 0, $_);
}

$report->printLine("\n\n\n\n    Summary for widgets: \n\n");
$report->printLine("       Total:        $items\n");
$report->printLine("       Good Widgets: ".($items-$discard)."\n");
$report->printLine("       Bad Widgets:  $discard\n");

Back to Article

Listing 3

#!/usr/bin/perl

$|++;
use Time::HiRes qw(usleep);
use Term::Report;

my $report = Term::Report->new(
   startRow => 4,
   numFormat => 1,
   statusBar => [
      label => 'Widget Analysis: ',
   ],
);
my ($items, $discard) = (100,0);

my $status = $report->{statusBar};
$status->setItems($items);
$status->start;

$report->savePoint('total', "Total widgets: ", 1);
$report->savePoint('discarded', "\n  Widgets discarded: ", 1);

## Monitor inventory
for (1..$items){
   $report->finePrint('total', 0, $_);

   if (!($_%int((rand(10)+rand(10)+1)))){
      $report->finePrint('discarded', 0, ++$discard);
   }

   usleep(50000);
   $status->update;
}

## Update inventory
$status->reset({
    reverse=>1, 
    setItems=>($items-$discard), 
    start=>1
});

$report->savePoint(
   'inventory', 
   "\n\nInventorying widgets... ", 
   1
);

for (1..($items-$discard)){
   $report->finePrint('inventory', 0, $_);
   $status->update;
}

   $report->printBarReport(
      "\n\n\n\n    Summary for widgets: \n\n",
      {
            "       Total:        " => $items,
            "       Good Widgets: " => $items-$discard,
            "       Bad Widgets:  " => $discard,
      }
   );



Back to Article


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 
Dr. Dobb's TV