Channels ▼

Al Williams

Dr. Dobb's Bloggers

Linux File System Monitoring

August 05, 2011

A lot of the embedded systems I work on lately run Linux. I enjoy that, because I'm a long-time UNIX user and I run nothing but Linux on my personal computers and servers these days (well, assuming you count Android as a form of Linux, which I do).

However, I'm struck by how embedded Linux development is often more akin to dealing with a Linux server and not a Linux desktop. On the desktop you expect GUI software and grand interactions with DBUS. On a server, you are just trying to get stuff done, usually without human intervention on the server-side. I find I keep pulling my server tricks out to serve on my embedded systems.

The other day I was working on a system where an external entity was to transfer a file to an embedded board (say, for example, via FTP). When the file shows up, the embedded board was supposed to do something with it. To protect the innocent, I can't give you all the details, but as an illustration, let's say that you were working on a book reader and when a book file shows up in a certain directory, you want to add it to the book reader's database.

The simple but ugly way to do this is just scan the directory periodically. Here's a really dumb shell script:

  while true
     do
        for I in 'ls'
           do cat $I; rm $I
        done
   sleep 10
done

Just for an example, I dump the file to the console and remove it, but in real life you'd do something more interesting. This is really not a good script because it executes all the time (although most of it sleeping) and it just isn't a very elegant solution. By the way, if you think I should use for I in * try doing that in an empty directory and you'll see why I use the ls command instead.

Honestly, though. You want something more elegant right? Modern kernels (2.6.13 and later) have filesystem notifications in the form of an interface called inotify. You can use these calls programmatically with the sys/inotify.h header file. There is also a set of command-line programs you can install (packaged as inotify-tools, usually).

One of those tools is inotifywait and it makes for a nicer script. For example:

while true
     do
       if FN='inotifywait –e close_write,moved_to --format %f .'
        then
          cat $FN
          rm $FN
       fi
done

That's better, I think. It doesn't wake up frequently, just when something's changed. I figure any sane program putting something in the directory will either open the file for writing and close it, or it will move it. Either way will work and the %f tells the command to report the file name. There are other events you can wait for as well, of course.

The other command line, inotifywatch, also outputs file change events on its output, but I won't talk about it any further. If you think you need that capability, you can read the man page.

The script is still less than ideal, though. Presumably a system might have lots of different directories it wants to monitor. You really don't want to repeat this script, or a variation of it, for each case.

There is another program for that, called incron (you will almost surely have to install this one). The incron program is like cron but instead of time-based events, the events are based on file notifications. Once you install it, you will probably have to change /etc/incron.allow and /etc/incron.deny if you want to actually use it, especially as a normal user.

Since I mentioned electronic books, I'll show you how I use incron to automatically have Calibre (the book reader software) catalog incoming electronic books. You use the command "incrontab -e" to edit your incron table. The format is very picky (it wants spaces, not tabs, for example). Here's a line from mine (this is all one line, but will probably wrap when you read it):

/home/alw/Downloads/books IN_CLOSE_WRITE,IN_MOVED_TO /home/alw/bin/autocalibre $@/$#

This runs the autocalibre script with the full path name any time incron sees a file closed for writing or moved to the download directory (you can guess that incron uses inotify, and it does).

If you are really curious, here's the autocalibre script:

#!/bin/bash
# Automatically add a file in a book directory
# First we want to make sure it really is a kind we want
if [ "$1" == "" ]
then
  exit 1
fi
EXT=${1##*.}
if [ "$EXT" == "epub" -o "$EXT" == "EPUB" -o "$EXT" == "pdf" -o "$EXT" == "PDF" ]
then 
    calibredb add "$1"
fi
exit 0

This is the most elegant solution yet. A system program does all the waiting and our script only runs when necessary. You can do a lot with these tools, and not just in the embedded space. How are you going to use them? Leave a comment and share your ideas.

Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 


Video