Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

Parallel

Inside Real-Time Linux


Mar00: Inside Real-Time Linux

Jerry writes embedded software, primarily for medical devices. He can be contacted at [email protected].


Consider the choices you face when designing a medium-to-high volume system having some real-time requirements. Suppose the application has some database management and networking requirements, and also requires a nice GUI. It must also meet some minimal real-time requirements. Until recently, you had two choices. First, you could use a high-end real-time operating system (RTOS). These have high initial costs and royalty fees. Their technology also tends to be well behind that of standard desktop operating systems in all areas except their real-time capabilities. Thus, you can expect to pay a higher price to produce each copy of the system, as compared to a nonreal-time system -- not to mention a longer time-to-market, because the system will be developed with older, less capable networking and GUI facilities. Your second choice is to develop with a desktop operating system, and implement the real-time features with special-purpose hardware. This option probably results in a lower recurring cost, but often requires a full hardware design cycle and expertise that an otherwise software-only solution would not need.

In view of the high cost of implementing the needed minimal real-time capability then, you might opt for a method of grafting real-time functionality onto a standard desktop OS. Several such tools for Microsoft Windows NT are available, all of which share a common model in which real-time tasks run in an environment underneath NT. The normal Win32 API is not available to these tasks: They are provided by a proprietary API, including functions for communicating with NT processes. Thus, an application is divided into two parts, real time and nonreal time. The real-time part meets real-time constraints, but has a limited API. The nonreal-time part has no API limitations, and draws on all of the facilities provided by NT, but cannot be relied upon to provide a real-time response. This model works well for most real-time applications, as most designers separate applications into real-time and nonreal-time parts anyway, even when using an RTOS. For example, a data-acquisition application might have a real-time task that performs A/D conversions at fixed intervals, passing them to a nonreal-time process via some queueing mechanism. The nonreal-time process then uses OS facilities to save the data to disk, display it graphically, or send it to another machine over a network.

One disadvantage of using NT-based RTOS tools is their generally high cost -- in the same range as the high-end RTOS. On the other hand, Real-Time Linux (http://www.rtlinux.org/), a project started by Victor Yodaiken at New Mexico Tech, uses the same model as the NT-based systems. However, like Linux itself, Real-Time Linux (RTLinux) is freely available. The design of RTLinux is ingeniously simple. As you probably know, a conventional desktop operating-system kernel (such as Linux) can disable interrupts for an extended period of time for any number of reasons. During this time, the scheduler does not run, so user processes are stalled. A stalled real-time process might miss its deadline, which is unacceptable in a real-time system. A conventional RTOS avoids this difficulty through the use of careful kernel design. The kernel has to be designed from the beginning so that interrupts are disabled for only the briefest periods -- a few microseconds at the most. Such a design can be difficult. In contrast, conventional operating systems often disable interrupts for many milliseconds. And the Linux kernel is a complex design -- redesigning it for short interrupt times would be impractical. Thus, it would seem that a real-time version of Linux would not be feasible.

Theory of Operation

Operating systems must disable interrupts for brief periods so that important internal structures cannot be corrupted by intervening system calls. But no such danger exists if the interrupting task is restricted from making these troublesome system calls. With a restricted API, a real-time task can be allowed to interrupt at any time with no danger of OS corruption. Hence the division of an application into real-time and nonreal-time parts -- the real-time tasks can interrupt the kernel at any time, but have a restricted API so that they cannot corrupt kernel data structures. Meanwhile, the nonreal-time processes have no API restrictions, but cannot interrupt the kernel.

RTLinux implements this scheme by exploiting the best-known characteristic of Linux -- its open-source programming model. The Linux kernel has two macros for disabling and enabling interrupts -- CLI and STI, named after the corresponding instructions on the x86 platform on which Linux was originally implemented. In unmodified Linux, these macros simply disable and enable interrupts. But Real-Time Linux redefines them so that they simply set and clear a flag. When the flag is clear, interrupts are allowed to proceed. When the flag is set, interrupts are first routed to some RTLinux code that determines whether the interrupting entity is a Linux process or a real-time task. Real-time tasks are allowed to run, while the execution of Linux processes is deferred until a CLI is executed.

In this way, the core of RTLinux is implemented with minimal modifications to the Linux kernel source. The CLI and STI macros are redefined, with some fairly simple extra functionality added to them. The macro redefinition patch is applied to the kernel source, then the kernel is built from this source, resulting in a kernel with real-time capability as well as normal Linux functionality. Besides simple implementation, this design has the advantage that it is nearly immune from being broken by new versions of the Linux kernel. Although numerous kernel versions have appeared since the introduction of RTLinux, few modifications to RTLinux have been necessitated by these version changes.

Real-Time Linux API

The only other component of RTLinux is its API, which provides facilities for the control of the real-time characteristics of a task, as well as for communication between real-time tasks and nonreal-time Linux processes. These facilities are intentionally quite minimal, reflecting the RTLinux designers' views on how real-time operating systems should be implemented. Thus, RTOS users may look in vain for some facilities to which they are accustomed. But all essential real-time functionality is present, and the architecture is extensible, so anyone who would like extra functionality in the API can add it.

RTLinux takes advantage of the installable module capability of Linux. A user can install a prebuilt module into the kernel space; this code executes with kernel privileges. This capability was designed with device drivers in mind -- prior to the addition of installable modules, support for hardware had to be compiled into the kernel. Users who upgraded to a new modem or network card had to build a new kernel from the sources. The addition of installable modules allows hardware support to be added or subtracted dynamically; this capability is necessary for any OS to attain widespread user acceptance. In any case, RTLinux real-time tasks are installed in the same way. Thus, they have kernel privileges, and can crash the system if not implemented correctly. The advantages of having real-time tasks running in kernel space are several, including:

  • The scheduler can perform a context switch quickly if no change in processor context is necessary.
  • No danger of paging the real-time tasks out of memory exists.

  • Because the tasks execute with kernel privileges, access to I/O hardware is direct and fast.

The API can be divided into four groups:

  • Task creation and deletion.
  • Task timing and control.

  • Communication with Linux processes.

  • Interrupt handling.

For task creation, you use rt_task_create(); to remove a task, use rt_task_ delete(). The focus of RTLinux on data acquisition is reflected in rt_task_make_periodic() and rt_task_wait(). rt_task_ make_periodic() marks a task for periodic execution; it is then scheduled for execution at fixed intervals. When the task has completed its execution for a particular interval, it then executes rt_task_ wait(). This causes it to block until the next time interval. This model is ideal for fixed-interval applications. A task can be blocked with rt_task_suspend(); it can correspondingly be unblocked with rt_task_wakeup(). Finally, an application using floating-point operations must make an rt_use_fp() call. This results in slower task switching, so floating-point code should be avoided when possible.

The primary means of communication between real-time tasks and nonreal-time Linux processes is the RTLinux FIFO. Arbitrary data can be transferred between the two domains. rtf_create_fifo() and rtf_destroy() are used for creation or removal of a FIFO. rtf_put() and rtf_get() insert and remove data to and from a FIFO. A FIFO can be resized with rtf_resize(). And a handler can be attached to a FIFO via rtf_create_handler(). This is a function that is called whenever data is placed on or removed from a FIFO.

RTLinux can also capture hardware interrupts, allowing users to specify a function to be called when an interrupt occurs. The functions request_RTirq() and free_RTirq() are used for this purpose. Thus, a sort of RTLinux device driver can be implemented, with hardware events causing software actions to be executed. (For an excellent example of such a driver, see the rt_com module by Jens Michaelsen and Jochen Küpper in the RTLinux distribution. This module permits sending and receiving data over a serial port from within RTLinux tasks.)

A Data-Acquisition Example

Listings One and Two constitute an RTLinux program illustrating the structure you might use for a data-acquisition application. This application illustrates the simplicity of the RTLinux programming model. Listing One is the RTLinux portion. Like all RTLinux apps, Listing One (producer.c) conforms to the installable Linux module model. Thus, it has a function called init_module(), which is called by Linux when the module is installed (usually with the insmod command); and cleanup_module(), which is called when the module is unloaded (with the rmmod command). Note the simplicity of init_ module() -- it calls:

  • rtf_create() to create a FIFO for sending the acquired data to the Linux process.
  • rt_task_init() to start the real-time task.

  • rt_task_make_periodic() to set up that task for periodic scheduling.

This example schedules producer() for execution once per second; of course, most real data acquisition applications would acquire data faster.

Producer() is the core of the real-time task. It simply waits to be scheduled for execution (via rt_task_wait()), then places some data on the FIFO for consumption by the Linux process performing all of this in an infinite loop. A real application could use this structure to perform A/D conversions at fixed intervals, sending the acquired data to the Linux process for handling.

Listing Two (consumer.c) is the code for the Linux process. Like producer.c, it has a simple design. RTLinux FIFOs are visible on the Linux side as character devices; they are accessed via the normal UNIX-type file and device I/O function calls. Thus, the device is opened via the open() function, the program can block while waiting for input from the FIFO via select(), and when data is available it can be read using read(). This programming model is familiar to all experienced UNIX programmers, so writing the Linux side of an RTLinux application is quite straightforward.

This example, built using Red Hat Linux 6.0 and Real-Time Linux beta 15, is available electronically from DDJ (see "Resource Center," page 7) and from http:// tesla.stereotaxis.com/rtlinux/. The example is missing much that a real RTLinux app would need, but still illustrates the core RTLinux functionality well. A number of other more complete examples are available from the Real-Time Linux web site (http://www.rtlinux.org/); see, for example, the Frank program, which is similar in concept to this one, but provides a more realistic handshaking model, with multiple FIFOs.

RTLinux Future Directions

The RTLinux core provides all of the capabilities one needs to implement a real-time application under Linux. And for certain applications -- in particular those for data acquisition -- its API is sufficiently easy to use, so that you can likely use it out of the box to meet your needs. It is clear, however, that for more complex applications a cleaner API with higher level functionality would result in greater acceptance of RTLinux by programmers. To this end, several groups and individuals have contributed enhancements ranging from improved schedulers to improved documentation.

Fred Proctor and others devised a scheme for sharing memory between RTLinux tasks and Linux processes using the Linux mmap() function. This provides an additional communication method, providing an alternative to the standard RTLinux FIFO. (For more information, see "Linux, Real-Time Linux, and IPC," by Frederick M. Proctor, DDJ, November 1999, and http://www.isd.cme .nist.gov/projects/emc/shmem.html/).

Victor Yodaiken (the RTLinux founder at New Mexico Tech) and others, including Michael Barabanov, the primary implementor of the first versions of RTLinux, have started refurbishing the RTLinux API, with the goals of supporting more real-time functionality and using an industry-standard API. They chose the POSIX Real-Time Applications Support draft standard (IEEE Std. P1003.13). The purpose was to choose an API that would support multiple processors, portability among processor architectures, and I/O. These factors are likely to be increasingly important as RTLinux matures. Rather than simply extend the RTLinux API to support these features, the designers chose to implement the POSIX standard API. Such an approach will, of course, also help in porting applications to and from other POSIX-compliant systems. Most of the current RTLinux development is focused on this migration; as of this writing, Version 2 of RTLinux has been released.

Paolo Mantegazzo and others have created an alternative package called "Real Time Application Interface" (http://www .aero.polimi.it/projects/rtai/). Originally an RTLinux variant, it was started because Paolo needed a timer with less jitter than RTLinux exhibited. But lately, RTAI has become a much more ambitious project, with an implementation approach that is somewhat different from that of RTLinux. Ironically, its API remains quite close to that of RTLinux -- it has a compatibility API like that of RTLinux Version 1, and (also like RTLinux) is moving toward a POSIX-compliant API. So writing applications for the two systems is a quite similar experience. Many participants of both projects would like to see a reconvergence of RTLinux and RTAI, as the projects have nearly identical aims. A convergence might result in less duplicated development effort, but as of this writing they stand as competing systems.

On balance, Real-Time Linux is a remarkably powerful yet easy-to-use extension to the Linux operating system that can be used by system designers having hard real-time requirements to be met. Its power is clear -- it can meet hard real-time requirements while still permitting other parts of the application to use the facilities that Linux provides. Its simplicity lies in its API, which consists of less than 20 easily understood calls, and in its design and structure. In contrast to proprietary RTOS products, RTLinux is open, and can be examined, understood, and modified by an experienced programmer with a reasonable investment of time.

DDJ

Listing One

/* producer.c -- continually sends data at fixed intervals to a consumer of 
 * data. This structure can be used for a data-acquisition application.
 * Written by Jerry Epplin
 */

#ifndef MODULE
#define MODULE
#endif
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/version.h>
#include <linux/errno.h>
//#include <linux/cons.h>

#include <rtl_sched.h>
#include <rtl_fifo.h>

RT_TASK ProducerTask;  /* handle for the task */

/* data to be sent to the Linux process */
static char *Data[] = { "This ", "data ", "is ", "sent ", "to ", 
                                                   "the ", "consumer.\n" };
/* number of data elements */
static const int NumData = sizeof(Data) / sizeof(char *);

static const int FIFO_NUM = 1;  /* FIFO handle */

static const int MSEC_INTVL = 100;  /* msecs between producer events */

/* Producer() -- Task which produces data, sending
 * it to the Linux process for consumption.
 */
static void Producer(int t)
{
  int idx, ret;
  /* Keep sending data to the consumer at */
  /* periodic intervals. */
  for (idx=0 ; ; idx=(idx+1)%NumData)
  {
    /* wait until this task is scheduled */
    if ((ret = rt_task_wait()) != 0)
      printk("Producer() -- rt_task_wait() failed with %d.\n", ret);
    /* send data to the consumer process */
    if ((ret = rtf_put(FIFO_NUM, Data[idx], strlen(Data[idx]))) != 0)
      printk("Producer() -- rtf_put() failed with %d.\n", ret);
   }
}
/* init_module() -- Called by kernel when module is
 * loaded with 'insmod'.  Creates the FIFO and starts the task.
 */
int init_module(void)
{
  int ret;
  /* create the FIFO */
  if ((ret = rtf_create(FIFO_NUM, 2048)) != 0)
    printk("init_module() -- rtf_create() failed with %d.\n", ret);

  /* start the task */
  if ((ret = rt_task_init(&ProducerTask, Producer, 0, 3000, 4)) != 0)
    printk("init_module() -- rt_task_init() failed with %d.\n", ret);

  /* set up the task for periodic scheduling */
  if ((ret = rt_task_make_periodic(&ProducerTask, rt_get_time(), 
                                                 1000*MSEC_INTVL)) != 0)
    printk("init_module() -- rt_task_make_periodic() failed with %d.\n", ret);

  return 0;
}
/* cleanup_module() -- Called by kernel when module
 * is unloaded with 'rmmod'.  Removes the FIFO and deletes the task.
 */
void cleanup_module(void)
{
  rtf_destroy(FIFO_NUM);
  rt_task_delete(&ProducerTask);
}

Back to Article

Listing Two

/* consumer.c -- consumer of data produced by producer
 * Written by Jerry Epplin
 */

#include <stdio.h>
#include <errno.h>
#include <sys/time.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <rtl_fifo.h>
#include <rtl_time.h>

#define BUFSIZE 70

char buf[BUFSIZE];

int main()
{
  fd_set rfds;
  int i, n, ret;
  int fd0;
  
  /* open the RT-FIFO device */
  if ((fd0 = open("/dev/rtf1", O_RDONLY)) < 0) {
    fprintf(stderr, "Error opening /dev/rtf1\n");
    exit(1);
  }
  /* show the data for a while */
  for (i=0; i<21 ; ++i)
  {
    FD_ZERO(&rfds);
    FD_SET(fd0, &rfds);

    /* wait until data is present */
    ret = select(FD_SETSIZE, &rfds, NULL, NULL, NULL);
    if (ret > 0)
    {
      if (FD_ISSET(fd0, &rfds))
      {
        /* read the data out and display it */
        n = read(fd0, buf, BUFSIZE - 1);
        buf[n] = '\0';
        printf("%s", buf);
      }
    }
    else
      printf("select() failed with %d.\n", ret);
  }
  return 0;
}




Back to Article


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.