Real-time CORBA, Part 2: Applications and Priorities

The Real-time CORBA specification enhances regular CORBA by adding capabilities that control process, communication, and memory resources. These capabilities enable standard COTS middleware to improve the determinism of DRE applications by bounding their priority inversions and managing their system resources more predictably end-to-end. This column shows how to program the Real-time CORBA features that provide portable priority mapping and end-to-end priority preservation.


January 01, 2002
URL:http://www.drdobbs.com/real-time-corba-part-2-applications-and/184403812

January 2002/Object Interconnections/Figure 1

Figure 1: Key elements for a planetary mapping system

January 2002/Object Interconnections/Figure 2

Figure 2: Relationships among CORBA clients and servers for planetary mapping application

January 2002/Object Interconnections/Figure 3

Figure 3: Mapping CORBA priorities onto two ORB end systems with different native priority ranges

January 2002/Object Interconnections/Figure 4

Figure 4: A single priority is encoded into the object reference, which is then published to the client as a tagged component in an object reference

January 2002/Object Interconnections/Figure 5

Figure 5: An invocation from a client on one ORB end system to a server on another results in an invocation on an intervening ORB end system, each with different native priority ranges

January 2002 C++ Experts Forum/Object Interconnections

Object Interconnections: Real-time CORBA, Part 2: Applications and Priorities

Douglas C. Schmidt and Steve Vinoski

Introduction

This is the second in a series of columns that illustrate how CORBA is evolving to support DRE (distributed real-time and embedded) applications. DRE applications constitute an important class of distributed systems where predictability, efficiency, and fine-grained control over system resources are essential for success. Historically, these types of applications have not been well supported by CORBA or any other mainstream distributed-object computing middleware. To address this problem, the OMG adopted the Real-time CORBA specification into CORBA 2.4 in October 2000 [1]. This specification included standard interfaces and policies designed to allow DRE applications to configure and control the following system resources:

Our first column in this series [2] motivated the need for the capabilities offered by Real-time CORBA. This column shows C++ code examples that illustrate how to program the Real-time CORBA priority mechanisms. Subsequent columns will show how to program many other Real-time CORBA features.

Time Application Overview

We'll focus on a planetary mapping system to illustrate key Real-time CORBA capabilities. This application is based on various semi-autonomous vehicle use cases, such as the Mars Sojourner or unmanned ground vehicle robots used to clean-up toxic waste spills. The key elements in this application are shown in Figure 1.

In this system, semi-autonomous drones are involved in a distributed computation to map a planetary surface. The drones explore the planet's surface and periodically report its properties(e.g., its color, texture, and elevation). The data collected by drones is aggregated and delivered to mission operators, who can update the mission dynamically by redirecting drones and their actions. To keep the system affordable, drones aren't particularly "smart." They can therefore fall off the edges of planetary surfaces, such as boulders or cliffs, if they aren't stopped by an operator that coordinates their actions (e.g., by ordering them to a new position).

Figure 2 illustrates the relationships among the various CORBA clients and servers in our planetary mapping application.

Mission operators define high-level exploration goals for the drones via a Base_Station object, which provides set points for Controller objects. Controller objects control the drones remotely using Drone objects, which reside on the drone vehicles. Drone objects expose operations for controlling and monitoring individual drone behavior. Each drone sends information obtained from its sensors back to the Base_Station via its Controller object.

The IDL interfaces for these objects are described next. We first show the Drone interface:

interface Drone {
  void turn (in float degrees);
  void speed (in short mph);
  void reset_odometer ();
  short odometer ();
  // ...
};

Each drone talks to one controller, whose interface is shown next:

interface Controller {
  void edge_alarm ();
  void battery_low ();
  // ...
};

When a drone detects an edge, it sends high priority edge_alarm() requisitions to its controller, which is responsible for taking corrective action. The Base_Station interface is shown below:

interface Base_Station {
  exception Lack_Resources {};
  Controller new_controller (in string name)
    raises (Lack_Resources);
  void set_new_target (in float x, in float y,
    in float w, in float h);
  // ...
};

A Base_Station plays several roles:

When implementing the Base_Station, Controller, and Drone interfaces in our planetary mapping application, we must address the following design challenges associated with controlling DRE system resources and providing adequate end-to-end real-time QoS (quality of service):

System ResourcesQoS-Related Design Challenges
CPU Utilization and Priorities
  1. Ensuring portable ORB end-system priorities
  2. Preserving priorities end-to-end
  3. Simplify scheduling of end-to-end priorities
Threads and Synchronizers
  1. Supporting thread pools effectively
  2. Prevent exhaustion of threads by low priority requests
  3. Prevent bursts or long-running requests from exhausting maximum number of static and dynamic threads
  4. Buffering client requests
  5. Synchronizing objects correctly
Network Protocols and Connections
  1. Configuring custom protocols to obtain adequate QoS
  2. Controlling network and end-system resources to minimize priority inversion
  3. Avoiding dynamic connections since jitter can be detrimental to time-critical applications
  4. Minimizing priority inversions by ensuring applications don't share a connection between multiple threads running at different priorities

These QoS-related design challenges are similar to those of other DRE applications, such as avionics mission computing, telecom call processing, and hot rolling mill control systems. In particular, strict control over the priorities of threads that share processor resources is essential for many real-time applications. Earlier versions of CORBA provided no standard way for clients to indicate the relative priorities of their requests to ORB end systems. This feature is necessary, however, to minimize end-to-end priority inversion, as well as to bound latency and jitter for applications with deterministic real-time QoS requirements. The Real-time CORBA specification therefore defines platform-independent features to address the following design challenges associated with controlling the priority of operation invocations:

Design Challenge Example from Application Real-time CORBA Feature
Ensuring portable ORB priorities ORB must map heterogeneous native OS host thread priorities consistently from client to server Use standard Real-time CORBA priority mapping interfaces
Preserving priorities end-to-end ORB must ensure that requests don't run at the wrong priority on the server(e.g., major problems can occur if edge_alarm() operations are processed too late) Use the Real-time CORBA CLIENT_PROPAGATED or SERVER_DECLARED priority models

The remainder of this column illustrates how to apply the Real-time CORBA features outlined in the table above to address key thread-priority design challenges that arise in our planetary mapping application.

Ensuring Portable ORB Priorities

To maximize flexibility and select the best solution for different tasks, our planetary mapping application uses different types of hardware and software. As this heterogeneous computing infrastructure evolves over the lifecycle of the system, a recurring challenge is to ensure that the priorities of CORBA operations map correctly and efficiently onto the native OS host thread priorities consistently end-to-end. In earlier versions of CORBA, there was no way to designate priorities, much less map them portably across diverse real-time OS platforms.

Fortunately, the Real-time CORBA specification defines a priority mapping feature that supports heterogeneous real-time OS platforms. This feature defines two types of priorities — CORBA and native — to handle OS heterogeneity. Each one-way or two-way CORBA operation can be assigned a CORBA priority, which is an OS-independent value ranging between 0 and 32,767. Real-time CORBA allows each ORB end system along the path from source client to sink server to map these CORBA priorities onto native priorities that are appropriate for the underlying OS threading mechanisms. Applications can customize this mapping differently on different ORB end systems. For example, Figure 3 illustrates how CORBA priorities can be mapped onto two different ORB end systems with different native priority ranges.

The following C++ code example illustrates how to extend the standard RTCORBA::PriorityMapping class to define a pair of hook methods that convert to and from CORBA priorities to native priorities. This example is targeted for the LynxOS platform (e.g., the to_native method only uses native priorities in the range [128-255), which is the top half of LynxOS thread priorities).

class LynxOS_Priority_Mapping : 
  public RTCORBA::PriorityMapping {
  CORBA::Boolean to_native 
   (RTCORBA::Priority corba_prio,
    RTCORBA::NativePriority &native_prio) {
    native_prio = 128 + (corba_prio / 256);
    return true;
  }

  CORBA::Boolean to_corba 
   (RTCORBA::NativePriority native_prio,
    RTCORBA::Priority &corba_prio) {
    if (native_prio < 128) return false;

    corba_prio = (native_prio - 128) * 256;
    return true;
  }
};

The Real-time CORBA priority mapping feature enables DRE developers to program their applications with minimal concern for low-level details of the thread priorities in their underlying ORB end systems. In turn, this feature makes it easier to configure the various ORB instances in their DRE systems to take maximal advantage of the underlying OS, network, and hardware resources. It's important to recognize, however, that Real-time CORBA's priority mapping feature can't work miracles (i.e., no ORB middleware can magically imbue a non-real-time OS or communication infrastructure with completely deterministic behavior).

There are also some deficiencies with the Real-time CORBA priority mapping feature. In particular, the specification doesn't define a standard way to install a PriorityMapping object. The official rationale for not standardizing this installation process is to maximize ORB implementer options (e.g., to install the mapping at link time or at run time, whichever is most efficient for a particular use case). The following C++ code illustrates how to install a PriorityMapping using the TAO ORB [3], which is an open-source implementation of Real-time CORBA. TAO implements the priority mapping feature via a locality-constrained object called the PriorityMappingManager. We first show the standard CORBA code, which starts by obtaining a reference to an ORB and creating an instance of the LynxOS priority mapping object:

CORBA::ORB_var orb = 
  CORBA::ORB_init (argc, argv);
RTCORBA::PriorityMapping *lynxos_mapping = 
  new LynxOS_Priority_Mapping;

We next show the TAO-specific code. We use the standard CORBA resolve_initial_references feature to get an object reference to TAO's PriorityMappingManager and then narrow this to a reference to the TAO-specific PriorityMappingManager_var helper class:

CORBA::Object_var obj = 
  orb->resolve_initial_references ("PriorityMappingManager");
TAO::PriorityMappingManager_var manager = 
TAO::PriorityMappingManager::_narrow (obj);

Finally, we install the new priority mapping for LynxOS:

manager->mapping (lynxos_mapping);

Although the TAO-specific code that installs the priority mapping can be localized in a small section of the application, it would be nice if Real-time CORBA standardized this procedure because it would simplify portability between ORBs.

Preserving Priorities End-to-End

Assuming that we've installed the appropriate PriorityMappings throughout our planetary mapping system, our next challenge is to ensure that operations run at the right priority within their servers. For example, it's important that edge_alarm() operations run at a higher priority than other operations since otherwise drones may get stuck in awkward situations. To preserve priorities end-to-end, the Real-time CORBA specification defines a PriorityModel policy with two values, SERVER_DECLARED and CLIENT_PROPAGATED, which we describe below.

The SERVER_DECLARED model allows a server to dictate the priority at which an invocation made on a particular object will execute. In this model, a server designates the priority a priori when the object is activated. A single priority is encoded into the object reference, which is then published to the client as a tagged component in an object reference, as shown in Figure 4.

Although the server declares the priority, the client ORB is aware of the selected priority model policy and can use this information internally to ensure that invocations access client-side resources at the appropriate priority.

To illustrate the SERVER_DECLARED model, we'll show how to create and activate a Base_Station object. Since its operations are not time-critical, they run at lower priority than Controller::edge_alarm() operations. We start by using the standard CORBA::ORB_init factory operation to obtain an object reference to the ORB:

CORBA::ORB_var orb = CORBA::ORB_init (argc, argv);
CORBA::Object_var obj =
  orb->resolve_initial_references ("RTORB");

We then use obj to obtain an object reference to the Real-time CORBA ORB:

RTCORBA::RTORB_var rt_orb =
  RTCORBA::RTORB::_narrow (obj);

We need to obtain a real-time ORB in order to access certain operations that aren't available to non-real-time ORBs. Assuming that the RTCORBA::RTORB::_narrow operation succeeds, we can henceforth use Real-time CORBA features. The steps above are an example of the Extension Interface pattern [4], which allows multiple interfaces to be exported by a component to prevent bloating of interfaces and breaking of client code when developers extend or modify the functionality of the component.

After we initialize the rt_orb, we create a policy list initialized with the SERVER_DECLARED policy by calling an operation that's specific to Real-time CORBA.

CORBA::PolicyList policies (1);
policies.length (1);
policies[0] = rt_orb->create_priority_model_policy 
  (RTCORBA::SERVER_DECLARED, LOW_PRIORITY);

We then use this PolicyList to create a POA that contains Base_Station objects:

PortableServer::POA_var base_station_poa = 
  root_poa->create_POA ("Base_Station_POA",
    PortableServer::POAManager::_nil (),
    policies);

The LOW_PRIORITY argument to create_priority_model_policy is the default CORBA priority of objects that are activated in "Base_Station_POA". We next activate a Base_Station servant in the base_station_poa:

Base_Station_i *base_station = new Base_Station_i;
base_station_poa->activate_object (base_station);

We can now export the object reference for base_station to clients using a Naming Service or some other type of location discovery service or factory operation.

By default, SERVER_DECLARED objects inherit the priority of the POA in which they're created, as shown above. This behavior can be controlled by using the activate_object_with_priority operation defined in the RTPortableServer::POA interface. For example, we can activate another_server at ANOTHER_PRIORITY, as shown below:

RTPortableServer::POA_var rt_poa =  
  RTPortableServer::POA::_narrow (base_station_poa);
rt_poa->activate_object_with_priority 
  (another_servant, ANOTHER_PRIORITY);

Note that Real-time CORBA once again uses the Extension Interface pattern to obtain the real-time POA from an object reference to a non-real-time POA.

Although the SERVER_DECLARED model is useful for certain types of real-time applications, it's not suited for others. For instance, one way a server can avoid priority inversions is to process incoming requests at a priority equivalent to the client thread that invoked the operation originally. To support this use case, Real-time CORBA also defines a CLIENT_PROPAGATED model that allows clients to invoke operations whose priorities must be honored by servers. In this model, each invocation carries the CORBA priority of the operation in the service context list that's included as part of its GIOP request. Each ORB along the path between the client and server maps this end-to-end CORBA priority to a native OS priority and processes the request at this priority. If the client invokes a two-way operation, its CORBA priority will determine the priority of the reply.

Figure 5 depicts the case where an invocation from a client on one ORB end system to a server on another ORB end system results in an invocation on an intervening ORB end system, each running real-time operating systems with different native priority ranges for their threads.

Since we're using the CLIENT_PROPAGATED model, the CORBA priority of the client is transmitted along with the request. Each server along the invocation path maps the client's CORBA priority to a native priority that's appropriate for its host platform and end-to-end global priority. For example, on QNX the global CORBA priority could be mapped to a native OS priority of 16, whereas on LynxOS the same global CORBA priority could be mapped to a real-time thread with a priority of 128, and on Solaris this CORBA priority could be mapped to a priority of 136.

In our planetary mapping system, drones invoke critical operations on controllers in the base station. The Controller::edge_alarm() operation runs at the highest priority in the system, whereas the Controller::battery_low() operation runs at a lower priority in the system. The CLIENT_PROPAGATED policy is set on the server and exported to the client along with an object reference, as shown below. Assuming we've already obtained a reference to a real-time ORB as shown earlier, we create a policy list initialized with the CLIENT_PROPAGATED policy using an operation that's specific to Real-time CORBA:

CORBA::PolicyList policies (1); 
policies.length (1);
policies[0] = rt_orb->create_priority_model_policy 
  (RTCORBA::CLIENT_PROPAGATED, DEFAULT_PRIORITY);

DEFAULT_PRIORITY is the CORBA priority used to process requests from non-Real-time CORBA clients. We use the policies PolicyList to create a POA that contains Controller objects:

PortableServer::POA_var controller_poa = 
  root_poa->create_POA ("Controller_POA",
    PortableServer::POAManager::_nil (),
    policies);

Next, we activate a Controller servant in the controller_poa:

Controller_i *controller = new Controller_i;
controller_poa->activate_object (controller);

As with the base_station example above, we can now export the object reference for controller using a Naming Service or a similar location discovery service.

Although servers determine whether an object is created with the CLIENT_PROPAGATED priority model, clients ultimately determine the CORBA priority at which operations on an object are invoked. To select the CORBA priority at which an operation is invoked, a client uses its thread-specific RTCurrent object to change the priority of a thread explicitly. To illustrate this in C++, we first get the real-time ORB's RTCurrent object:

obj = orb->resolve_initial_references ("RTCurrent");
RTCORBA::Current_var rt_current =
  RTCORBA::Current::_narrow (obj);

We then change the current thread's CORBA priority:

rt_current->the_priority (VERY_HIGH_PRIORITY);

This call will use the to_native hook method described earlier to map the VERY_HIGH_PRIORITY CORBA priority to the appropriate native OS thread's priority and set the thread to run at that priority. The RTCurrent object can also be used to query the current thread's priority. Now that we've set the right priority, we'll invoke the edge_alarm() operation in the client thread running at the VERY_HIGH_PRIORITY native thread priority:

controller->edge_alarm ();

Since the controller object reference was created in a server POA configured with the CLIENT_ PROPAGATED policy, its CORBA priority will be sent along with the edge_alarm() request.

Each server along the end-to-end operation path will use the Real-time CORBA priority mapping mechanism to convert the CORBA priority into the corresponding OS native priority and process the operation at that priority. By applying the appropriate real-time analysis tools, such as TimeWiz from TimeSys or RAPID RMA from Tri-pacific, it's possible to determine the feasibility of scheduling all the time-critical operations in our planetary mapping application.

Concluding Remarks

The Real-time CORBA specification enhances regular CORBA by adding capabilities that control process, communication, and memory resources. These capabilities enable standard COTS middleware to improve the determinism of DRE applications by bounding their priority inversions and managing their system resources more predictably end-to-end. This column illustrates how to program the Real-time CORBA features that provide portable priority mapping and end-to-end priority preservation. Our next column will explore Real-time CORBA features that support threading and synchronization.

A PowerPoint tutorial version of our columns on the Real-time CORBA specification can be downloaded at <http://www.cs.wustl.edu/~schmidt/TAO/RT-CORBA.ppt>. We appreciate the help of Irfan Pyarali, Carlos O'Ryan, and Angelo Corsaro for co-authoring this tutorial and providing comments on this column. If you have comments, questions, or suggestions regarding Real-time CORBA or our columns, please let us know at [email protected].

References

[1] Object Management Group. "The Common Object Request Broker: Architecture and Specification Revision 2.4," OMG Technical Document formal/01-02-33, October 2000, <www.omg.org/cgi-bin/doc?formal/01-02-33>.

[2] D.C. Schmidt and S. Vinoski. "Object Interconnections: Real-time CORBA, Part 1: Motivation and Overview," C/C++ Users Journal C++ Experts Forum, December 2001, <www.cuj.com/experts/1912/vinoski.htm>.

[3] The ACE ORB (TAO), <www.cs.wustl.edu/~schmidt/TAO.html>.

[4] D. Schmidt, M. Stal, H. Rohnert, and F. Buschmann. Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects(Wiley and Sons, 2000), <www.cs.wustl.edu/~schmidt/POSA/>.

[5] Richard E. Schantz and Douglas C. Schmidt "Middleware for Distributed Systems: Evolving the Common Structure for Network-centric Applications," Encyclopedia of Software Engineering (Wiley and Sons, 2002), <www.cs.wustl.edu/~schmidt/PDF/middleware-chapter.pdf>.

Steve Vinoski is chief architect and vice president of Platform Technologies for IONA Technologies and is also an IONA Fellow. A frequent speaker at technical conferences, he has been giving CORBA tutorials around the globe since 1993. Steve helped put together several important OMG specifications, including CORBA 1.2, 2.0, 2.2, and 2.3; the OMG IDL C++ Language Mapping; the ORB Portability Specification; and the Objects By Value Specification. In 1996, he was a charter member of the OMG Architecture Board. He is currently the chair of the OMG IDL C++ Mapping Revision Task Force. He and Michi Henning are the authors of Advanced CORBA Programming with C++, published in January 1999 by Addison Wesley Longman.

Doug Schmidt is an associate professor member at the University of California, Irvine. His research focuses on patterns, optimization principles, and empirical analyses of object-oriented techniques that facilitate the development of high-performance, real-time distributed object computing middleware on parallel processing platforms running over high-speed networks and embedded system interconnects. He is the lead author of the book Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects, published in 2000 by Wiley and Sons. He can be contacted at [email protected].

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.