OMG's Data Distribution Service Standard

The OMG Data Distribution Service (DDS) Standard specifies a mandatory API for data-centric publish-subscribe


November 20, 2006
URL:http://www.drdobbs.com/embedded-systems/omgs-data-distribution-service-standard/194900002

With the advent of ubiquitous networking and cheap processing, data distribution is emerging as a key paradigm for disseminating information in real-time and embedded systems. The applications are characterized by complex dataflows for example one-to-many, many-to-one, many-to-many dataflows with differing update rates, reliability, and bandwidth requirements; real-time performance which may be defined in terms of tight latency, throughput, or bandwidth requirements; fault-tolerance which may be defined in terms of not having a single point of failure or supporting self-healing communication, or having redundant communication paths; dynamic configuration changes where participants may join or leave arbitrarily; and the ability to scale with network size. The applications typically reflect the fluid and ever-changing nature of the real world.

The need for data-distribution is so fundamental that domains as diverse as command and control systems, traffic monitoring, financial data-distribution, combat systems, industrial sensing all use it as a core technology. The lack of established standards has led to the development of ad-hoc custom one-off solutions with sub-optimal performance and robustness that end up being very expensive to maintain. Commercial implementations of publish-subscribe data-distribution middleware have existed and have been deployed with success. However each vendor provided a different API and slightly different semantics for equivalent concepts.

This situation was similar to the one prior to the adoption of UML. Successful modeling tools existed but widespread acceptance, availability of trained developers, and broad knowledge exchange did not come until the UML standard consolidated terminology and notation. The recent adoption of the Data-Distribution Service (DDS) for Real-Time Systems specification by the Object Management Group (OMG) will do for data-distribution what UML did for modeling technologies.

The OMG Data Distribution Service (DDS) Standard specifies a mandatory API for data-centric publish-subscribe (DCPS) communication, and an optional data local reconstruction layer (DLRL). The DDS specification is programming language agnostic; it is specified via a platform independent model in UML, and a platform-specific model in IDL, which can be mapped to programming language APIs via standard mappings. In the rest of this article, we highlight the key aspects of the OMG DDS/DCPS specification and illustrate it by building a mock distributed real-time application.

The DDS Communication Model

Data Centric Publish-Subscribe

The DDS publish-subscribe model connects anonymous information producers (publishers) with information consumers (subscribers). A distributed application is composed of processes called "participants", each running in a separate address space, possibly on different computers. A participant may simultaneously publish and subscribe to typed data-flows identified by names called "Topics". The interface allows publishers and subscribers to present type-safe API interfaces to the application.

The typical DDS application architecture can be represented as a "data-flow bus" as in Figure 1. The data model means you can essentially ignore the complexity of the data flow; each node gets the data it needs from the bus. The communications are decoupled in space (nodes can be anywhere), time (delivery may be immediately after publication or later), and flow (delivery may be reliably made at controlled bandwidth).

Strongly Typed Global Data Space

Participants using DDS can read/write data efficiently and naturally with a typed interface. Underneath, the DDS middleware will distribute the data so that each reading participant can access the "most-current" values. In effect, the service creates a global "data space" that any participant can read and write. It also creates a namespace to let participants find and share objects.

Figure 1: DDS may be regarded as a data-bus for distributed applications.

To increase scalability, topics may contain multiple independent data channels identified by "keys". This allows nodes to subscribe to many, possibly thousands, of similar data streams with a single subscription. When the data arrives, the middleware can sort it by the key and deliver it for efficient processing.

DDS also provides a "state propagation" model. This model allows nodes to treat DDS-provided data structures like distributed shared-memory structures, with values efficiently updated only when they change. There are facilities to ensure coherent and ordered state updates. An optional part of the specification called the Data Local Reconstruction Layer (DLRL) abstracts most of the underlying publish-subscribe semantics and presents a very-near analog to distributed shared memory.

Quality of Service Per Data Flow

DDS allows for fine control over Quality of Service (QoS) on a "per data flow basis". Each publisher-subscriber pair can establish independent quality of service (QoS) agreements. This aspect, unique to DDS, enables application designs that easily support extremely complex, flexible data flow requirements.

QoS parameters control virtually every aspect of the DDS model and the underlying communications mechanisms. Many QoS parameters are implemented as "contracts" between publishers and subscribers; publishers offer and subscribers request levels of service. The middleware is responsible for determining if the offer can satisfy the request, thereby establishing the communication or indicating an incompatibility error. Ensuring that participants meet the level-of-service contracts guarantees predictable operation necessary for real-time systems.

Automatic Discovery and Management of Data Flows

DDS is designed to automatically discover publishers and subscribers for each topic, and autonomously establish data flows between them as permitted by the settings of the quality of service parameters.

The DDS model provides fast location transparency. That makes it well suited for systems with dynamic configuration changes. It quickly discovers new nodes, new participants on those nodes, and new data topics between participants. The system cleanly flushes old or failed nodes and data flows as well.

Unreliable Connectionless Transports

DDS is fundamentally designed to work over unreliable transports, such as UDP or wireless networks. No facilities require central servers or special nodes. Efficient, direct, peer-to-peer communications, or even multicasting, can implement every part of the model.

Figure 2 summarizes the above five key aspects of the DDS communication model.

Figure 2: Fundamental aspects of the DDS communication model.

The DDS Programming Model

Figure 3 illustrates the key objects in typical DDS distributed application. A "domain" is an abstract concept referring to a collection of DomainParticipant objects that can communicate with each other. A "DomainParticipant" object represents the membership of an application in a domain. Thus, a domain represents a communication plane: participants may communicate only with other participants on the same domain. The DomainParticipant is a container of all other DDS objects.

Figure 3: Object diagram of a typical DDS application. The domain comprises of three DomainParticipants communicating over three different Topics. The first DomainParticipant contains a Subsriber constaining a single DataReader and a Publisher containing two DataWriters. The second DomainParticipant contains a single subscriber containing two DataWriter bound to two different topics. The third DomainParticipant contains a single Publisher containing a single DataWriter.

A "Publisher" object is responsible for data issuance, and may publish data of different data types. A "DataWriter" object is a typed facade to a publisher; participants use DataWriter(s) to communicate the value of and changes to data of a given type. Once new data values have been communicated to the publisher, it is the Publisher's responsibility to determine when it is appropriate to issue the corresponding message and to actually perform the issuance (the Publisher will do this according to its QoS, or the QoS attached to the corresponding DataWriter, and/or its internal state).

A "Subscriber" object receives published data and makes it available to the participant. A Subscriber may receive and dispatch data of different specified types. To access the received data, the participant must use a typed "DataReader" attached to the Subscriber.

The association of a DataWriter object (representing a publication) with DataReader objects (representing the subscriptions) is done by means of the "Topic". A Topic associates a name (unique in the system), a data type, and QoS related to the data itself. The type definition provides enough information for the service to manipulate the data (for example serialize it into a network-format for transmission). The definition can be done by means of a textual language (e.g. something like "float x; float y;") or by means of an operational "plugin" that provides the necessary methods.

Figure 4 shows the DDS class diagram. A DomainParticipantFactory is a singleton used to create and delete DomainParticipants. A DomainParticipants is bound to the domain given by DomainParticipant::get_domain_id(). The DomainParticipant acts as factory for creating and deleting the DomainEntities Topic, Publisher, and Subscriber. The Publisher acts as a factory for creating and deleting strongly types DataWriters for a previously created topic. The Subscriber acts as a factory for creating and deleting strongly types DataReaders for a previously created topic. The DataWriter::write() operation allows the user application to update the data with anew value. The DataReader::take() operation is used to take the received data sample from the DDS middleware.

[Click image to view at full size]
Figure 4: UML class diagram showing the DDS entity inheritance and containment relationships. Key methods are shown.

The abstract Entity interface defines the common operations supported by all the DomainParticipant, and the DomainEntities (i.e. Topic, Publisher, DataWriter, Subscriber, DataReader). The get_qos() and set_qos() operation can be used to retrieve and modify of the QoS; the get_listener() and operations can be used to retrieve and install user listeners for specific status conditions that can be detected by the DDS middleware. The exact list of QoS policies and listeners is specific to each concrete entity type.

An Example DDS Application

Figure 5 shows a concrete but simple distributed application. An embedded single-board computer (SBC) is hardwired to a temperature sensor and connected to an Ethernet transport. It is responsible for gathering temperature sensor data at a specific rate. A workstation, also connected to the network, is responsible for displaying that data on a screen for an operator to view. A variation of this scenario can have multiple temperature sensors wired to the embedded computer each periodically providing the temperature measurement at a different spot, and multiple workstations interested in processing the temperature measurements.

Figure 5: An embedded single-board computer publishes temperature data. A display workstation subscribes to the data.

Identify the domains, participants, topics

The first step to in creating this example is to identify the communication domains, and the participants and topics in each domain. In general, an application can belong to multiple domains as in Figure 6 by creating several DomainParticipants. Multiple domains provide effective data isolation. For example, in a very large systems, it make make sense to have different domains for each functional area in overall system, or to add experimental functionality in a new domain.

Figure 6: Multiple domains provide effective data isolation. Three applications are communicating Command/Control data in Domain A, three other applications are communicating Status data in Domain B, and two other applications are communicating in Domain C.

For this example, the distributed application will use a single domain comprising two participants and a single topic called "Temperature" for exchanging sensor data. A participant on the embedded computer publishes updates to the "Temperature" topic; a participant on the workstation that subscribes to the "Temperature"topic. Additional processing workstations can receive the sensor measurements simply by creating participants that subscribe to the "Temperature" topic.

Define the topic data types and generate data type support code

Since Topics are strongly typed in DDS, the next step is to define the data types associated with the topics. We define a type "SensorInfo" (Example 1) in the OMG Interface Description Language (IDL) since DDS middleware can support multiple language bindings. The SensorInfo data type contains an "id" field to identify the temperature sensor (when multiple temperature sensors are attached to the embedded computer), and a "value" field to specify the measured temperature value. The "id" field is marked as a "key" field; this lets the DDS middleware distinguish between updates from different sensors.

struct SensorInfo {
    long id; //@key
    long value;
};
Example 1

Given the IDL type definition, a DDS middleware provided code generator is used to generate language specific code to deal with the data type. The generate code will contain a definition of the type in the target language (C++ in our example), standard type specific APIs specified by DDS, and middleware implementation specific code.

For the SensorInfo data type, the following standard classes are generated: SensorInfoTypeSupport, SensorInfoDataWriter, SensorInfoDataReader.

Common steps in writing a participant application

The steps in writing a participant application are common, whether it is a publishes or subscribes to data.

1. Create a DomainParticipant.

A DomainParticipant is created bound to a domain as in Example 2, using the DDS standard APIs.

DDSDomainParticipantFactory* factory = NULL;
DDSDomainParticipant* participant = NULL;
DDS_DomainParticipantQos participant_qos;
DDS_DomainId_t domain_id = 10;

factory = DDSDomainParticipantFactory::get_instance();
if (factory == NULL) { /* handle error */ }

factory->get_default_participant_qos(participant_qos);

participant = factory->create_participant(domain_id, 
                                          participant_qos, 
                                          NULL /* participant_listener*/);
if (participant == NULL) { /* handle error */ }
Example 2

2. Register user data types

The generated SensorInfoTypeSupport::register_type() standard method is used to make the type available for use by the DomainParticipant as in Example 3.

DDS_ReturnCode_t retcode;

retcode = SensorInfoTypeSupport::register_type(participant, "SensorInfo");

if (retcode != DDS_RETCODE_OK) { /* handle error */ }

Example 3

3. Create Topics

A named Topic is created bound to a registered user data type as in Example 4 using the standard DDS APIs. A Topic is automatically establishes a data flow between its DataWriters and DataReaders.

DDS_TopicQos topic_qos;
DDSTopic* topic = NULL;

participant->get_default_topic_qos(topic_qos);

topic = participant->create_topic("Temperature", "SensorInfo",
                                  topic_qos, 
                      NULL /* topic_listener*/);

if (topic == NULL) { /* handle error */ }
Example 4

4. Write the publishing participant application

The publishing application on the embedded computer creates a Publisher and a DataWriter as follows.

5. Create Publishers

Given a DomainParticipant, a Publisher is created as shown in Example 5. A Publisher groups a set of DataWriters for different Topics. It also provides methods to suspend/resume publications, and for batching coherent changes.

DDS_PublisherQos publisher_qos;
DDSPublisher* publisher = NULL;

participant->get_default_publisher_qos(publisher_qos);

publisher = participant->create_publisher(publisher_qos, 
                                          NULL /* publisher_listener */);
if (publisher == NULL) { /* handle ... error */ }
Example 5

6. Create DataWriters

Given a Publisher and a Topic belonging to the same DomainParticipant, a strongly typed SensorInfoDataWriter is created as in Example 6. The DataWriter provides methods to send data samples, and manage the lifecycle of a keyed data channel.

DDS_DataWriterQos writer_qos;
SensorInfoDataWriter *writer = NULL;

publisher->get_default_datawriter_qos(writer_qos);

writer = publisher->create_datawriter(topic,
                                      writer_qos, 
                                      NULL /* writer_listener */);
if (writer == NULL) { /* handle error */ }
Example 6

7. Send data updates

Given a SensorInfoDataWriter, new updated values of SensorInfo are published to the subscribers using the SensorInfoDataWriter::write() method, as in Example 7.

SensorInfo sensor_info;   
DDS_InstanceHandle_t sensor_instance_handle = DDS_HANDLE_NIL; 
DDS_ReturnCode_t retcode;

while (1) {

    /* gather_sensor_measurement(&sensor_info); */

   retcode = writer->write(sensor_info, sensor_instance_handle);

   if (retcode != DDS_RETCODE_OK) { /* handle failure */ }
}
Example 7

Figure 7 summarizes the key objects involved in sending data samples (updates) the subscribers of the associated Topic.

Figure 7: A strongly typed DataWriter (DW) sends data samples S Q(updates) to all the subscribers of the associated Topic.

8. Write the subscribing participant application

The subscribing application on the display workstation creates a Subscriber and a DataReader as follows.

Given a DomainParticipant, a Subscriber is created as shown in Example 8. A Subscriber groups a set of DataReaders for different Topics. It also provides methods for ordered access to the received samples across the DataReaders.

DDS_SubscriberQos subscriber_qos;
DDSSubscriber* subscriber = NULL;

participant->get_default_subscriber_qos(subscriber_qos);

subscriber = participant->create_subscriber(subscriber_qos, 
                                            NULL /* subscriber_listener*/);
if (subscriber == NULL) { /* handle error */ }
Example 8

9. Create DataReaders

Given a Subscriber and a Topic belonging to the same DomainParticipant, a strongly typed SensorInfoDataReader is created as in Example 9. The SensorInfoDataReader object is attached to a DataReaderListener, whose on_data_available() method is invoked to process data samples as they are received.

DDS_DataReaderQos reader_qos;
SensorInfoDataReader* reader = NULL;

DDSDataReaderListener* reader_listener = new MyReaderListener(); 

subscriber->get_default_datareader_qos(reader_qos);

reader = subscriber->create_datareader(topic,
                                       reader_qos, 
                                       reader_listener);
if (reader == NULL) { /* handle error */ }
Example 9

Note that DDS also provides a "wait-based" access scheme that lets a caller wait for data to become available. A caller can also poll for available data. In addition a DataReader provides methods to wait for historical data samples and peek at available data samples.

9. Receive data samples

The MyReaderListener::on_data_available() method is called when data samples are available for retrieval by the application. The received SensorInfo data samples that match a desired state criteria are taken from the DDS middleware using the SensorInfoDataReader::take() method, as in Example 10. The "data_seq" contains a sequence of SensorInfo samples; "info_seq" contains additional information about the state of each sample. The contents of these sequences are loaned my the middleware to the application. After the data samples, the application must return the loan by calling SensorInfoDataReader::return_loan().

class MyReaderListener : public DDSDataReaderListener {
  public:
    virtual void on_data_available(DDSDataReader* reader);
};

void MyReaderListener::on_data_available(DDSDataReader* reader)
{
    SensorInfoDataReader *myReader = (SensorInfoDataReader *)reader;
    SensorInfoSeq data_seq;
    DDS_SampleInfoSeq info_seq;
    DDS_ReturnCode_t retcode;
    int i;

    retcode = myReader->take(data_seq, info_seq, 
                    DDS_LENGTH_UNLIMITED, 
                    DDS_ANY_SAMPLE_STATE, 
                    DDS_ANY_VIEW_STATE, 
                    DDS_ANY_INSTANCE_STATE);

    if (retcode != DDS_RETCODE_OK) { /* handle error */ }

    for (i = 0; i < data_seq.length(); ++i) {
        /* display_sensor_measurement(&data_seq[i]); */
    }

    myReader->return_loan(data_seq, info_seq);
}
Example 10

Figure 8 summarizes the key objects involved in receiving data samples (updates) from the publishers of the associated Topic.

Figure 8: A strongly typed DataReader (DR) receives data samples S (updates) from the publishers of the associated Topic.

Tuning the Quality of Service

The QoSPolicy settings available on the DataReader and DataWriter objects can be used to fine tune characteristics of the data flow between them. The default policies are tuned to the desired settings, and may be specified the time of DataWriter/DataReader creation or later via the get_qos()/set_qos() operations.

For example, periodic DataWriters can indicate the rate at which they can publish by offering guaranteed update deadlines via the "Deadline" QoSPolicy. By setting a deadline, a compliant DataWriter promises to send a new update at a minimum rate. DataReaders may then request data at that or any slower rate via the TimeBasedFilter QosPolicy. DataReaders can also specify a deadline for receiving the next sample update. If the deadline is missed, the the middleware can notify a listener installed by the application.

DataWriters may offer levels of reliability (via the Reliability QoSPolicy), parameterized by the number of past issues they can store to retry transmissions (via the History QoSPolicy). DataReaders may request differing levels of reliable delivery, ranging from fast-but-unreliable "best efforts" to highly reliable in-order delivery. This provides per-data-flow reliability control.

Figure 9: Under exclusive ownership a Topic channel can only be updated by a single DataWriter, the one with the highest ownership-strength S.

The middleware can automatically arbitrate between multiple DataWriters of the same Topic as specified by the Ownership and OwnershipStrength QosPolicies (Figure 9). Under exclusive ownership, a channel in a Topic can only be updated by a single DataWriter. DataReaders receive from the highest ownership-strength active DataWriter. This provides automatic fail-over; if a strong DataWriter fails, all DataReaders immediately receive updates from the backup (weaker) DataWriter. A DataWriter can also specify duration after which a sample should be considered stale via the "Lifespan" QosPolicy (Figure 10).

Figure 10: The Lifespan QoS Policy specifies the duration after which a received data sample should be considered stale, and discarded by the middleware.

DataWriters can specify how long previously published data is saved via the Durability QosPolicy. Late-joining DataReaders to durable DataWriters can then be automatically updated with past values by the DDS middleware.

The DDS specification defines many more QosPolicies. The behavior of each entity can be tuned on a per entity instance basis (Figure 4).

Responding to Status Changes

Each DDS entity is associated with a set of Status objects, whose value can be updated by the the get_*_status() methods on the entity. An application can poll for status changes, install listeners to be automatically notified by the middleware upon status changes, or wait on condition variable that gets triggered upon specific status changes.

The application can respond to specific status changes of interest. For example, the application can be notified when a DataWriter is no longer active or its liveliness is changed; or when another topic exists with the same name but different characteristics; or when a QosPolicy value was incompatible with what is offered; or when a deadline was missed; or when a new DataReader (or DataWriter) matching a Topic is discovered; or when data is available; or when samples are lost or rejected.

The application response to status changes can be tuned on a per entity instance basis.

Probing Further

We have looked at the key features of the OMG DDS specification, and highlighted some of the key QosPolicies and Status available for DDS entities.

Other QoS parameters control when the middleware detects nodes that have failed, suggest latency budgets, set delivery order, attach user data, prioritize messages, set resource utilization limits, partition the system into namespaces, and more.

DDS can also support content-based subscriptions. An application can use a ContentFilteredTopic to declare a filter expression by which only selected data samples of the specified Topic would be received and presented to the application. For example, an "alarm" application could setup DataReaders to only receive samples when a temperature exceeded a certain threshold.

DDS also supports a MultiTopic construct, which specifies a logical grouping of several Topics. It allows an application to select fields from multiple types and recombine them into a new type. As a large application evolves, MultiTopics can be a powerful tool: the data types used within a subsystem may change, but the contents of those new types can be recombined to look like the old types, preserving the interface between subsystems.

Conclusion

The Data Distribution Service is a OMG specification that creates a very simple architecture for data communication, while enabling very complex data patterns. Topics allow endpoint nodes to be abstracted from each other, so nodes can enter and leave the distributed application dynamically. The QoS parameters can be changed on a per entity basis. This per entity configurability is the key to supporting complex data communication patterns. The DDS API for sending and receiving data frees developers from having to worry about any low-level network programming.

References

Data Distribution Service (OMG document ptc/200403-07).

Catalog of OMG IDL / Language Mappings Specifications

Rajive Joshi and Gerardo Pardo-Castellote: "A Comparison and Mapping of Data Distribution Service and High-Level Architecture", Paper Id 03F-SIW-68, 2003 Fall Simulation Interoperability Workshop, Fall 2003.

Gerardo Pardo-Castellote: "OMG Data Distribution Service: Architectural overview", IEEE International Conference on Distributed Computing Systems, 2003.

Stan Schneider and Bert Farabaugh: "Is DDS for You?"

Data Distribution Service Resources


RAJIVE JOSHI, Ph.D. is a Principal Engineer at Real-Time Innovations (RTI) Inc. Dr. Joshi specializes in object-oriented and component-based software architecture and design. He is responsible for the DDS C++ API and the pluggable transports framework in RTI's NDDS product which implements the DDS specification. He can be reached at [email protected].

GERARDO PARDO-CASTELLOTE, Ph.D. is the Chief Technology Officer at RTI. Dr. Pardo-Castellote specializes in Real-Time software architectures and networking. His professional experience includes time-critical software for data acquisition and processing, real-time planning and scheduling software, control system software, and software-system design. He can be reached at [email protected].

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.