Tilo is a student of medical informatics in Heilbronn and Heidelberg, Germany. He also works as a freelancer for Siemens Med, and can be reached at [email protected].
Palm computing devices -- the PalmPilot, Palm III, and their cousins -- are designed to exchange data with stationary host machines. Unfortunately, Palm's desktop software for handling the data transfer runs only on Windows and Macintosh. You can develop your own custom synchronization code (called "conduits") using Palm's Conduit Development Kit, which interacts with Palm's HotSync Manager (HSM). However, the HSM itself has several weaknesses:
- It will not run as a Windows NT service.
- It offers poor support for multiple simultaneous connections.
- There are no management features for multiple PalmPilots.
- It is tied to the Windows platform.
To address these weaknesses, I wrote the SyncBuilder framework. It lets you write Java applications that can communicate with an arbitrary number of PalmPilots and can be deployed on any host platform. SyncBuilder works independently of the HotSync manager, supports the use of Network HotSync, and facilitates code sharing and reuse by offering a clearly defined interface for the synchronization modules.
SyncBuilder (available electronically from DDJ; see "Resource Center," page 5, and at http://come.to/SyncBuilder) is built on top of Kenneth Albanowski's Pilot-Link package (ftp://ryeham.ee.ryerson.ca/pub/ PalmOS/), a freely available (under the Library GNU Public License) package for building synchronization code on UNIX platforms. A port to Win9x/NT is available at http:// lemming.stud.fh-heilbronn.de/~christ/ pilot-xfer/. SyncBuilder derives from the Pilot-Link Java binding, and is also available under the LGPL. Because some of SyncBuilder still relies on Pilot-Link, it has limitations on platforms other than UNIX and Windows 9x/NT. However, there are plenty of things that SyncBuilder can do on all platforms, and the examples in this article have been designed to work on all platforms. Nevertheless, I am working hard to remove the existing limitations, one by one.
PalmPilot Basics
Before you can synchronize with the PalmPilot, you need to have a rough idea of what kind of data entities you may find on this gizmo. What follows is a quick, albeit incomplete, rundown:
- Databases. Each application on the device has zero or more databases that contain a header with some information (date of creation, number of records, and so on), variable-length records that contain the actual data, a Creator ID (CID), a type, and a name. The CID is used to link the database to an application. The CID, name, and type can be used to identify the database.
- Records. Each database contains records. The records store the content of the database. They are of arbitrary length, and they contain their own headers that contain information about the record (category, modified, deleted, and the like).
- Application information, which is linked to an application's database. Every application can have as many application information blocks as it has databases. These information blocks are typically used to store information about the database categories.
- Preferences. Each application may have arbitrary preferences that are associated with that application as a whole.
- User information, card information, and network HotSync information. These are various items that can be retrieved from the device. They are global and not tied to a specific app.
Desktop Link Protocol
The PalmPilot and host communicate through the Desktop Link Protocol (DLP). The PalmPilot initiates the connection, using either a direct serial cable or modem to reach the host. After an initial handshake, the host takes control. Using DLP, the host can order the PalmPilot to find, retrieve, and manipulate all of the aforementioned data entities. When the host is finished, it informs the PalmPilot, and both parties will disconnect.
Although DLP is a datagram-oriented protocol, the PalmPilot only uses it over reliable streaming connections (either an error-correcting serial connection or TCP/IP). The serial protocol stack consists of three layers and an extra mode for the negotiation of interface parameters; see Figure 1.
DLP is based on a simple request/response pattern, with the host sending the requests. The host assembles a datagram, which consists of a command code and a block of parameters. It then sends this datagram to the PalmPilot. The PalmPilot replies with a response datagram that contains a response code and a block of return values. The layout of these datagrams is illustrated in Figures 2, 3, and 4.
For example, if the host wanted to read a record from an open database, it would construct a datagram that contained the command code for "Read Record by Index," the database handle, and the record index in the parameter block. The PalmPilot would reply with a datagram that contained a response code indicating successful execution, and a return value with a verbatim binary copy of the record's contents.
If the host wanted to write a record to the PalmPilot, it would place the data and the database handle in the parameter block, and the command code would be set to "Write Record." The response datagram in these cases is very short, because it only contains the response code and the unique ID of the newly created record. Similar read and write operations are available for all the data entities on the PalmPilot. Some entities can also be looked up in various ways. The database records can be found through their index, through their unique ID, through their category (PalmOS lets you place records into 15 user-defined categories), and through their state of modification.
As the developer, you do not have to concern yourself with the internals of the DLP protocol. An array of easy to use methods will conceal all the complexity from you. These methods will do all the conversion tasks between Java data types and the byte-arrays of the datagrams for you. They also relieve you from the burden of having to remember the command codes of the DLP protocol.
Conduits
Conduits are dynamically loadable modules that tell the HotSync Manager software how to synchronize a specific PalmPilot app's data with a specific desktop app. Conduits use an API exposed by the HotSync Manager to handle the communications.
Because it can be tedious to operate on this low level, Palm sells the Conduit Development Kit (CDK), a C++ framework that encapsulates most of the synchronization logic and all of the necessary calls to the HotSync Manager. This framework contains a set of classes that can scan a database on the PalmPilot and retrieve the state of modification of all the records. You have to implement code that creates a similar list for the mirror database (the database of the desktop application that is supposed to be synchronized with the PalmPilot application). The C++ framework can then compare all of the states and decide how to update both databases.
You have to implement helper code that can translate single database records from the PalmPilot application's format to the mirror database's format. This process is described in an article by Stu Slack, published in PDA Developer magazine (September/ October 1996, also available at http:// www.wwg.com/published/conduit.html).
Palm Computing has also released CDK Java, a similar development kit for Java. However, this kit is still tied to the Windows platform and needs to cooperate with the HSM, which means it inherits all of its weaknesses.
SyncBuilder
SyncBuilder, on the other hand, supports all of the previously mentioned data entities, and communicates through either a direct serial cable, or through Network HotSync. Network HotSync works on all platforms, but serial connectivity is only supported on Windows 9x/NT and UNIX machines. Modem HotSync is not supported at all; you can get identical results using Network HotSync.
The framework began as a set of Java classes that encapsulated the native UNIX code of the Pilot-Link library. While this gave me a quick, working implementation on my Linux box, it restricted me to UNIX platforms. The Pilot-Link code was also not reentrant, which made it hard to use in conjunction with the multithreaded Java applications. I therefore decided to reimplement all the functionality of the Pilot-Link library in Java, with an emphasis on networking through TCP/IP and on full thread safety. This conversion is not complete yet, but the following parts are done:
- The ability to use Network HotSync.
- Support for all operations of the DLP protocol.
- Special support for some of the PalmPilot's built-in applications (Memo Pad, ToDo list, Datebook, Mail, and Address Book).
- Support for PalmPilot database files (.PRC and .PDB files).
Only synchronization through a direct serial cable still relies on native code.
The design of the framework is centered around the packages com.syncbuilder.sync, which contains classes for establishing connections between the host and the PalmPilot, and com.syncbuilder .storage, which contains classes that mirror the data entities.
The com.syncbuilder.sync.Dlp-class links these two packages by encapsulating the functionality of DLP. Every time you establish a connection with the PalmPilot, you can obtain an instance of Dlp. The Dlp class's methods take objects from the storage package as their parameters, and return objects from the same package. Many methods in the Dlp class open or create databases. These methods give you com.syncbuilder.storage.Database objects as the return value. The Database class has a set of methods to read/write records, which delegate their work to nonpublic methods of the Dlp class. This might not be a clean approach, but it is usable.
Other packages include com.syncbuilder .device, which offers wrappers for the global data entities (UserInfo, CardInfo, NetSyncInfo), and com.syncbuilder.util, which offers various utility classes (packing/unpacking of data structures, and so on).
The Database Class
The com.syncbuilder.storage.Database class is one of the key classes in SyncBuilder. You will not be able to develop synchronization applications without a working knowledge of it. Database is also one of the keys to developing reusable code for custom PalmPilot applications.
The Dlp object's openDB() method returns a Database object, which represents an opened database. This object gives you access to the records and properties of the database. You can also retrieve the Application Information block of the database, which is represented by the AppBlock class. The subclass CategoryAppBlock can store the names of up to 15 database categories within the Application Information block. This ability is used by most of the built-in applications.
Most of the methods in these classes (such as getRecord() and putRecord()) are self explanatory. Their parameters require an understanding of the ways in which the Palm devices organize their data. Figure 5 is an Entity-Relationship (ER) diagram of the Palm device. The class diagram in Figure 6 is organized similar to the ER diagram and should help you determine how the classes of the framework map to the entities on the Palm device.
Most PalmPilot applications use a different format for the data layout of their records, and contain application information that is different from that of any other application. SyncBuilder accommodates these differences by using specializations (inheritance) of the relevant classes. If you wish to develop support for a proprietary application, you will have to create specializations of three classes: DatabaseImpl, Record, and AppBlock. You might want to look at the subpackages in com.syncbuilder.storage (generic, appointment, ToDo, and the like).
Database uses DatabaseImpl, which implements the methods that are different between different databases. DatabaseImpl's main responsibility is to return the right specializations of Record and AppBlock. For example, the getRecord() method of the Database class specifies Record as the return parameter, but the type that is actually returned is a specialization of Record. The Database class delegates the creation of the right specializations to DatabaseImpl. I separated DatabaseImpl from Database to make it more reusable. I am already reusing it for a class that deals with in-memory databases (com.syncbuilder.storage.MemoryDatabase).
When you obtain a Database object through the Dlp, you specify the DatabaseImpl you wish to use as a parameter to the openDB() method. Record is responsible for properly parsing and constructing the internal representation of a database's records. Your own specialization needs to implement the pack() and unpack() methods. You might also wish to add accessor methods for the record's attributes. pack() is responsible for packing the record's attributes into the application's native byte structure. The native structure and its meaning are defined by the application that runs on the Palm device. You should derive meaningful attributes from that structure and write a conversion routine. The unpack() method does the exact opposite. It unpacks the application's native byte structure into the record's attributes. The methods in com.syncbuilder.util.PackUtils can help you write pack() and unpack() methods.
Implementing AppBlock is only necessary when the Palm application wishes to use application information blocks. The methods of this class are comparable to those of the Record class. This is not surprising, because they both inherit from the same parent class (com.syncbuilder .storage.Block). You will have to implement the pack() and unpack() methods again. If you wish to support database categories, like the built-in applications do, you can let your AppBlock class inherit from com.syncbuilder.storage.CategoryAppBlock instead of com.syncbuilder .storage.AppBlock. CategoryAppBlock already contains helpful support methods for parsing and creating categories.
You can use these classes right away to do simple data retrieval from your newly supported application. If you wish to do two-way synchronization, you need to determine the synchronization logic, and then write a specialization of the SyncHandler class. Listing Two shows the basic layout of a synchronization specialization and explains how all the parts fit together to form a synchronization application. The SyncHandler's init() method needs to reserve all the resources that are required by the synchronization procedures. The service() method is invoked for every Palm device that connects to your host. It provides you with a Dlp object and thus allows you to open a database and do synchronization work. Finally, the destroy() method is responsible for freeing up all resources when the synchronization application is shut down.
Examples
Listing One shows the typical structure of a simple synchronization application, which reads information from the Pilot and adds a nonsensical datebook entry. The application first determines how the user wants to synchronize (serial or network), then sets up an instance of ServerSocket and invokes the ServerSocket's accept() method. This is a blocking method that returns a Link object when it has successfully established a connection with a PalmPilot. The Link object's getDlp() method returns a Dlp object, which gives you access to all the functionality of the DLP protocol. SyncBuilder comes with special support for the PalmPilot's built-in applications. For example, there is a special DatabaseImpl class that supports the Datebook application, and you can feed that DatabaseImpl class to the Dlp's openDB() method to open the Datebook's database.
If you run this example through a serial cable, you should not wait longer than a few seconds before pressing Return; otherwise, the Pilot will consider the connection dead. Network HotSync does not have this limitation.
Listing Two shows you how to write a simple program that can serve many PalmPilots simultaneously through the Network HotSync protocol. Figure 7 illustrates the control flow for this program. The key element here is the SyncServer class, a multithreaded server that takes care of connection management. It delegates each incoming connection to a SyncHandler object. The SyncHandler object gets a Link to the connected device and can execute all DLP operations through this Link. You will have to provide your own implementation of SyncHandler to do anything meaningful. The implementation in Listing Two is called SimpleHandler. It is important to understand that all connections are handled simultaneously by the same SyncHandler object. This raises the same synchronization issues as JavaSoft's Servlet technology, but it can generally be handled easily.
The SyncHandler interface is one of the keys to software reuse in this framework (the other one is the ability to reuse implementations of DatabaseImpl, Record, and AppBlock). You can encapsulate all of your synchronization strategy into a SyncHandler, and then share it with others. You can also use a SyncHandler either with single-threaded code, or with the SyncServer. You can combine SyncHandlers by chaining them. In that scenario, one SyncHandler would be used to invoke the service() method of other SyncHandlers, perhaps based on the identity of the PalmPilot.
Conclusion
SyncBuilder is an open and continuing effort to develop a powerful framework for all of your synchronization needs. It is usable now and will be better in the future. Planned additions include user-management capabilities, configurable error-handling, and ready-to-use SyncHandlers for some of the more popular applications (e-mail, Satellite Forms). I appreciate all comments, success stories, and contributions of code.
DDJ
Listing One
/* samplecode/dlptest.java:
* Copyright (C) 1998, Tilo Christ
* This is free software, licensed under the GNU Public License V2.
* See the file COPYING for details.
*/
/* $Id: dlptest.java,v 1.1 1998/09/16 08:57:51 christ Exp $ */
import com.syncbuilder.sync.*;
import com.syncbuilder.storage.*;
import com.syncbuilder.device.*;
/** A test for the PalmOS DLP-functionality. This is also a sanity check
for the new Pure Java Implementation of the DLP.
* @author Tilo Christ
*/
public class dlptest
{
public static void main(String[] args)
{
try
{
String port = null;
System.out.print("Port to use (\".\" means
Network HotSync) [/dev/pilot]? ");
System.out.flush();
port = com.syncbuilder.util.Util.readLine();
if (port.equals(""))
port = "/dev/pilot";
if (port.equals("."))
{
Socket.setSocketFactory( new NetworkHSSocketImplFactory() );
System.out.println("Initiate Network HotSync now.");
}
else
{
System.out.println("Please hit the HotSync button now.");
}
ServerSocket ssock = new ServerSocket(port);
Link link = ssock.accept();
Dlp dlp = link.getDlp();
System.out.println("The connection-protocol has a version of: "
+ link.getProtocolVersion());
System.out.println("Information about the HotSync user will");
System.out.println("be read from your device");
System.out.println("as soon as you start the action.");
System.out.println("Press <RETURN> to start the action...");
com.syncbuilder.util.Util.readLine();
UserInfo uinfo = dlp.getUserInfo();
System.out.println( uinfo.toString() );
System.out.println("The user-info should have been displayed now.");
System.out.println("Press <RETURN> to continue...");
com.syncbuilder.util.Util.readLine();
System.out.println("The message on your PalmOS device should");
System.out.println("switch from 'Identifying User'
to 'Synchronizing'");
System.out.println("as soon as you start the action.");
System.out.println("Press <RETURN> to start the action...");
com.syncbuilder.util.Util.readLine();
dlp.switchMessage();
System.out.println("The message should have switched now.");
System.out.println("Press <RETURN> to continue...");
com.syncbuilder.util.Util.readLine();
System.out.println("The message on your PalmOS device should");
System.out.println("switch from 'Synchronizing' to
'Synchronizing Datebook'");
System.out.println("as soon as you start the action.");
System.out.println("Press <RETURN> to start the action...");
com.syncbuilder.util.Util.readLine();
Database dbase = dlp.openStockDB
(new com.syncbuilder.storage.appointment.DatabaseImpl());
System.out.println("The message should have switched now.");
System.out.println("Press <RETURN> to continue...");
com.syncbuilder.util.Util.readLine();
System.out.println("All " + dbase.getRecordCount() + "
entries from your datebook will be read and displayed");
System.out.println("as soon as you start the action.");
System.out.println("Press <RETURN> to start the action...");
com.syncbuilder.util.Util.readLine();
for (int i = 0;; i++)
{
Record rec = dbase.getRecord(i);
if (rec == null)
break;
System.out.println( rec.toString() );
}
System.out.println("The records should have been printed now.");
System.out.println("Press <RETURN> to continue...");
com.syncbuilder.util.Util.readLine();
System.out.println("A test entry will be added to
your datebook at the current date");
System.out.println("as soon as you start the action.");
System.out.println("Press <RETURN> to start the action...");
com.syncbuilder.util.Util.readLine();
com.syncbuilder.storage.Record record = dbase.createRecord();
((com.syncbuilder.storage.appointment.Record)record).description =
"Dinner at the White House";
dbase.putRecord(record);
System.out.println("The record should have been added
now with UID " + record.getID().getValue() + ".");
System.out.println("Press <RETURN> to continue...");
com.syncbuilder.util.Util.readLine();
dlp.endHotSync();
}
catch(Throwable t)
{
t.printStackTrace();
}
}
}
Listing Two
/* samplecode/SyncServerTest.java:
* Copyright (C) 1998, Tilo Christ
* This is free software, licensed under the GNU Public License V2.
* See the file COPYING for details.
*/
/* $Id$ */
import com.syncbuilder.sync.*;
import com.syncbuilder.device.*;
/** Demonstrates use of the com.syncbuilder.sync.SyncServer-class.
* Will not work through PalmPilot's regular HotSync app; only through
* the Network HotSync app (obtain separately from Palm Computing).
* @author Tilo Christ
*/
public class SyncServerTest
{
public static void main(String[] args)
{
try
{
// Create a ServerSocket for use with Network HotSync.
Socket.setSocketFactory( new NetworkHSSocketImplFactory() );
ServerSocket ssock = new ServerSocket(null);
// Create the SyncServer from the ServerSocket. The SyncServer
// will serve all clients using SimpleHandler (see the class
// SimpleHandler below). SimpleHandler will mostly print diagnostic
// messages and read the UserInfo from the device.
SyncServer sync_server = new SyncServer(ssock, new SimpleHandler());
System.err.println("Initiate the Network HotSync procedures now.");
// The SyncServer starts running NOW!
sync_server.start();
System.err.println("Press <RETURN> to stop the service");
com.syncbuilder.util.Util.readLine();
// The SyncServer is shut down
sync_server.stop();
}
catch(Throwable t)
{
t.printStackTrace();
}
}
}
/** Non-public class. Used by SyncServer to handle incoming connections. */
class SimpleHandler
implements com.syncbuilder.sync.SyncHandler
{
/** Invoked one time at start-up */
public void init()
{
System.err.println("SimpleHandler.init()");
}
/** Invoked every time a device connects */
public void service(Link link)
throws Exception
{
System.err.println("SimpleHandler.service() with "
+ link.toString() );
Dlp dlp = link.getDlp();
// Do it ten times, so it takes more time...
for (int i=0; i < 10; i++)
{
UserInfo uinfo = dlp.getUserInfo();
System.err.println( uinfo.toString() );
}
dlp.endHotSync();
}
/** Invoked one time at shutdown */
public void destroy()
{
System.err.println("SimpleHandler.destroy()");
}
}
Copyright © 1999, Dr. Dobb's Journal
