|
Symbian's EDBMS strikes a balance between functionality and small footprint, letting you create high-value database apps that run without alteration across diverse mobile device families.
June 27, 2006
URL:http://www.drdobbs.com/mobile/symbian-database-components/189601913
Regan Coleman is co-founder of Xenient, a U.S.-based partner of Symbian, providing SymbianOS consulting, application development, and training
in the U.S. and Canada. He was also a co-founder of Nobel Technology, a SQL Server consulting company, Context Integration, a 500-person web integrator, and Vectiv, a company providing real estate management software. Regan has a BA in Applied Math from Harvard Universtity. He can be reached at [email protected].
SymbianOS, the Symbian operating system that's specifically designed for mobile phones, is currently being implemented on three families of mobile devices:
Smartphone/Pearl devices look like typical cell phones, with one-handed operation and a phone-type keyboard. An example of a Smartphone-type device running SymbianOS is Panasonic's x800. PDA/Quartz devices look like PDAs, typically have a stylus/pen, and support handwriting recognition. An example of such a device running SymbianOS is Sony Ericsson's P910 or the Motorola A1000. Communicator/Crystal devices might look like cell phones, but feature a clamshell design which unfolds to display a qwerty keyboard. An example Crystal phone is the Nokia 9300 or 9500. Again, what all of these devices from different companies have in common is that they run the same operating system--SymbianOS.
As the licensees of SymbianOS released SymbianOS-based devices, it became apparent that certain portions of the OS were specific to the device family. For example, handwriting recognition might not make sense for one-handed smartphones that lack pen-type input. The issue of how to make software apply to a wide variety of devices is not a problem faced solely by Symbian. Java, for example, split into J2EE, J2SE, and J2ME (and profiles within J2ME). And taking an operating system example, there are several Microsoft Windows operating system variants, such as Windows XP Tablet PC edition, Windows NT Embedded, Windows Mobile 2003 for Smartphone, Windows Automotive, and the like. Symbian responded by offering additional user interface layers specific to the device family.
SymbianOS, however, is the same operating system across all these devices. For instance, SymbianOS shares a common universal UI layer known as "Uikon/EikStd" across the device families. Sticking to the common UI layer lets you create applications that work across device families (though this also limits the ability to take advantage of features native to the device). At the risk of straying too far afield, there is a thin upper UI layer on top that is specific to each device family. Furthermore, since SymbianOS is object-oriented and customizable, licensees are free to implement custom classes derived from the SymbianOS base classes and brand these custom classes on top of the SymbianOS layers. Nokia, for example, has done this with its Series60 layer on top of SymbianOS. Another example is UIQ, a graphics layer on top of SymbianOS developed by UIQ (a Symbian subsidiary).
That said, my focus here is not on the thin UI layer, but on the database component, which admittedly has nothing directly to do with the UI layer. Thus this discussion is relevant to the variety of SymbianOS phones and companies licensing SymbianOS--Arima, BenQ, Fujitsu, Motorola, Nokia, Panasonic, Samsung, Sanyo, Sendo, Siemens, and SonyEriccson, to name a few. The sample applications and code I present in this article work on devices running SymbianOS from all these companies.
Throughout the design, SymbianOS is designed with a variety of assumptions, foremost among them that memory is precious. When there is an opportunity to save memory, the benefits of a smaller footprint are weighed strongly against the costs. SymbianOS is a rich, object-oriented, 32-bit real-time OS that previously used a microkernel architecture. As of version 8.0b and beyond, however, a nanokernel architecture makes it possible to get SymbianOS up and running in a 500KB footprint, (though realistically for more functionality a larger footprint is typical). Furthermore, the kernel does not assume the presence of persistent storage--it can run entirely in ROM.
Operating systems such as SymbianOS, PalmOS, and earlier generation in-house proprietary phone operating systems developed by phone manufacturers were designed for memory-constrained devices. That philosophy requires a careful weighing of memory costs for each additional feature. That is a different approach than an operating system such as Windows or Solaris, for example, which are designed to run on machines with lots of memory and don't need to apply a rigorous discipline to weighing footprint costs. For that reason, the footprint/functionality of operating systems such as SymbianOS and PalmOS have a natural advantage over a retrofitting approach; that is, taking an OS such as Windows or Solaris, and attempting to squeeze the OS onto a memory-constrained device. Just as new features of the operating system as a whole are weighed carefully against the increased memory usage, the internal SymbianOS database component functionality is carefully weighed--the results are database components with a tiny footprint. With the tiny footprint comes limitations in functionality; the goal of the SymbianOS internal database is not how many features can be added, but which are the essential features that can create a viable product, in a tiny footprint.
An early name of SymbianOS was "EPOC" and vestiges of that name remain in certain places in the operating system--the entry point of program executables in E32Main(), for example. The SymbianOS Data Base Management System component is still referred to as "EDBMS," short for "Epoc Data Base Management System."
The DBMS server runs as a separate server process. Like all non-kernel processes in SymbianOS, the DBMS server does not run with supervisor privilege. Requests made to the DBMS server. Like most requests to servers on SymbianOS, requests are routed through the kernel. Client/server requests to the DBMS server use kernel executive calls. Kernel executive calls do not require a context switch for the passing of the request. Messages are stored in internal kernel message slots allocated for the message. Internally, the DBMS server utilizes SymbianOS's active object approach to multitask, which, rather than having the overhead of additional threads, minimizes thread creation and thus context switches, and is an object-oriented to multitasking as well.
Remembering that SymbianOS is designed with a minimal approach, there are some features a larger footprint DBMS might implement, but are missing in the SymbianOS DBMS. To be clear, this is not a full-featured database that is going to let you, say, perform complex queries. But that is not the goal, and there are other solutions for application developers seeking a fuller DBMS implementation. The goal of Symbian's provided DBMS is to facilitate the storage of small amounts of information in a tiny footprint implementation.
In classic Symbian fashion, and in keeping with an object-oriented approach, the DBMS implementation was designed to be extensible. Technically, the DBMS component really just defines API classes, and a wide variety of DBMS implementations could exist. In this article, I discuss the implementation that comes with the operating system, which physically stores the database as a file in the filesystem. However, it is certainly possible to use these API classes with an in-memory implementation for example. In fact, not only would it be possible to implement an in-memory implementation, but the framework is designed to encourage future implementations. This approach of designing for future extensions is similar to the Symbian approach to sockets; for example, where an extensible communication framework was designed. By creating an extensible architecture, plugins for new communication protocols are easily added to SymbianOS. This was the case with early protocols such as WAP and Bluetooth, and also true more recently for protocols such as 802.11x.
The good news is that the Symbian DBMS implementation has a tiny footprint. In spite of the tiny footprint, important features such as the ability to build and maintain multiple indexes on tables are supported. The bad news is there are some major limitations that go along with that. Probably the biggest is the lack of support for joins. If you need joins, use one of the third-party implementations, being aware of the larger footprint that entails. However, for many mobile applications, the EDBMS component actually suffices. For example, the SymbianOS communications database uses the EDBMS component. In addition to internal system databases, however, the EDBMS component is also available to third-party developers.
To access the database server, use the RDbNamedDatabase class. This is the most common approach, and allows multiple readers/writers to connect to the same database. SymbianOS, a sophisticated operating system, fully supports preemptive multithreading, and it is not uncommon for multiple writers to want to access the same database. RDbStoreDatabase is less commonly used; it bypasses the DBMS server which is slight benefit in terms of not having the additional DBMS server layer. However, the DBMS server layer provides multi-user benefits.
Thus applications using RDbStoreDatabase only allow one writer at a time to have a particular database open. Another benefit of the less-used RDbStoreDatabase is that that class provides for the storing of other data besides tables in the same file as the database tables. Listing One is an example use of RDbNamedDatbase which lists the tables in a database.
// tables.cpp // #include "eustd.h" #include_LIT(KFile,"c:\\mydatabase.cdb"); // Simple console program to list tables in a database LOCAL_C void doExampleL() { TInt i; // handle into the DBMS server RDbs myDbs; // handle for our database RDbNamedDatabase myDatabase; // we have to connect to the DBMS server first User::LeaveIfError( myDbs.Connect()); //handles use the cleanup stack CleanupClosePushL(myDbs); // Open the contacts database using the default format User::LeaveIfError( myDatabase.Open(myDbs,KFile,_L("")) ); //handles use the cleanup stack CleanupClosePushL(myDatabase); CDbTableNames* myTableNames; TInt numTables = 0; // retrieve a list of tables in the contacts database myTableNames = myDatabase.TableNamesL(); numTables = myTableNames->Count(); for ( i = 0 ; i < numTables; i++) { console->Printf( (*myTableNames)[i]); console->Printf(_L("\n")); } // pause so the console can be seen console->Getch(); // DBMS connection and database handles must be closed CleanupStack::PopAndDestroy(2); delete myTableNames; }// end doExampleL
The Symbian operating system supports a VFAT-type filesystem that's similar to DOS. A SymbianOS database is just another file on a filesystem, and can be arbitrarily named in an arbitrary location. So, for example, you could store a file named "my database" in the C:\Documents folder. Within my one database file 'C:\Documents\my database' can be stored several tables.
The details of the SymbianOS database API are proprietary, specific to Symbian, and usually not the best approach to creating tables. The industry-standard language for creating tables, retrieving data, and modifying data is SQL. I recommend using SQL with the SymbianOS DBMS wherever possible.
Furthermore, the documentation states that contrary to normal expectations, commands utilized using SQL on the Symbian DBMS actually perform at least as fast as the equivalent operation using the C++ API. One exception to the "SQL is about as fast as the C++ API" is when using bulk inserts--for a few rows, there's not much difference, but for more than a few rows, the C++ API outperforms using SQL's INSERT statements, according to Symbian
In summary, my recommendation is to use the SQL interface whenever possible. It's standard, easier to use, and the performance downsides are minimal. In some cases, however, there is functionality that the SQL interface does not provide, so you have to use C++ to implement the functionality. Probably the three most important examples are compacting the database, transaction management, and notification.
Database compaction involves reorganizing the internal layout of the file to create a smaller file. Transaction management involves explicit rollbacks and commits. Notification is an interesting feature--users of a database can use the RDbNotifier class to get notification of certain events in the database, such as a transaction rollback.
A typical table creation session using Symbian SQL might look like Listing Two.
create table authors ( author_id counter, last_name varchar(40), first_name varchar(40) )
The first step in data retrieval is expressing the desired query. To process the rows, a rowset structure client-side needs to be instantiated. The most common class client-side for handling the rowset has a confusing name: RDbView. It's confusing because:
After the query is sent to the DBMS server, the next step is to call RDbRowset::Prepare(). Prepare() basically allocates structures server-side for the evaluated rows. The TDbWindow argument to Prepare() lets you make suggestions as to the size of the server-size structure. Finally, the RDbRowset::Evaluate() method is called. This is an incremental evaluation, which causes the server-side structures prepared previously to be filled. The EvaluateAll() method should be used with caution--SQL rows retrieved can be quite large--unless you know you're only going to get a small rowset as a result (say one row, since you have a unique index), it's more prudent to use Evaluate(), and evaluate server-side in incremental steps.
SymbianOS avoids a lot of the problems caused by methods that block through ubiquitous asynchronous non-blocking methods, and the API into the Symbian DBMS is no exception. Most calls have asynchronous versions, and are designed to allow processing to occur in incremental steps. Open select statements in any SQL environment are notorious for causing problems due to the potential for large rowsets--this can cause incorrectly--designed database clients to hang or cause the unnecessary allocation of large amounts of memory, something that is anathema in memory-constrained environments. The SymbianOS developer has the option for specifying the server-side buffer space usage. In the following example, the developer is requesting the server allocate memory for three rows before, and three rows after, the current row. Listing Three is code that performs retrieval.
...
//here is my select statement
_LIT(KRetrieveData,"select * from authors");
//Prepare server-side buffer of 3 rows before and after
User::LeaveIfError(myRowset.Prepare(myDatabase,TDbQuery(KRetrieveData),TDbWi ndow(3,3)));
// I need to cleanup any RClass -- this includes myRowset
// so I'll put myRowset on the cleanup stack
CleanupClosePushL(myRowset);
// My simple program only needs to evaluate once
User::LeaveIfError(myRowset.EvaluateAll());
//subsequent retrieval
while ( myRowset.NextL() )
{
// get the current row
myRowset.GetL();
author_id = myRowset.ColUint(1);
last_name = myRowset.ColDes(2);
first_name = myRowset.ColDes(3);
// do something with the results.
// in my simple program, I just print each row to the console
console->Printf(_L("%d "), author_id);
console->Printf(last_name);
console->Printf(_L(" "));
console->Printf(first_name);
console->Printf(_L("\n"));
}
Note that this buffer is server-side. Remember that the client only retrieves one row at a time.
Since most of the calls to EDBMS are asynchronous, typically the developer makes the EDBMS call from within a client-side active object. Using that combination avoids causing applications to block due to long-running database operations. However, if for some reason you are not using active objects client-side, or in other cases, sometimes the RDbIncremental class can be useful. This lets clients do things such as create an index in steps. Instead of having to wait a long time while an index is being built, for example, client-side programs can start the index creation process at one point, then later on ask the EDBMS server to do a little more work on the index creation, and so on.
Again, RDbView is not a normal database view; on the contrary, it is a client-side rowset. For certain queries, there's a more efficient client-side rowset structure: RDbTable. This is another rowset structure--the efficiency is server-side. Whereas in the RDbView example, the internal server-side result structure is actually a dynaset, a separate server-side copy of the data. When using RDbTable, the server-side structure is not a dynaset, but instead has pointers referring directly to the table structures.
Thinking through the types of applications that use the Symbian DBMS gives the conclusion that updates beyond the simple functionality are not high on the priority list, and the Symbian implementation is no exception. The SQL update is somewhat limited, allowing non-joining where clauses, but not complex expressions in the SET portion of the update statement. To update a column value with a complex expression, the expression calculation would have to be done outside of SQL.
Developers who choose not to use SQL to modify table data are required to:
Using SQL as much as possible requires much less programming and is a good example of why I recommend using SQL.
delete from authors where author_id = 0
SymbianOS supports multiple indexes on multiple columns on a table, and indexes can be unique. Listing Five is Symbian SQL for creating two multiple-column indexes on the same table.
create unique index id_index on authors(author_id) create index index2 on authors(last_name, first_name) As you can see, this is straightforward SQL.
Symbian's DBMS server supports a rich variety of datatypes, from bits, tinyints to smallints, integer, bigint, as well as real, float, and double precision. Auto-increment columns are supported as well. There is a date type, a time type, and a timestamp, char(n), varchar(n), binary(n), and varbinary(n).
If a program will be the exclusive user of a database, it is possible to bypass the DBMS server layer by using client-side access. In that case the client program directly accesses the system file server which is inside the base of the OS, and data retrieval will be faster than in the case where server. This is done by opening the database using the RDbStoreDatabase structure.
As mobile devices get smarter and smarter, there will be continue to be a wide variety of choices for deploying databases. Obviously, server-based databases (where the mobile device stores data on some large server machine to which the mobile device has some connection) should be given a hard look. If that is impractical, consider some local alternatives. A local alternative, in contrast to storing all data on a server, involves storing some or all the application data on the mobile device. A full-featured relational database implementation packs a lot of functionality but adds to the footprint on each device, and can require royalties and/or licenses per device. Since the internal Symbian DBMS is part of the OS, there's no per-device royalty. The Symbian DBMS implementation is a powerful implementation given its footprint--clearly it works for some applications, evidenced by Symbian's use of its DBMS for standard Symbian system applications such as Contacts, the Symbian standard application for phone contacts. In addition to internal use, and system applications, for some third-party applications using an all-local, or temp-local and replicate strategy, Symbian's royalty-free, small footprint, provided DBMS, makes sense as well.
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.