Dr. Dobb's Sourcebook January/February 1997

CORBA Trading on the WWW

There is one CORBAService that has direct counterparts on the Internet today, and whose merging with these counterparts should prove very interesting. The service is the CORBA Trader and its Internet equivalents are popular WWW search engines, web crawlers, and white pages such as Yahoo, HotBot, Alta Vista, and Four11.

The CORBA Trader offers a brokerage facility in which clients specify queries in a constraint language. The Trader responds by providing matching CORBA object references that the client can use to bind to a server of its choice. Client queries are specified in terms of Boolean combinations of (name, value) relations. For example, a client can request "return all objects for which ( price>$10 AND distance<10 km) OR (supplierMicrosoft)." Traders are particularly useful in very large distributed systems containing thousands or even millions of objects. On such massive scales, it is impractical to insist that every object instance be identified by a unique name. Rather, instances must be differentiated by their properties.

To populate itself, the Trader needs to be informed by candidate objects that they wish to be listed in its repository, or the Trader must actively seek out these objects. The CORBA Trader specification only supports the former (passive) mode, in which objects register themselves with a Trader. The Trader then stores these registrations in a database, indexing for all the relevant searchable properties.

The Trader offers interfaces to candidate server objects, allowing them to register themselves, as well as interfaces to client objects, allowing them to form predicates and submit searches. These interfaces are all specified in standardized IDL.

The CORBA Trader, as described, should appear familiar to any reader who has performed a search using Alta Vista or a similar facility. These search engines all have the ability to "Add a URL" (equivalent to an object registering itself with a trader). Many of them, given a URL, also perform a "web crawl" from that point to further index any hyperlinked URLs off the original. That is, they not only accept passive registrations but also actively accumulate more. Many of them invite the submission of keywords (along with a passive registration) and index further keywords found on web crawls. These keywords are stored along with the URLs themselves, and are searchable. Hence, the keywords correspond, somewhat crudely, to the property tuples of a Trader. It is a crude analogy because the only relation that can be performed on keywords is string equality or substrings, whereas genuine properties support all arithmetic, Boolean, and string comparative relations.

When it comes to storage capacity and efficiency of searching, the WWW search engines are truly awesome. The larger ones index millions of URLs and tens of millions of keywords and continue to grow in tandem with the WWW itself. Most miraculous, however, is the speed with which a search for, say, +celebrity - "nuptial agreement" can return matches.

However, the search engines lack a programmatic interface (or even a consistent user interface). They are all somewhat different not just in implementation, but in how they are accessed, how predicates are formed, and how URLs are registered.

So, an opportunity exists for synergy between the Trader specification and the Internet facilities. What the CORBA Trader offers the engines is a standardized programmatic API, represented in CORBA IDL. This is important when the items being sought from the search engines are not just interesting web pages, but intelligent objects that export functional interfaces; and the clients seeking them are not people using GUI interfaces, but client objects using APIs. Finally, the CORBA specification also generalizes the string keyword capability to more powerful predicates.

In return, the engines offer highly scalable datastores, blindingly fast search algorithms, and accumulated stores of server objects (well, web references anyway -- call them "pure data objects") that have already been categorized. They make a great match!

-- R.R.