John Hebeler is a division scientist at BBN Technologies. Andrew Perez-Lopez is a Software Engineer at BBN Technologies. They are co-authors of Semantic Web Programming.
Service-Oriented Architecture (SOA) is the de-facto architectural standard for many small and large distributed solutions. Its growing relevance is reflected not only in the bevy of rich services the SOA stack offers, but also the variety of available implementations, many of which are open-source. Yet SOA remains challenged in several areas dealing with scale, growing complexity, diverse knowledge management, and the inherent dynamics of large, distributed systems. These challenges unnecessarily constrain SOA solutions causing them to fall short of their ultimate potential.
Semantics can help. Semantics form expressive, useful computing abstractions or concepts that address these challenges and strengthen the SOA mission to use and leverage complex information and services across an entire enterprise or the globe.
In this article, we examine adding semantics to SOAs through four examples, each contributing in different ways yet adhering to the SOA standards and goals. Two examples illustrate "SOA-ize" semantic services and the other two "Semantic-ize" the SOA. All contribute to improving the scope and capabilities of what is possible with a SOA solution.
SOA exists as a set of standards and services that form a dynamic distributed service fabric that provides a step forward into distributed computing possibilities. The interface standards allow services to interact despite fundamental platform differences. SOA and services is analogous to the World Wide Web (WWW) and documents -- the WWW exists as a set of standards that allow document contributions based on many technologies and techniques all following the same document standards. SOA attempts to do the same with services. Whereas WWW standards unleashed a world of information, SOA attempts to unleash a world of services. However, services can be much more complex and temperamental than documents (and documents themselves have gotten pretty complicated). The SOA stack addresses some of this complexity by integrating a set of management services. The vastness and richness of the network services can easily overwhelm a SOA solution even with a powerful stack of services.
Theoretically, a SOA implementation can exist at a variety of levels from a small solution integrating a handful of services up to a large, dynamic solution integrating thousands of services. In practice, SOA strongly leans to the former -- integrating a few, well-controlled services. Yet the real power and possibilities require a much bigger leap into the thousands. That leap incurs a multitude of challenges including service complexity, information complexity, and service management complexity.
Services are the fundamental building blocks of a SOA solution. Service complexity refers to the degree of sophistication associated with the tasks the service actually accomplishes. Whereas a document is merely a bunch of organized words, services do things -- potentially very complex things. Additionally, they are dynamic. Services are often bound to the objects that drive their behavior. But these objects themselves can limit the ultimate complexity of the overall application. Objects or instances are bound to their creating class and often that class is static. Attributes and relationships are typically also bound to the creating class. Too often in object oriented software for services, there is little analysis of the object structure as the underlying data model. Is it consistent? Is it logical? Is it cohesive? These answers are left solely to the programmer, and the hard-coded software. Complex behavior frequently yields complex code -- often unreadable code. Some tools exist to help externalize this inherent complexity but most do not. Programming languages can also limit the complexity and thus the power of individual services. SOA solutions integrate multiple services, which, of course, compound this phenomenon.
Information processing is often at the heart of a SOA solution or almost any computer solution -- information is the "crown jewels" of computing. Just as the complexity of services stands as a challenge to SOA system designers, so too does information complexity. Most data technologies incorporate limitations that force the information to reside in multiple places and to be reassembled to produce the desired results. A typical way of addressing this data distribution issue is with XML. Databases, combined with business logic, produce data elements that can form the desired answer in XML. XML is a step in the right direction, but it struggles with complex relationships and dynamic constraints. The actual meaning of the data, the information, is separated in the document and in the XML schema, or even worse, the schema of the database back-end, and therefore very difficult to manage. These limitations of the information format and knowledge representation force all of the services in the enterprise, which should be focused on business logic, to handle a higher degree of complexity just to get started. This, of course, further aggravates the service complexity challenge outlined above.
SOA solutions enable the dynamic integration and collaboration of multiple services. This requires service management -- and this can be very challenging. In a real sense, the integration of services forms new, more complex services. We examine three aspects of service management:
- Service abstractions
- Service registration
- Service dynamics
Each represents a major aspect of service management.
Service abstractions describe the services. The richer and more complete the abstractions, the more easily the service is found and properly used. Most SOA services exist as low-level or simple abstractions. This low level forces the programmer to take up the slack and really understand the service beyond the simple abstraction descriptions. For example, say a software developer determines the need for a service, like a catalog lookup. Given a high-level abstraction like "catalog lookup" the developer would, in an ideal SOA world, find a service and be able to integrate it easily into an application at that high level. However, as we in the real world know, to take advantage of the catalog lookup service, a SOA service too often requires inspecting complex APIs defined in Web Service Description Language (WSDL), which identifies particular end point names, the numbers and types of variables, and return values for each supported method. Even those detailed WSDL descriptions, however, do not describe how the parameters serve the method, the purpose of the method, or any expectations with regard to errors and performance. This forces you, the programmer, to invert your development process in an unnatural way. Instead of finding a service that matches the needs of your application, you must design your application to match the requirements of the services. In other words, the services don't really serve your specific need, they serve their own needs in their own particular way. You must call and use the service according to the service's usage patterns and not your own. Of course, all of this is moot if you design all the services, or if all of the assumptions are shared by every service, etc -- but that is extraordinarily limiting in terms of the services that your application can exploit, and is really contrary to the goal of large-scale SOA. The key is inverting this perspective and enabling a service to align with your system's needs. Thus, your SOA system accesses the services or aggregation of services at the level of abstraction that makes sense to your needs. This is somewhat analogous to views for a relational database. This service view must also accommodate similar services, failed services, and so on. Ideally, you just want your overall system to operate correctly and not have to change drastically to accommodate the specific syntax or other dependencies of a particular service implementation.
Service registration remains a real challenge for SOA. How can you compose the best SOA without full knowledge of what services are available and how to interact with them? The Universal Description, Discovery, and Integration (UDDI) standard has fallen short of realizing its larger goals. The inherent complexity of using XML to represent such rich knowledge has produced an unwieldy solution that befuddles many developers. Many SOA solutions now register their services in hand-carved proprietary solutions or use no UDDI implementation at all, and some even try to use a Wiki. This is a direct blow to the scale of SOA efforts. A quick look at programmableweb.com illustrates this problem. This site lists thousands of web services, yet only a handful are used to support almost all the mashups and the vast majority of mashups use only two or fewer services (with a 40% chance that one of these is Google Maps). Sadly, most offered services are not used in any mashups. This situation could be the result of inadequate services but it is more likely due to the inability of developers to find and utilize the services they want.
For SOA to work properly, a service registration system must clearly describe and search a vast array of different service offerings available in the given SOA. These requirements demand a solution that is more than just a common set of syntaxes. Explicit syntax agreements are important, for without them nothing would work. However, determining a common syntax (or set of syntaxes) only allows exact matches or at best a basic category search. This works fine for services that you control (and name) but does not stand up to the challenge of integrating services that originate and evolve from many sources. Actually, for services you create, the registry is not even that important -- you already know the descriptions. However, large-scale integration needs much more than a common syntax -- it needs a common semantics. Capturing the information about what a service is and what it does and communicating that information effectively in an automated way remains a key challenge for large, dynamic SOA implementations.
Service abstractions and registration contain another inherent challenge -- dynamics. Over time, services change -- that is a simple rule, and they usually change for the better. Traditional SOA techniques often create brittle connections to services that may break on the slightest change to the service. Again, this is no trouble if you develop and control the services, but again, that is contrary to SOA's goal. This inhibits the use of services to two types: those that you control or those that are so big they will never (or at least very rarely) change. This set describes only a small fraction of the available services, and requires that SOA applications have regular, and often significant, maintenance. Without better ways to deal with the dynamics of web services, the growth of an SOA solution will remain stunted and more costly.
These SOA struggles result in a reduced awareness of what is available, a limited capacity to compose services, and a diminished exploitation of valuable services due to uncontrolled changes. In an ideal world, SOA could seamlessly link services around the globe, but these factors significantly threaten that vision.