Email	Print Reprint
	add to:
	Del.icio.us Digg Google Furl	Slashdot Y! MyWeb Blink

Before you begin consolidating your company's business logic, you must decide whether to integrate at the data, application programming interface, method, or user interface level.

So, what have I covered so far in this enterprise application integration (EAI) series? In the first installment, “Enterprise Application Integration From the Ground Up” (Apr. 1999), I explained the different types of middleware and where they work best. The second installment, “Mastering Message Brokers” (June 1999) examined how message brokers measure up to the challenge of EAI. While these technology components of EAI are significant, developers and enterprise architects must keep a much larger picture in mind. In this final installment of my EAI series, I’ll look at some of the higher architectural issues of EAI, including approaches as well as enabling technology.

It’s important to remember that EAI entails more than connecting applications. If that’s all there were to it, I would discuss application programming interfaces (APIs), message queues, and other connectivity solutions that exist only at the programming level. EAI is actually a new approach to integrating applications, which may revolutionize the way you plan new application development or think about re-engineering existing applications. Reading this article is only your first step to mastering EAI.

Undoubtedly, there are a number of instances of non-integrated systems in your enterprise today, possibly for inventory control, sales automation, general ledger, and human resources, to name just a few. These systems were typically custom-built with a specific business problem in mind for a narrow set of users, utilizing contemporary development technology—such as the COBOL and DB2 movement of the mid-1980s, the UNIX and C revolution of the late 1980s, and the client/server trend of the early 1990s. To make matters worse, these systems were largely built without any notion of interoperability. Integration simply was not a requirement for the developers at the time, and EAI entails fixing these past mistakes.

While the technology has aged, the business value of the applications remains constant. Indeed, systems built using older technology have remained critical to the enterprises they serve. Unfortunately, it’s difficult to adapt many of these business-critical systems to let them share information with other, more advanced, systems without a significant investment in time and money—not to mention risk.

Packaged applications such as SAP, Baan, and PeopleSoft—which are “closed applications” themselves—have only compounded the problem. Like custom enterprise systems, packaged applications were not designed to easily share information and processes.

EAI is a response to traditional enterprise application development and packaged application usage. The essence of EAI is unrestricted data and business process sharing among any of an enterprise’s connected applications or data sources. While the goal of EAI is clear, its procedures and technology are more a science than a process.

The notion of simply sharing information between applications is easy enough to understand. However, many enterprises need to share both data and methods without making significant changes to the source or target systems. In other words, EAI must join applications at an existing point of integration, saving the expense and risk of having to change, test, and re-deploy applications. This is known as noninvasive EAI, and is the focus of most early efforts.

Of course, noninvasive EAI only applies to a select set of problem domains, and does not provide the infrastructure for reusing business logic, such as sharing distributed objects or transactions. However, to properly bind applications at the business process level, developers must make sweeping changes to the participating applications, letting them share common methods and thus data. This is, of course, intrusive EAI, and while the benefits are high, so are the risks and costs.

Implementing EAI

When contemplating EAI in your organization, you must first understand the sum and content of the business processes and data in your organization. Your staff also needs to understand how these business processes are automated (and sometimes not automated) and the importance of all business processes. Depending on your enterprise, this may demand a significant amount of time and energy. In addition, many organizations seek new methodologies to assist them in this process while looking closely at best practices.

To successfully implement EAI, organizations must understand both business processes and data. They must select which processes and data require integration. This process can take on several dimensions or levels of abstraction, including data level, application program interface level, method level, and user interface level.

Data-Level EAI

Data-level EAI means moving information between two or more databases in order to integrate the applications. You use data level EAI to extract information out of one or many databases, perhaps transforming its schema and content so it makes sense to the database receiving the data, and then placing the data in the target databases. While this may sound simple enough, more complex data-level EAI problem domains may entail moving data between hundreds of databases of varying brands and models. The idea here is to avoid changing the application logic while moving data between applications. The relevant technology is traditional database middleware, data movement software (such as products used in the world of data warehousing), application servers, and message brokers.

Accessing data in the context of EAI demands doing an “end run” around application logic and user interfaces, as shown in Layer 1 of Figure 1. Applications and interfaces were not designed to accomplish EAI, but rather to work independently. As a result, successful implementation of data-level EAI requires sneaking behind the application’s logic and extracting or loading the data directly into the database through its native interface. Fortunately, most applications built in the past two decades or so decouple the database from the application and interface, making this a relatively simple task. However, this doesn’t mean that data-level EAI is always a good idea. You must consider how tightly coupled the data is to the application logic. Moving data between databases without understanding the entire application is a dangerous maneuver.

Figure 1: EAI Levels

Data-level EAI provides simplicity and speed-to-market, and is typically cheaper to implement than other forms of EAI such as application interface, method, and user interface. What’s more, you don’t need to employ new and expensive EAI tools like message brokers when you’ve limited the scope of your EAI solution to moving information between databases. Simple data replication and transformation tools may provide all the power you need.

Cost and simplicity are clear advantages of data-level EAI because you rarely have to alter business logic. As a result, you don’t need to endure countless testing cycles or the risk and expense of implementing newer versions of an application within any enterprise. Indeed, most users and applications won’t know that data is being shared behind the scenes.

Given the task’s conceptual simplicity and real life complexities, how does an enterprise implement data-level EAI? Ultimately, it comes down to understanding where the data exists, gathering information about the data, and applying business principles to determine which data flows where, and why. Many are compelled to create an enterprise-wide data model complete with meta data (data about data) to help EAI architects and developers better determine source data, as well as the potential target systems.

The numerous database-oriented middleware products that let architects and developers access and move information between databases simplify data-level EAI. These tools and technologies let you integrate various database brands, such as Oracle and Sybase. They also let you integrate different database models, such as object-oriented and relational models. However, if you’re looking to move information from the data level to the application interface method or user interface levels, consider using more sophisticated EAI technology such as message brokers or application servers.

API-Level EAI

API-level EAI, like data-level EAI, avoids the cost and risk of having to change the source and target applications by using existing application program interfaces that might exist within custom or packaged applications. Using these interfaces, developers can cohesively bind applications together, letting them share business information and business logic, albeit in a loosely coupled but noninvasive manner. The only limitations that developers face are the application interfaces’ specific features and functions, which vary from pretty good to nearly impossible to use. SAP R/3’s Application Linking and Embedding (ALE) interface is an example of an API. However, it would be a stretch to call SAP an open application with an easy-to-use interface.

Simply put, application interfaces are those that developers expose from a packaged or custom application to access the application’s various levels or services. Some interfaces are limited in scope, while many are feature-rich. These interfaces can let you access business processes or let you access the data directly. Sometimes they allow access to both.

Developers expose these interfaces for two reasons. The first is to let you access business processes and data encapsulated within the applications, without forcing other developers to invoke the user interface. Such interfaces let external applications access the services of these packaged or custom applications without actually changing the packages or applications. The second reason is to let you share encapsulated information.

Application interfaces as a type of EAI are distinct from method-level EAI (discussed next) and user interface-level EAI (discussed later). While you can distribute the methods that exist within enterprises among various applications, they are typically shared via common business logic-processing mechanisms, such as application servers or distributed objects. User interface-level EAI is similar to application interface-level EAI in that both make business processes and data available through an interface exposed by the package or custom application; user-level EAI simply does this through the existing user interface (for example, 3270, 5250, or OLE automation).

This approach distinguishes application interface-level EAI from other types of EAI. The potential complexities of the application interfaces, as well as the dynamic nature of the interfaces themselves, add to the difference. Because interfaces vary widely in the number and quality of the features and functions they provide, it is nearly impossible to know what to expect when invoking an application interface. Packaged applications are all over the map in terms of which interfaces they expose.

Packaged applications (which are most often present in a typical EAI problem domain) are only now opening up their interfaces to allow for outside access and, consequently, integration. While each application determines exactly what these interfaces should be and what services they will provide, there is a consensus in providing access at the business model, data, and object levels.

When accessing the business model, or the innate business processes, you typically invoke a set of services through user interfaces. For example, you can access credit information for a particular individual through the user interface by driving the screens, menus, or windows. You can also access this information by invoking an API provided by the packaged application vendor.

In the world of custom applications, anything is possible. With access to the source code, you can define a particular interface or simply open the application with a standard interface. For example, rather than accessing the user interface (scraping screens) to reach an existing COBOL application residing on a mainframe, you can build an API for that application and thus expose or extend its services. In most cases, this requires mapping the business processes, once accessible only through screens and menus, directly to the interface.

This approach’s downside is cost. Half the costs go toward application changes and the subsequent testing and redeployment. In many cases, it is much cheaper (and just as effective) to simply access the application information by automating access to user interfaces from a program. This is, of course, user interface-level EAI.

If the world were perfect, all the features and functions provided by packaged applications such as SAP, PeopleSoft, Oracle Financials, and Baan would also be accessible through their well-defined APIs. However, the world is not perfect, and the reality is a bit more sobering. While almost all packaged applications provide some interfaces, they are, as mentioned already, uneven in their scope and quality. While some provide open interfaces based on open interface standards such as Java APIs (for example JavaBeans) or object request brokers (ORBs), many provide more proprietary APIs that are useful only in a limited set of programming languages (for example, COBOL and C).

Most disturbing, many packaged applications don’t offer an interface. With these applications, an application or middleware layer can’t access the services of that application. As a result, the business processes and data contained within the application remain off-limits.

Method-Level EAI

Method level EAI integrates applications by binding them together at the method level, as shown in Figure 1, Layer 2. This is, as described earlier, the most invasive form of EAI, requiring that major changes take place within all participating applications. We can do this through any number of traditional application method-sharing technologies such as distributed objects or application servers. This generally means creating a hybrid or composite application, which can provide the infrastructure for accessing shared business processes.

Attempts to share common processes have a long history, starting more than 10 years ago with the distributed object movement and multi-tiered client servers—a set of shared services on a common server that originally provided the infrastructure for reuse, and now facilitate integration. Reuse is important in this context. By defining a common set of methods, you can reuse those methods among enterprise applications. This significantly reduces your need for redundant methods and applications.

Absolute reuse, or the tight integration you get from reuse, has yet to be achieved on the enterprise level. The reasons for this failure range from internal politics to the inability to select a consistent technology set. In most cases, the limit on reuse is the lack of enterprise architecture and central control. So what does this failure of reuse have to do with EAI? The answer is: everything.

Using the EAI tools and techniques to integrate an enterprise not only helps you share common methods, but also provides the infrastructure to make such sharing a reality—integrating applications so that information can be shared and providing the infrastructure for reuse. This might sound like the perfect EAI solution, but the downside is that it’s also the most invasive level of EAI. Unlike data-level and application interface-level EAI, which do not typically require changes to either the source or target applications, method-level EAI requires that you change many, if not all, enterprise applications.

In the world of EAI, you’ll also hear a lot about composite applications. Composite applications are nothing more than applications bound together through a common set of services. This, of course, has been the ultimate goal of distributed objects and transactional technology for years, and is certainly the goal of method-level EAI. So, make no mistake: we are walking down the same alley here.

The enabling technology for method-level EAI is any product or standard that lets applications invoke each other’s methods or access shared methods on a central server. Examples of this technology include distributed objects, based on either CORBA or COM, and the new breed of application servers. Developers implement these solutions by creating composite applications or by changing existing applications to let them share methods.

Considering the invasiveness and expense of method-level EAI, you should understand its opportunities and risks clearly when assessing its value. Sharing common business logic between many applications—and so integrating those applications—is a tremendous opportunity. However, it comes with the risk that the expense of implementing method-level EAI will outpace its value.

User Interface-Level EAI

User interface-level EAI uses the user interface as a noninvasive point of integration, as shown in Layer 3 of Figure 1. For instance, this approach lets you integrate existing mainframe applications through their 3270 terminal interfaces, typically when no other points of integration, such as the database or API, are available.

This process uses windows and menus to access the relevant data that must be extracted and moved to another application or data store. While it sounds like an inefficient and perhaps even ill-conceived approach—something of a stop-gap measure—there is a great deal of it going on.

Other EAI levels (as described earlier) might be more technologically appealing and efficient, but in many applications, the user is the only available mechanism for accessing logic and data. In spite of its inefficiency in getting into an application, the user interface has the advantage of not requiring any changes to the source or target application.

In the context of user-level EAI, the user interface is the EAI interface. In a process known as screen scraping, or accessing screen information through a programmatic mechanism, middleware drives a user interface (for example, 3270 user interface) to access both processes and data.

For the reasons I noted earlier, user interface-level EAI should be your last-ditch effort for accessing information from older systems. You should turn to it only when there is no well-defined application interface (for example, SAP’s ALE), such as those provided by most ERP applications (albeit some ERP vendors, such as PeopleSoft, do scrape screens as their interface mechanisms), or when it doesn’t make sense to leverage data-level EAI. Having said that, you needn’t avoid user interface level EAI completely. In most cases, it successfully extracts information from existing applications and invokes application logic. Moreover, if you perform user interface-level EAI correctly, there’s virtually no difference between the user interface and a true application interface.

The enabling technology for user-level EAI has been around for years. Most 3270 emulators, for instance, provide APIs to access screen information. Moreover, middleware vendors, such as application server and message broker vendors, provide connectors to user interfaces that let developers convert screen information directly into messages for transport to other target systems and hide the complexities from middleware users.

EAI Evolves

The future of EAI depends on whom you consult. While some see EAI evolving from plumbing and middleware to process automation, many developers believe that portals and XML will drive EAI in the near future. A new, momentum technology might also appear on the horizon.

At first, most EAI will occur through noninvasive integration projects, binding applications together at the database, method or user interface levels. I am already seeing this trend. However, the future of application integration is method-level EAI, creating a composite application to bind applications together. Clearly, the development industry likes taking baby steps into any new approach and technology, and moving from noninvasive to invasive is only natural.

However, the industry has yet to get the plumbing right to really solve the EAI problem. While many middleware vendors claim to possess the ultimate EAI solution, there’s a long way to go before developers can effectively integrate all applications within an enterprise. Lingering problems include application interfaces that require new programming and maintenance, as well as the inability to provide a single solution that binds applications at both the data and method levels. Thus far, you’ll have to roll your own solution using “best of breed” middleware technology. Many organizations consider that a high-risk approach.

The middleware market needs to consolidate around EAI. While message brokers are good at moving information in real time from system to system, application servers are better at providing the infrastructure for sharing common business processes. Application server and message broker vendors are working together to address this problem, but they must create a more tightly coupled solution to a typical EAI problem domain.

The next frontier is integrating EAI with process automation and workflow. This means creating a set of processes on top of your enterprise’s existing processes, and binding them together to automate existing business processes. While developers have been solving workflow problems for years, integration with middleware, and now EAI, is more recent. Clearly, this is where EAI is heading—that is, if we can solve the plumbing problem.

Extensible markup language, or XML, a transplanted web standard, may simplify EAI. As the enterprise integration problem became more evident, EAI architects and developers saw the value of using XML to move information throughout an enterprise. Even more valuable, you can apply XML as a common text format to transmit information between enterprises, supporting supply chain integration efforts. As a result, many are calling XML the next EDI (electronic data interchange).

XML provides a common data exchange format, encapsulating both meta data and data. This lets different applications and databases exchange information without having to understand anything about each other. To communicate, a source system simply reformats a message, a piece of information moving from an interface, or a data record as XML-compliant text, and moves that information to any other system that can read XML. However, XML is still too new, and we’ve yet to understand how well it will fit with EAI. It’s a text-processing standard that everyone can agree on, and not much more. The jury is still out on using XML as a common messaging and data storage standard.

However you think EAI will evolve in the future, it’s important to keep your sights on the business problem it’s trying to solve. Applications created over the years continue to have value—and as the cost of redeveloping those applications rises, many enterprises are making the cost-effective decision to integrate rather than rebuild critical business systems. You can count on this trend to continue as the tools and technology become more powerful and developers and enterprise architects learn more about them.

RELATED ARTICLES Wind River Hypervisor 1.1 Released Parallel Pattern 5: Stencil Distributing Work Across Cores Using .NET Other Voices: Knowledge Management Basics Inform System Release Lifecycle Server+client Fusion: An Architectual Proposal		TOP 5 ARTICLES No Top Articles.