Michael Pilone is a senior software engineer for Vangent where he is currently working on Java-based web service technologies and large scale geographical data indexing and searching.
It wasn't too long ago when it would be common to see a notice on your favorite website that it is "Currently Down for Scheduled Maintenance". However, with the push toward all data in the cloud and more web "applications" over the past few years, users are demanding constant availability and access to data and resources. Just look at the Internet outcries when Twitter or Google gmail goes down for more than a few minutes and you can understand the importance of building highly available systems. In this article, I present examples of how to configure and test Java web service and enterprise applications to leverage Oracle Real Application Clusters (RAC), Oracle Universal Connection Pool (UCP), and Fast Connection Failover (FCF) to provide high availability.
Oracle RAC is a clustering solution that allows you to present multiple interconnected Oracle databases as one server to the end user or applications. Oracle RAC uses Oracle Clusterware to link together multiple servers to create one large single database. Normally there is a one-to-one relationship of an Oracle database to an instance when running on a single server; however in a RAC environment, there is a one-to-many relationship where a single database is simultaneously hosted on multiple instances. The obvious advantage is that any one of the instances could fail while the database remains available on the other instances in the cluster. There are other advantages to RAC, including performance scaling and workload distribution, however a full discussion of RAC is beyond the scope of this article. For more information, refer to the Oracle Real Application Clusters Administration and Deployment Guide.
With Oracle RAC in place, database client applications can leverage a number of high availability features including:
- Fast Connection Failover (FCF): Allows a client application to be immediately notified of a planned or unplanned database service outage by subscribing to Fast Application Notification (FAN) events.
- Run-time Connection Load-Balancing: Uses the Oracle RAC Load Balancing Advisory events to distribute work appropriately across the cluster nodes and to quickly react to changes in cluster configuration, overworked nodes, or hangs.
- Connection Affinity (11g recommended/required): Routes connections to the same database instance based on previous connections to an instance to limit performance impacts of switching between instances. RAC supports web session and transaction-based affinity for different client scenarios.
These features are enabled in a Java application through the use of the Oracle Universal Connection Pool (UCP) and, other than configuration, require no special support code.
Oracle Server Environment
Before any high availability (HA) work can be done on the client side, a few key Oracle server components must be in place. This article is not meant to be a guide to installing and configuring Oracle RAC (there are many books on the subject) but there are a few key features that must be enabled by your database administrator (DBA).
Obviously the RAC environment must have at least two nodes available for testing. Your DBA will give you the hostnames for these nodes but you can assume that they will be something like "sales1node1" and "sales1node2". These are the two hosts that your Java client will connect to and therefore need to be running the standard Oracle listener process. Your DBA should be able to tell you the port number that the listeners are bound to (usually 1521).
Oracle databases are represented to clients as a service. A single database can have multiple services associated with it which are defined by service names, such as sales.foo.com or finance.foo.com. The service names should not be confused with the listener host names even though a service name may look like a host or domain name. In a RAC environment, different instances in the cluster can offer the same service to clients which allows for failover if one instance goes down, distribution of load based on the capabilities of the hardware, and service migration for planned downtime. Therefore it is important that the service name and not the instance name (i.e., Oracle System Identifier (SID)) is used when configuring the client. Figure 1 shows how two different instances (sales 1 and sales 2) can both offer the same service (sales.foo.com) while only one instance (sales 2) is offers a second service (finance.foo.com).
The Oracle RAC Load Balancing Advisory must be configured with service-level goals for each service for which load balancing is enabled. Enabling the advisory allows the RAC nodes to advise clients on how to distribute work across the cluster to ensure optimal use of resources and to avoid slow, hung, or down nodes. The Oracle Real Application Clusters Administration and Deployment Guide contains the details for configuring the advisory but the basic process is to execute a SQL statement that associates a load balancing goal with a service such as:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'sales' -, goal => DBMS_SERVICE.GOAL_THROUGHPUT -, clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT);
The final server side piece is the setup of the Oracle Notification Service (ONS) which propagates Fast Application Notification (FAN) events between the Oracle RAC database and the JDBC connection pool. On the client side, ONS can be configured in a local or remote configuration with remote being preferred because it eliminates the need for a local ONS server process on the client machine. To support remote connections to the ONS server, the DBA must modify the ons.config file on the database servers to set the remoteport and the list of ONS nodes. Your DBA should be able to tell you the remote port to use when configuring remote ONS support on the client side.
Now that the Oracle server side setup is complete and you obtained all the needed information from a DBA, you can test the FCF and ONS setups and configure the JDBC clients.
Most recent comment
RAC vs. DataGuard?
Comment by kallhands481 Aug 11, 2010, 12:28 PM EDT
Very insighful/useful article...thank you. I'm not sure if it will directly apply to our development efforts, and am hoping you can provide some additional info.
Our data environment implements DataGuard 10g w/ fast failover. Are there significant differences in failover events/messages from DataGuard and RAC that will need to be taken into account?