Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

JVM Languages

Can J2EE Manage Cache and Pool Memory?


Oct01: Java Q&A

Art is a member of IBM's WebSphere team. He can be contacted at [email protected].


One of the most important features of Java is garbage collection (GC) — the ability of the Java run time to gather objects no longer in use by applications and free their memory for use elsewhere. In most situations, garbage collection greatly simplifies application logic. Unlike C++ applications, Java apps do not have to explicitly keep track of which objects aren't needed and can be freed. Or so the story goes.

In truth, there are a number of special cases where memory management still must be handled by the application. One such case is when your application implements data caches or pools of objects. Data caching is done to avoid the repeated execution of some costly processing. Your app needs some chunk of data and needs it at several different places and times. Fetching this data involves a relatively costly operation such as a database query, remote method invocation, or huge calculation. You must do the fetch once initially to get the data, but you incur memory costs by keeping a local copy of it in order to save the cost of refetching it each time you need it.

Object pooling avoids the repeated creation and destruction of a particular class of object. Say your application needs to frequently create Foo objects. The Foos are used relatively briefly, then abandoned and garbage collected. You find that the application performs better if you save and reuse abandoned Foos rather than creating new ones. Reusing a Foo saves both the cost of destruction during GC and of creating a new object; the only remaining cost is reinitializing the Foo's attributes. According to Practical Java, by Peter Haggar (Addison-Wesley, 2000), and my experience with poorly performing Java apps, one of the top nonpathological reasons for poor performance is excessive object creation/destruction.

Caching and pooling are potentially even more important for web applications than for standalone apps. A servlet request (or a request to an EJB server) runs on a box shared by many users. Likewise, although each server request runs in its own thread, a request still shares a Java Virtual Machine (JVM), and thus shares memory with many other requests.

Caching and pooling share two key design aspects. First, it is possible (often inevitable) that the number of cached data or pooled objects grows to a point where it impacts available memory. This requires your application to manage memory by freeing objects on a regular schedule. If your application doesn't, you'll eventually see an OutOfMemoryError. Secondly, no matter which object has been freed, it can always be recreated on demand by refetching the data or creating a new object for the pool to give away. During the time the object does live in the cache or pool, you save significant processing cost and so the caching/pooling effort is still worth it.

But what about the effort to write memory-management code over and over? It isn't trivial to write code that keeps track of which objects are in use at the moment, which can be freed if necessary, and how often to preemptively free them. This latter logic is especially tricky. If you free objects when there is not a real need for memory, you lose some of the performance benefit of caching or pooling. If you wait too long, some part of your application (or worse, some part of someone else's) gets an OutOfMemoryError.

But isn't tracking object use and periodic freeing of objects what the Java garbage collector does? Why not hook the GC? As of JDK 1.2 (or Java2, J2EE, or whatever), core Java's java.lang.ref package lets you do just that. The classes you want are SoftReference, WeakReference, and WeakHashMap. (PhantomReference, also in java.lang.ref, is a completely different animal. It is not related to memory management per se. PhantomReference is used to do finalize()-like logic in a more flexible way than overriding the finalize() method; see the javadoc for details.) The first two classes are used as an alternative to a more typical object reference in your app. For example, a typical class Bar that includes a reference to an object Foo might look like Example 1(a). (Now that there is more than one kind of reference, I will refer to this kind as a "strong" reference.) If that same class instead held a SoftReference to its Foo, it would look like Example 1(b).

A Bar that held a WeakReference to its Foo would look the same as for SoftReference, except the WeakReference class would be used instead of SoftReference. In both cases, the Soft/WeakReference object acts as an indirect reference to a Foo. The Reference object is held by the owning class Bar; to get to its Foo object, Bar invokes myFooRef.get(). (By the way, the Foo object is referred to myFooRef's "referent." And myFooRef is called the Foo's "reference.")

The Java garbage collector is aware of all instances of WeakReference and SoftReference, as it is aware of all instances of any class. More to the point, the GC knows that it can free the object referred to by a WeakReference or SoftReference if it runs short of memory. The GC frees up completely abandoned objects first, but if it still needs memory to satisfy an outstanding request, it begins freeing softly or weakly referenced objects until it satisfies the need. Only if the GC cannot satisfy the need in this way will it throw an OutOfMemoryError.

This is a key point. That your application can help avoid an OutOfMemoryError by using Reference objects is the hook into GC you wanted. Using Soft/WeakReference lets the GC keep track of certain objects in your application, freeing them only when there is an actual memory shortage. Your application holds onto your objects as long as possible, yet does not cause an OutOfMemoryError due to otherwise unbounded growth. Performance is comparable or better overall than if you had private memory management code, since the necessary logic is the same but the triggering and reference list keeping is done once by the GC instead of twice by you and the GC.

Besides the one additional hop to get to your Foo object, there is an added cost to using References. How do you know whether the GC has harvested one of your objects? Whenever you call myFooRef.get() to retrieve your actual Foo object, you must check if the return is Null. A Null from get() indicates that the GC has been there and you must fetch your data again (in the case of caching) or look elsewhere in the pool for an object to reuse.

References: Soft Is Better Than Weak

The GC treats SoftReference and WeakReference somewhat differently. Each is aimed toward helping in a different situation. The javadoc for WeakReference indicates that it can have its referent freed at any time, even if there is a strong reference to it somewhere. This would make WeakReference a poor candidate for any object that needs to stay for any length of time, since the object could vanish between getting it and calling a method on it. In fact, on Windows anyway, I found that a strong reference protects the referent of a WeakReference just as it does for a SoftReference.

However, WeakReferences prove to have another objectionable behavior — they aren't freed as needed, as SoftReferences are. The test program WeakReferenceTest (available electronically; see "Resource Center," page 5) shows, when one WeakReference referent is freed, all other known WeakReference referents appear to be freed as well. Even recently used referents are freed. This makes WeakReference unsuitable for caching or pooling because the longer you keep objects around, the greater the performance benefit.

A SoftReference can have its referent freed only when there are no strong references elsewhere. This makes SoftReference good for pools, which pass your objects to other classes for use, then take them back when they are no longer needed. If a pool holds its collection of not-in-use objects by means of SoftReferences, the GC can free up objects that may have been created during a spike of need, but now sit around unused. It is proper that the GC cannot touch in-use objects in a pool because of the client's strong reference to the object. An important side effect of this is that the "extra hop and check for Null" logic need only be done by the pool itself; a client using a pooled object has a strong reference to it and treats it like any other object.

SoftReference is also good for cached data, even when the referent object isn't in a collection, but is owned by some specific object (such as Bar in previous examples). When a method in Bar needs to use a cached data object Foo, which is held via a SoftReference, the pattern of use looks like Example 2. The only strong reference to myFoo is a local variable that is freed when the method completes. The presence of the local strong reference keeps myFoo around for the balance of the method logic, but lets myFoo be garbage collected between calls to the method. Even if more than one thread is executing this method at the same time, myFoo stays around until the last local strong reference is freed. Any other necessary synchronization, such as to control writing to myFoo by two threads, is still up to you.

Lastly, SoftReference is good for both caches or pools because the GC frees them only as needed. As SoftReferenceTest (also available electronically) shows, the GC frees only enough SoftReference referents to satisfy the outstanding memory requests (with some rounding). In addition, the GC may optionally implement least-recently-used (LRU) logic, which says that the least-recently-used referents are freed first. SoftReferenceTest demonstrates this. For example, Windows 2000 version 5.00.2195 includes LRU logic.

What You Can Do for Your GC

Again, the way to tell if a referent has been garbage collected is to call SoftReference.get() and look for a Null. This means that the SoftReference objects must stay around. Although SoftReference is not a large class, enough of them accreting eventually exhausts memory — exactly what you were trying to avoid. Have you just shifted the problem?

No. The java.lang.ref package includes the ReferenceQueue class that acts as the anchor to a linked list of References. The References on the queue are those with referents that have been freed. At any point in your application (say, when you are getting an object from a pool), you can pull References off the queue and remove these dead References from your pool. Yes, this should guarantee that you never get a hit on your "check for Null" logic, but paranoid me would leave the check in anyhow. Some data cache designs also benefit from this ability; if you hold your data objects (that is, their SoftReferences) in a collection, then ReferenceQueue is for you.

Once removed from your collection, References get garbage collected like any other object. However, you cannot reuse Reference objects (Soft or otherwise) because of the design; you can set the referent in the constructor. I have not dug into the GC code in this area, but this is probably to make the tracking/freeing process faster. Since, typically, the referent objects are larger than the Reference objects, the net gain is still substantial.

Do you have to do anything special with referent objects to make them work with this framework? The answer is mostly no. The one thing that must be true is that the referent objects have either no finalize() method or that the finalize() logic makes no memory demands. Remember that a referent is garbage collected because memory is in short supply at the moment. Adding to memory demands — by even so much as manipulating a String — probably causes a cascade of memory demands as referent after referent is garbage collected trying to meet the demand. If, on the other hand, your finalize() is attempting to free additional objects, then it should work just fine.

Another special consideration is that the referent objects should (typically) be larger than the SoftReference objects you are creating for them. It is not effective memory management to save memory by doubling your need for it. A SoftReference object is approximately O+S+32 bytes in size, where O is the minimum size of a Java object (the size of the Java root object Object plus the size of a single subclass with no instance variables), and S the delta size of an additional subclass with no instance variables. Say, for example, you are pooling Foo objects. Foo is a subclass of Poo. Foo and Poo each have two instance variables of type long (64 bits, or 8 bytes). In this example, a Foo object is approximately the same size as a SoftReference and it may not be worth it to use a SoftReference for each Foo.

Instead, you may choose to have several FooPools, each referred to by a SoftReference (the FooPools hold the Foos directly, not via References). GC is able to free up an entire pool if it needs space, and your code can continue to use the remaining pools. You can create new pools later as needed. This technique gives all the advantages of using SoftReference while controlling the need for extra objects. The maximum number of elements in an individual pool could even be calculated dynamically and/or tuned based on the size of the objects being pooled.

The Sample/Test Programs

I used the accompanying JAR files (available electronically) to test whether SoftReference works as advertised. SoftReference claims three specific behaviors that need to be verified:

  • When a JVM/process needs memory, the GC will free up objects that are only referenced by a SoftReference object. As long as such objects exist and are of sufficient size to meet the memory needs of the JVM, no OutOfMemoryError is thrown.
  • Prior to the object actually being freed, the reference to it in a SoftReference (one or more) is set to Null. This way, application code can tell that the object is no longer available without causing a NullPointerException.

  • Objects chosen by the GC to be freed are selected on an LRU basis.

The tests I did used a while(true) loop to create SoftReference objects and put them in a List. Each SoftReference object holds an instance of HugeObject, a class created for the test that takes up many thousands of bytes for each instance. Each SoftReference object (actually, a CountableSoftReference, a class written for the test) is assigned a number at creation, ranging from 0 to whatever. This lets you know the last time a given SoftReference object was used (when it was created), thus verifying if the LRU scheme works properly. Eventually, enough HugeObjects are created by the loop to exhaust available memory. This should trigger the promised behavior.

It is awkward to test the three behaviors simultaneously because, if the first one proves to be true, then the test app runs forever. Attempts to execute tracing code during the object freeing process (for example, in a finalize() method on the object) cause additional memory demands, which then can cause an OutOfMemoryError. Consequently, I used two test cases. Both test cases are to be run in an actual JVM; the test will not run properly — that is, will not show the proper results — under most debug tools. To run the test cases, J2EE 1.2.2 must be installed and be in your CLASSPATH. To execute either of the tests, enter the following on a command line: java -classpath .\softreferencetest-XXXXX.jar;%CLASSPATH% SoftReferenceTest.SoftReferenceTest [number], where XXXXX completes the name of the particular JAR file you wish to run, and number lets you tune the test program to your system's available memory to properly demonstrate the LRU logic (or lack thereof) in the JVM. Each HugeObject is assigned a number from 0 to whatever, representing the order in which it was created during the test. The parameter number is the number of HugeObjects that is too high to have been garbage collected, yet is low enough to have been created before you ran out of memory and triggered the end of the test. The default number is 30.

The JAR file softreferencetest-stopAfterFirstGC.jar verifies the second and third behaviors. The expected output (using System.out.println) looks like the file softreferencetest-stopAfterFirstGC.lst (available electronically). The test halts when the GC is freeing the first HugeObject. In this test, HugeObject does have a finalize() method and that method triggers dumping of all information and termination of the test. The fact that the finalize() method is called at all partly verifies the first behavior. As you can see from the test output, a number of SoftReferences now point to Null (verifying the second behavior). The reason you see many of them pointing to Null, but only one actual HugeObject being garbage collected, is that it is a two-step process; a group of SoftReferences have their pointers set to Null and then the now-abandoned HugeObjects are garbage collected. (Remember that you halted the test when the first HugeObject was about to be garbage collected.)

As you can also see from the test output, it was the LRU SoftReferences that were pointed to Null. The smaller the ID number on the SoftReference, the earlier it was created and, thus, the "least-er" it was used (except for those that were touched at a later time). This verifies the third behavior.

The other JAR file, softreferencetest-forever.jar, completes the verification of the first behavior. The expected output looks like softreferencetest-forever.lst (available electronically). It runs forever because it is designed to not interfere with the first behavior; here, HugeObject has no finalize() method. As long as a real application object does not have a finalize() method that does anything requiring memory, it too can be used with SoftReferences in this manner. While this test is running, open the Windows Task Manager to the Applications tab and watch CPU Usage and Memory Usage. The latter repeatedly rises, then drops to a lower value. Each time it drops, the hourglass cursor appears. This is when the Java GC is running, freeing up memory. When you are satisfied SoftReferences work correctly, end the task and review the output.

I have included electronically a couple more JAR files (weakreference-teststopAfterFirstGC.jar and weakreferencetest-forever.jar) that demonstrate the not-as-useful behavior of WeakReference with regard to caching and pooling. These execute the same tests as described earlier, but use WeakReferences instead of SoftReferences. To execute either of the tests, enter the following on a command line: java -classpath .\Weakreferencetest-XXXXX.jar;%CLASSPATH% WeakReferenceTest.WeakReferenceTest [number].

Conclusion

The miniframework provided in java.lang.ref provides excellent logic for managing memory usage in application data caches and object pools. Still, you must apply good design principles and some special considerations to make best use of this framework. The result is a significant net savings in the code you have to write and equivalent (or better) performance than if you had written the reference tracking, triggering, and freeing logic yourself. Sun got this one right.

DDJ


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.