Channels ▼


Stateful Web Applications that Scale Like Stateless Ones

The Real Problem: Stateless Applications Eat Developer Brains

The elephant in the room in all stateless architectures is that there is no such thing as "stateless." Pushing state out of the application server doesn't get rid of that state, it just forces you to manage it somewhere else. And, it does so at the expense of your most precious commodity: your poor developer brain. It's hard to write stateless applications. Even if it doesn't seem hard to you, the plain fact is that while you are busy dealing with the effects of a stateless architecture, you could instead be implementing new features and improving existing ones.

The reason externalizing data is hard is because all of the work of managing object state that the JVM used to do for you is now your explicit responsibility. You must now tell the clustering engine that you've made a change so that it may commit your changes to the cluster. And, before reading an object's state, you must remember to check it out of the clustering service.

Managing state data by hand is hard. That's why the JVM goes to so much trouble to do it for you.

Enter Terracotta: Code Like Your Mom Used to Make

What if you didn't have to do any of this funny business to get scalability and reliability? What if the JVM had access to a service that you could plug into to make its heap durable, arbitrarily large, and shared with every other JVM in your application tier? Enter Terracotta, network-attached, durable virtual heap for the JVM. In the spirit of full-disclosure, I'm a co-founder of Terracotta and work there as a software developer.

Terracotta is an infrastructure service that is deployed as a stand-alone server plus a library that plugs into your existing JVMs and transparently clusters your JVM's heap. Terracotta makes some of your JVM heap shared via a network connection to the Terracotta server so that a bunch of JVMs can all access the shared heap as if it were local heap. You can think of it like a network-attached filesystem, but for your object data; see Figure 1.

[Click image to view at full size]
Figure 1

The Terracotta server handles keeping your clustered object data persistent and coherent and coordinating threads across attached JVMs. The Terracotta client libraries use standard bytecode manipulation techniques to make your existing code cluster aware. All of this is done transparently, driven by a declarative configuration file or through annotations, not through an intrusive API.

Terracotta manages the clustered virtual heap the same way the JVM manages the physical heap. Field updates to clustered objects are automatically and efficiently distributed by Terracotta to the cluster so that all participating JVMs have an up-to-date and stable view of your object data. Terracotta also manages thread interaction between JVMs by making your synchronized(), wait(), and notify() calls cluster aware.

Your code thinks it's talking to threads and objects in the local JVM. Terracotta lets you deploy that same code on as many JVMs as you need to meet your capacity demands without having to write a "stateless" application.

A Terracotta Case Study

To see how this works in the real world, let's take a look at how a large publishing company recently used Terracotta to solve a critical database overload problem in their stateless web application.

The application in question is a test proctoring service that holds examinations for up to 5000 concurrent users. Before using Terracotta, the application was deployed using a stateless architecture backed by a relational database. As users proceeded through the examination, their answers were committed to the database so no in-flight examinations would be in jeopardy should one of the application servers go off-line.

At that level of traffic, the database was already at 70 percent utilization. However, the business was growing and the forecast of double the number of concurrent users in a matter of months presented the application team with a choice between scaling up the overloaded database at a significant cost or investigating some sort of alternative scale-out architecture.

The application team settled on using Terracotta to store the in-flight examination data in the durable virtual heap. They were able to keep state data -- the incomplete examination -- as plain Java objects until the examination was over at which time the results were posted all at once to the database. By eliminating the chatter with the database during the examination, they were able to double the number of concurrent users and simultaneously reduce their database utilization from a hot 70 percent to a much cooler 30 percent.

Figure 2

Terracotta let them use a natural object model where examinations were represented as simple objects, rather than suffering the impedance mismatch of saving and loading them to and from the database on every request. Yet, because those objects were durable and available to any application server in the cluster, they retained all of the virtues of a stateless architecture with none of its vices.

With their Terracotta-enabled architecture, any application server can be decommissioned at a moment's notice and all of its traffic diverted to other active members of the web cluster without any loss of data or interruption of a user's workflow through the examination. More application servers can be deployed as needed to keep up with demand. And, not only is this scale achieved without additional load on the database, the database load has actually decreased because they no longer use it to hold state data.


Stateless architectures are hard to build and hard to scale. Many an unsuspecting application has been taken down the stateless route, only to find that the added stress on other parts of the application infrastructure confound scalability goals.

Luckily there are tools like like those described here that can restore sanity to beleaguered scale-out applications and their developers. With a shared, durable heap, you can write applications with the simplicity of a stateful, single-machine deployment; but, when it comes time to scale, you can deploy on purpose-built scaling infrastructure, unburdening the database and the rest of your infrastructure from the demands of high-availability and increasing capacity.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.