Perhaps the most significant development and least discussed trend of 2011 has been the astonishing drop in the prices of RAM. This change is rapidly reshaping all aspects of computing (and therefore programming) in every category of computing device.
If you haven't looked at IT hardware lately, here's a reverse sticker shock: Today, you can buy a Dell server with 96GB of RAM for less than $5,000. And if you want to go really big, a 64-core server with 512GB of RAM, costs less than $30K. That's an almost throw-away price for hardware that truly qualifies as supercomputer-scale.As IT organizations embrace this favorable pricing, they're trying to figure out what to do with all the RAM. This is a problem that not unlike the one college students face after they'be bought a 12GB laptop for less than $2K. Too often they come up with the solution: Do nothing. The extra resources lie fallow and I mean all the resources: the CPU, the RAM, the disk, and the network bandwidth all remain substantially underutilized. It's like driving a Ferrari in stop-and-go traffic.
This state of affairs won't last long. Right now, predictably, the high-end is figuring it out first. Many capacious servers are being used to host virtual machines. The VM solution carves up the capacity into familiar-size computing environments and the performance benefit is derived from distribution of work across the many VM instances on the same physical machine. Such groups of VMs then interact with other hardware nodes running their own VMs. What they face collectively is the problem of shared data access, which is handled by one of two primary solutions: a shared key- value (KV) store such as Terracotta's Big Memory and ehCache products, or a in-memory data fabric such as Oracle's Coherence layer (formerly, Tangosol). Other vendors, such as IBM and VMware, have comparable solutions. These products make any data item in the memory system available quickly to any node and computing device that needs it. In the case of an update to a data field, the fabric makes sure all updates are synchronized and done correctly.
Some big apps, however, are not ideally run across a series of VMs cum memory fabric. Instead, they prefer to work in one huge (generally JVM-based) instance. The more the application favors computing-intensive activity, the more it makes sense to have its data on the local stack in the same huge process. In such a case, special VMs are needed so as to not stop-the-world to do garbage collection of, say, 64 GB of RAM objects. Azul Systems makes its living in this space with hardware and software solutions. The software version of the Azul high-capacity JVM is called "Zing" and it handles stacks up to 512 GB, with continual concurrent garbage collection and heap compaction. In considering this option, I personally would also evaluate Oracle's progress in integrating JRockit technology with the HotSpot VM. For a while, JRockit was a leader in this technology and so it might offer a less expensive alternative. Unlike Azul, where situations such this problem constitute is its bread and butter, JRockit and/or its descendants would need to be evaluated carefully to make sure they truly do scale correctly.
Ari Zilka, the CTO and founder of Terracotta, made a good point recently that vast memory makes it easy to suck whole databases into memory, have them run there, and then spill over transactions to a persistence layer as needed for record keeping and capacity management. The problem he points out is that you have to use a database where this model works well. Not only being able to perform transactions in memory, but also handling issues like loading the data into memory and sharding the database across several nodes.
If a KV store is added to the mix, Zilka points out, the program now has the problem of having lots of different ways of interacting with objects: the KV map, the in-memory database, the program's own heap, and likely some serialization to disk. If you throw long, slow-running processes, such as Hadoop, into the mix, you are confronted with an array of different object-access mechanisms that have their own unique latency, which must be managed to be able to complete transactions efficiently. The operational skills behind this have not yet evolved to the point where there are known good practices for handling so much data in so many different forms in memory. I expect the next few years will show much progress in this regard, and Dr. Dobb's will be reporting on the advances as they appear.
The benefits, though, of this use of massive RAM are so profoundly compelling that most IT organizations are happy to dance on the cutting edge to enjoy them. A prime example is complex event processing (CEP), which analyzes event data on the fly and looks for certain patterns. One example is a financial services firm. When a customer swipes her card, her profile is loaded into memory, and a recap of all the IP addresses from which she has swiped the card are analyzed on the fly (Is this a store? The third different address today? Was this card used at two different locations today without enough time between them for the same person to make the purchase?). Because CPUs are fast and the data is in memory, this analysis and much more can be done in the interval between the card swipe and the authorization. The benefits of so much analysis are clearly real and will only increase in the years ahead.
In an upcoming editorial, I'll discuss what big, cheap RAM brings to the developer at the desktop. There's lots of goodness there, too.