Building the Digital Data Vault

Leave it to Michael Swaine to work the "Svalbard Global Seed Vault" into an article that's supposed to be about computer programming. And leave it to me to let him do it. That's what happens when an article comes in late, and a distressed editor doesn't have time to demand relevance. But when you get down to it, I guess Michael's recent Gates and the Doomsday Seed Vault did have some relevance, thanks to a reference to a scruffy-looking penguin smoking a cigar.

In many ways, the problem Svalbard Global Seed Vault researchers are trying to solve ("to provide a safety net against accidental loss of diversity in traditional genebanks") is akin to the one researchers at the University of California-Santa Cruz have tackled. But at UCSC, scientists are trying to save digital data, rather than seeds.

"The problem is how to build a large-scale data storage system to last 50 to 100 years," explains Ethan Miller. Most large-scale data storage these days involves tape, but digital tape has problems. Which is why Miller, along with grad students Mark Storer and Kevin Greenan and researcher Kaladhar Voruganti, developed Pergamum, which uses hard disk drives to provide energy-efficient, cost-effective storage.

For reliability, Pergamum uses two levels of redundancy--within and between disks--to protect from both disk failures and latent sector errors when writing data to a disk. The system doesn't care what the actual storage medium is, as long as the device can implement a simple protocol that lets it function as part of the network. Each unit is essentially a self-contained box with a network connection. Flash memory provides low-power and persistent storage so that many operations can be performed without activating the hard drive.

While I'm sure Michael will have full technical details real soon now, I'd suggest seeing their paper instead of waiting.

-- Jonathan Erickson
jerickson@ddj.com

Posted by Jon Erickson at 06:27 PM  Permalink