Since 2008, the National Institute of Standards and Technology (NIST) has been on the hunt for the next cryptographic hash function. It has evaluated several proposed functions, and chosen five as of them as potential finalists. From those five, one will become the new SHA-3 standard. This article examines these finalists, with an eye to their operation and features. I examine how each one reduces a given message into a unique hash.
White PapersMore >>
- Solving Big Data Challenges with Simplicity & Speed
- Optimize Your SQL Environment for Performance & Flexibility
In a sequel article next week, I will provide the code (as
NSString), test it, and compare the results against those produced by the current OpenSSL standard. For this article, you need only a basic understanding of cryptography.
The Need for a New Hash
Many of today's security packages rely on three standard functions to supply hashes that are cryptographically secure. These functions are products of open-source projects and thus are free of licensing issues. But under attack from high-speed hardware, cloud computing, and advanced algorithms, cracks in these hash functions have appeared, making them unreliable.
Consider the MD5 hash function, for instance. First introduced in 1992 by Ronald Rivest, this function emits hashes that are 128-bits long (16 bytes). It uses a Merkle-Damgård block to process the message data. It takes about 4 rounds of block operations to generate a hash. The function's internals are detailed in "The MD5 Message-Digest Algorithm" (RFC-1321).
But MD5 was found to be insecure. In 2009, researchers Tao Xie and Dengguo Feng reported they were able to break MD5 with a collision attack. In theory, such attack should take at least 264 cycles to find a collision. This 2009 attack, however, succeeded after 220.96 cycles, and it was done with off-the-shelf computer hardware. (More information on the subject can be found here.)
Next came SHA-1. It was developed in 1995 by the US National Security Agency (NSA) and published as document FIPS PUB-180 by NIST. It too uses a Merkle-Damgård block to process its message data. The function takes around 80 rounds of block operations to produce a 160-bit hash.
SHA-1 is meant as a viable replacement for MD5. It is currently unbroken, but several test attacks have revealed potential flaws. For instance, in 2005 Rijmen and Oswald made a collision attack that worked on 53 of the 80 rounds of SHA-1, a safety margin of about 1.5. Later attacks have reduced the margin of safety even further. Details of the latest attack are made public by the open-source project HashClash.
As a result, SHA-2 was designed by the NSA as a successor to SHA-1. Like SHA-1, it uses a Merkle-Damgård block to process the message data. But SHA-2 can emit hashes in one of three bit lengths: 256, 384, and 512 bits. It uses 64 rounds of block operations to produce a 256-bit hash (32 bytes) and 80 rounds for a 512-bit hash. Technical details on SHA-2 are published by the NIST under the document FIPS PUB 180-2.
Currently, SHA-2 has no known weaknesses. The best attack was a pre-image one directed against 41 rounds of SHA-2/256 and 46 rounds of SHA-2/512. In both attacks, SHA-2 maintained a safety margin of at least 1.5. So far, there are no recorded attacks successful against a full-round SHA-2.
And yet SHA-2 failed to displace SHA-1. One reason for its failure is the lack of urgency in the developer community to make the switch. Another is the cost in time and effort to migrate from SHA-1 to SHA-2. Moreover, many developers prefer to wait for the next standard, which is SHA-3.
Seeking a New Hash
It is a matter of time before someone does come up with a successful attack against a full-round SHA-2. To guard against this event, NIST began a public contest in 2008 to find a hash function to serve as the SHA-3 standard.
The contest has three rounds of tests and evaluations. Candidates for the contest must meet four design goals:
- A candidate algorithm must have security traits at least equal to those for SHA-2. It should resist known attack types like pre-image and collision. Failure to meet this goal results in immediate rejection.
- The candidate must lend well to cryptanalysis. Its output should show enough randomness to obscure any correlation between message and hash. Possible design flaws should be isolated and studied carefully. The same applies to corrections made to address those flaws.
- The candidate function must be easy to implement as either hardware or software. It should be able to use the latest technology like multiple processor cores and GPUs. Conversely, it should be able to work within the resource-frugal realm of embedded systems.
- Finally, all candidates must practice code diversity. No two should have the same compression function or use the same mix operations. This helps ensure that no two candidates are vulnerable to the same attack.
Fifty-one candidate functions were submitted for the first contest round. Out of those candidates, 14 were chosen for the second round. The rest were rejected due to flaws in design or to weaknesses found by cryptanalysis. Some were rejected due to poor performance.
Of the 14 round-two candidates, five have reached the third and final round. The other nine were rejected solely on performance and design issues. Nevertheless, these same nine have no known weaknesses. They may still be useful for tasks that do not require high security. In 2012, one of the final five candidates will become the new SHA-3 standard.