Red-Team Application Security Testing

• What underlying software fault or design decision caused this bug?
• What were the symptoms of failure that should have alerted the tester to the presence of the bug?

• What test-design strategy would have enabled testers to expose the vulnerability before the software was released?

Our method was to analyze these bugs, then generalize the patterns of behavior and invent potentially useful test techniques. The result was How to Break Software Security: Effective Techniques for Security Testing, by James Whittaker and Herbert H. Thompson (Addison-Wesley, 2003), which describes focused testing techniques to expose security bugs. In this study, we identified four main classes of security bugs.

Dependency failures. Many security bugs are not caused directly by the application under test; rather, they are inherited from libraries, files, and other resources external to the application. Either the application depends on an external resource to provide security and that resource becomes unavailable, or the security provided by a resource is faulty. Three key resources that cause many of these types of vulnerabilities are the file system, the registry, and dynamically loaded code libraries. Testers must ask when, where, and how the application access data stored in the registry and file system, and whether this is appropriate based on the sensitivity of the data stored there. For libraries, we must ensure that the application responds securely to library failures.

One good example of a dependency failure is the corruption of network data. Applications may be able to securely handle complete network failure, but may not have anticipated a user manipulating data contained in the payload of a packet manually. One reason for this is that applications communicating over the network often assume that remote machines sending packets will send the packets in the proper form and constrain the length and value of certain data. Listing One shows the structure of a basic network-packet corrupter that intercepts and manipulates packet-payload data on the wire. Listing Two shows a specific corruption function that searches the payload of a packet for a string, and if that packet contains that string, manipulates some of the payload data.

Unanticipated user input. Unanticipated input scenarios are a second major source of security bugs. Applications are normally programmed to deflect illegal input, but considering all such possibilities is an enormous task. The best example of this is the notorious buffer overrun. Buffer overruns are caused by entering long strings into input fields; the strings must be longer than the internal buffer assigned to hold them. In certain situations, the overflow can actually cause arbitrary (and sometimes malicious) code to be executed by the host computer.

Design vulnerabilities. A third source of security vulnerabilities are those that have been designed into the software inadvertently. Obviously, no ethical developer would purposefully design insecure code but often legacy code or improper assumptions about user behavior will cause design decisions to be made that are not in the best interest of security.

At the top of the list are flaws that allow easy access to the application. If the application ships with user accounts, make sure that all administrative or root accounts force the users to change the default password. Make sure that the default network port configuration is minimal; that is, that no ports are left open without security mechanisms in place. Make sure that the product ships with any security settings tuned to maximum.

Implementation vulnerabilities. Even perfect designs can be made insecure through imperfect implementation. Security can be spelled out meticulously in a specification and yet be implemented in such a way as to cause insecurity. The best example of this is the so-called "man in the middle" attack, which takes advantage of good security that is implemented improperly.

The attack takes advantage of time discrepancies between the code that implements security around some piece of sensitive information and the code that actually uses that same information.

In studying security bugs, we found that we needed a tool that lets us both observe all the application's interactions with its environment—the registry, filesystem, kernel, and other applications—and also control these applications. The result of our efforts is a freely available tool called "Holodeck," which is available from DDJ (see "Resource Center," page 5) or at http://www.sisecure.com/.

The hacking and security communities also offer a wealth of tools for security testers. Two of the best sites we've found are http://astalavista.box.sk/ and http://www.sysinternals.com/.

—H.H.T. and S.G.C.