Hypervisor Security
Some tout virtualization as a technique in a "layered defense" for system security. The theory postulates that since only the guest operating system is exposed to external threats, an attacker who penetrates the guest will be unable to subvert the rest of the system. In essence, the virtualization software is providing an isolation function similar to the process model provided by most modern operating systems.
Published Hypervisor Subversions
However, common enterprise virtualization products have not met security requirements for high robustness and were never designed or intended to meet these levels. Thus, it should come as no surprise that the theory of security via virtualization has no existence proof. Rather, a number of studies of virtualization security and successful subversions of hypervisors have been published.
In 2006, the SubVirt project demonstrated hypervisor rootkits that subverted both VMware and VirtualPC. [3]
The BluePill project took hypervisor rootkits a step further by demonstrating a malware payload that was itself a hypervisor that could be installed on-the-fly, beneath a natively running Windows operating system. [4]
Tavis Ormandy performed an empirical study of hypervisor vulnerabilities. The researchers generated random I/O activity into the hypervisor, attempting to trigger crashes or other anomalous behavior. The project discovered vulnerabilities in QEMU, VMware Workstation and Server, Bochs, and a pair of unnamed proprietary hypervisor products. [5]
Clearly, the risk of an "escape" from the virtual machine layer, exposing all guests, is very real. This is particularly true of hypervisors characterized by monolithic code bases. As one analyst has said, "Virtualization is essentially a new operating system …, and it enables an intimate interaction between underlying hardware and the environment. The potential for messing things up is significant." [6]
At the 2008 Black Hat conference, security researcher Joanna Rutkowska and her team presented their findings of a brief research project to locate vulnerabilities in Xen. [7] One hypothesis was that Xen would be less likely to have serious vulnerabilities, as compared to VMware and Microsoft Hyper-V, due to the fact that Xen is an open source technology and therefore benefits from the "many-eyes" exposure of the code base.
Rutkowka's team discovered three different and fully exploitable vulnerabilities that the researchers used to commandeer the computer by way of the hypervisor. Ironically, one of these attacks took advantage of a buffer overflow defect in Xen's Flask layer. Flask is a security framework that is the same one used in SELinux. It was added to Xen to improve security.
Rutkowka's results further underscore an important principle: software that has not been designed for and evaluated to high levels of assurance must be assumed to be subvertible by determined and well-resourced entities.
High Assurance Approach
However, the hypervisor need not hamper security efforts. For example, Integrity is an operating system that has achieved a high assurance Common Criteria security certification. [8] Designed for EAL 7, the highest security level, Integrity meets what the National Security Agency deems is required for "high robustness:" protection of high value resources against highly determined and sophisticated attackers.
Our operating system is being used in NSA-approved cryptographic communications devices, avionics systems that control passenger and military jets, life-critical medical systems, secure financial transaction systems, and a wide variety of other safety and security-critical systems.
We have found that a security kernel can provide domain separation with virtualization duties relegated to user-mode applications. This approach achieves a high level of assurance against hypervisor escapes.
Integrity provides a full-featured API and SDK, enabling the creation and deployment of secure applications that cannot be trusted to run on a guest. Thus, critical security applications and data such as firewalls, databases, and cryptographic subsystems can be deployed both alongside and securely separated from general-purpose operating environments such as Windows or Linux.
The combination of virtualized and native applications results in a powerful hybrid operating environment, as in Figure 5, for the deployment of highly secure yet richly functional systems. In the following section, we shall discuss how this hybrid architecture is especially critical for the flexibility required in embedded systems.
