Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼


System Virtualization

Hypervisor Security

Some tout virtualization as a technique in a "layered defense" for system security. The theory postulates that since only the guest operating system is exposed to external threats, an attacker who penetrates the guest will be unable to subvert the rest of the system. In essence, the virtualization software is providing an isolation function similar to the process model provided by most modern operating systems.

Published Hypervisor Subversions

However, common enterprise virtualization products have not met security requirements for high robustness and were never designed or intended to meet these levels. Thus, it should come as no surprise that the theory of security via virtualization has no existence proof. Rather, a number of studies of virtualization security and successful subversions of hypervisors have been published.

In 2006, the SubVirt project demonstrated hypervisor rootkits that subverted both VMware and VirtualPC. [3]

The BluePill project took hypervisor rootkits a step further by demonstrating a malware payload that was itself a hypervisor that could be installed on-the-fly, beneath a natively running Windows operating system. [4]

Tavis Ormandy performed an empirical study of hypervisor vulnerabilities. The researchers generated random I/O activity into the hypervisor, attempting to trigger crashes or other anomalous behavior. The project discovered vulnerabilities in QEMU, VMware Workstation and Server, Bochs, and a pair of unnamed proprietary hypervisor products. [5]

Clearly, the risk of an "escape" from the virtual machine layer, exposing all guests, is very real. This is particularly true of hypervisors characterized by monolithic code bases. As one analyst has said, "Virtualization is essentially a new operating system …, and it enables an intimate interaction between underlying hardware and the environment. The potential for messing things up is significant." [6]

At the 2008 Black Hat conference, security researcher Joanna Rutkowska and her team presented their findings of a brief research project to locate vulnerabilities in Xen. [7] One hypothesis was that Xen would be less likely to have serious vulnerabilities, as compared to VMware and Microsoft Hyper-V, due to the fact that Xen is an open source technology and therefore benefits from the "many-eyes" exposure of the code base.

Rutkowka's team discovered three different and fully exploitable vulnerabilities that the researchers used to commandeer the computer by way of the hypervisor. Ironically, one of these attacks took advantage of a buffer overflow defect in Xen's Flask layer. Flask is a security framework that is the same one used in SELinux. It was added to Xen to improve security.

Rutkowka's results further underscore an important principle: software that has not been designed for and evaluated to high levels of assurance must be assumed to be subvertible by determined and well-resourced entities.

High Assurance Approach

However, the hypervisor need not hamper security efforts. For example, Integrity is an operating system that has achieved a high assurance Common Criteria security certification. [8] Designed for EAL 7, the highest security level, Integrity meets what the National Security Agency deems is required for "high robustness:" protection of high value resources against highly determined and sophisticated attackers.

Our operating system is being used in NSA-approved cryptographic communications devices, avionics systems that control passenger and military jets, life-critical medical systems, secure financial transaction systems, and a wide variety of other safety and security-critical systems.

We have found that a security kernel can provide domain separation with virtualization duties relegated to user-mode applications. This approach achieves a high level of assurance against hypervisor escapes.

Integrity provides a full-featured API and SDK, enabling the creation and deployment of secure applications that cannot be trusted to run on a guest. Thus, critical security applications and data such as firewalls, databases, and cryptographic subsystems can be deployed both alongside and securely separated from general-purpose operating environments such as Windows or Linux.

The combination of virtualized and native applications results in a powerful hybrid operating environment, as in Figure 5, for the deployment of highly secure yet richly functional systems. In the following section, we shall discuss how this hybrid architecture is especially critical for the flexibility required in embedded systems.

[Click image to view at full size]
Figure 5: Virtualized environments alongside native applications. (Source: Green Hills Software, 2008)

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.