The Pentium's Enhanced v86 Mode

Robert is an independent consultant on the x86 architecture. He can be reached at [email protected].

Months before the Pentium Pro was released, rumors were circulating that Intel was going to usher in a new era of Glasnost by making public most of the infamous Appendix H documents. (See my November 1997 column for a description of Appendix H.)

It seemed futile for Intel to continue hiding this information. Much of the Model-Specific Register had already been hacked and cracked by engineers like Christian Ludloff (http://www.sandpile.org/). I, too, played a role in liberating Appendix H, publishing the first article describing four-MB paging (see "Undocumented Corner," DDJ, May 1996). In short, the time was ripe for Intel to come clean and set free the only remaining subject matter in Appendix H -- a description of the Virtual Mode Extensions to v86 mode.

Intel continued to keep the Virtual Mode Extensions (VME) information secret. In July 1995, Intel released the newest edition of the Pentium processor developer's manuals which contained a description -- albeit an incomplete one -- of the Model-Specific Registers (MSRs). Intel's published list was identical to that published by the notorious Appendix H Liberation Movement, down to the same missing items. Unfortunately, none of these manuals contained the descriptions of fourMB pages or VME. Intel would take another year before officially releasing this information.

As the introduction of the Pentium Pro approached, rumors became more persistent that Intel would release the remaining secrets in Appendix H. During this period, I called Intel many times, and was told a variety of stories regarding the release of the manuals.

In preparation for the imminent release of the Pentium Pro, and the remaining Appendix H secrets, I prepared a Pentium Pro "special edition" release of my own. With the aid of some colleagues and help from the electronics-industry trade press, I wrote an article describing VME for the November 11, 1995 (COMDEX) edition of an industry-leading trade publication. The COMDEX edition meant that an extra 100,000 copies would be printed, and distributed at the giant COMDEX trade show. The article was a success, appearing alongside Intel's official introduction of the Pentium Pro processor.

Although the Pentium Pro was released with much fanfare, the Pentium Pro manuals were not released. Intel kept promising their release, supposedly as early as December 1995. December came and went -- as did January, February, and March. Finally, in April 1996, Intel released the Pentium Pro manuals.

The computer industry and the Internet newsgroups were abuzz with news of the manuals and of Intel's new Glasnost. The manuals indeed contained the descriptions of four-MB pages, two-MB pages (see DDJ, July 1996, for a description of two-MB pages), and VME. But the real question remained: Were these descriptions accurate, or were they fraught with inaccuracies like other recent Intel technical manuals? Unfortunately, upon close inspection, the manuals have turned out to be just as bad as their predecessors.

Searching for VME

Of all the secrets in Appendix H, none were more closely guarded than the details of the Virtual Mode Extensions (VME). Introduced in the Pentium, VMEs were later retrofitted to late-model Intel 486 DX4s. When reading the Intel Pentium manuals, you might not even notice the mention of enhancements to virtual-8086 mode (v86 mode). The manual is not silent on the subject, however. There are at least 27 references to VME, as well as obnoxious statements that refer the reader to "Appendix H." Of course, when you turn to Appendix H, you find that you must sign a nondisclosure agreement (NDA) with Intel before obtaining the information. Still, even without the NDA, you can use these manuals and other sources of information to reverse engineer the remaining details.

The 27 references to VME in the Pentium manual are a good start for reverse-engineering purposes. Intel filed for a patent in Britain on some VME details. Since patents are always publicly available, the British patent is another good source of information. With a good understanding of v86 mode, you could infer most of the remaining details of VME solely from those 27 references. All that's needed to reverse engineer the remaining details is an understanding of v86 mode, a little ingenuity, experimentation, persistence, and no qualms about hitting the Big Red Switch. If you have $10,000 to burn, an in-circuit emulator (ICE) might be helpful, too. (See my July 1997, September 1997, and November 1997 columns, respectively, for in-depth discussions of ICEs.)

The Need for VME

When the 80286 was introduced, Intel implemented a means to multitask various computer programs. The 80286 multitasking capabilities were limited, and never used in mainstream (that is, Microsoft) operating-system software. The 80286 lacked the hardware capabilities necessary to manage virtualized versions of the microprocessor. Hence, the 80286 was not a good platform for operating-system designers to create multiple Virtual DOS Machines (VDMs).

Intel addressed these shortcomings with the 80386. The 80386 received some beefed up multitasking capabilities, and, most importantly, a new operating mode -- v86 mode. v86 mode was intended to be used just as the name implies -- as a virtualized 8086. To create a virtualized 8086, Intel added the mechanism that allowed the transition from the v86 task to the operating-system kernel and back to be as painless as possible. But OS kernel support wasn't as painless as software developers would have liked.

When multiple v86 tasks are running concurrently, the operating-system kernel needs to arbitrate and control the computer resources. In some cases, the kernel must redirect a specific interrupt to a specific v86 task -- like a disk service request or keyboard interrupt. In other cases, the kernel might want to redirect an interrupt to all v86 tasks -- like a timer tick. Playing into this interrupt-arbitration nightmare are microprocessor instructions that are intended to limit a task's ability to receive interrupts (like the CLI and STI instructions). These instructions are "IOPL-sensitive," meaning their successful execution depends upon the I/O Privilege Level of the currently executing v86 task. These instructions -- CLI, STI, PUSHF, POPF, INT, and IRET -- all have some influence on the FLAGS register, specifically the Interrupt Flag (IF). The relationship between these instructions and the arbitration of interrupt services will become clear as the explanation of VME unfolds.

These day, there are two main uses for v86 mode:

• DOS memory managers.
• DOS sessions under a multitasking operating system (MTOS) -- like a Windows DOS box.

Under a memory manager, DOS remains a single-tasking operating system that still executes instructions in a serial manner. Therefore, hardware resources don't need to be arbitrated (except I/O ports subject to I/O protection from the I/O permission bit map -- like DMA ports), and IOPL-sensitive instructions don't need to be restricted. However, running a DOS session under Windows is quite different. Nearly all of the resources need to be restricted. The DOS session should not have direct access to the hardware with the ability to program any and every device. Nor should the DOS session have the ability to directly control the interrupt flag (IF) of the EFLAGS register. v86 mode has support for restricting such accesses. The IOPL settings in the EFLAGS register serve to restrict IOPL-sensitive instructions that modify IF (the I/O port instructions that are IOPL-sensitive in protected mode are not IOPL-sensitive in v86 mode). The I/O permission bit map restricts access to I/O ports. However, there are shortcomings with the standard v86 mode:

• IOPL must be set to less than 3 when the OS needs to virtualize the interrupt flag. When a Virtual Device Driver (VDD) needs to simulate a hardware interrupt into a VDM, it must be able to detect when the VDM is interruptible. Therefore, IOPL must be less than 3 so that the interrupt flag can be virtualized (see Chapter 10 of The Design of OS/2, by Michael S. Kogan and Harvel M. Deitel, Addison-Wesley, 1996). Since IOPL is less than 3, performance is significantly degraded by attempts to execute interrupt-flag-sensitive (IF-sensitive) instructions, which always fault to the v86 monitor.
• An IOPL setting of 3 yields better system performance than an IOPL setting less than 3. This setting reduces the trapping overhead of executing IF-sensitive instructions, but allows Virtual DOS Machines (VDMs) to disable interrupts. This creates a potential data integrity problem for the whole system.
• An operating system may allow a VDM to receive real (external) interrupts or virtual interrupts. This is a policy decision made by the OS implementors. If a v86 task only receives virtual interrupts, then it can be demand-paged, whereby it is swapped out to disk when real memory is needed for other purposes. If the v86 task receives real (external) interrupts, then it cannot be demand-paged, since the interrupt handler may be paged out when the interrupt occurs. The OS may not have enough time to swap-in the entire task before another interrupt occurs; likewise it would be too complicated and still too time consuming to bring in just the portion of the task, which contains the interrupt service routine.
• All INT-n instructions cause a switch out of v86 mode. When IOPL<3, an INT-n faults to the v86 monitor. When IOPL=3, the INT-n attempts to invoke the protected mode interrupt handler associated with that particular interrupt (success depends on the DPL of the gate being used). The monitor or interrupt handler must either emulate the interrupt's functionality, or restart the v86 task so that it executes the interrupt itself (this is known as "reflecting the interrupt" or "interrupt redirection"). DOS kernel routines are accessed through software interrupts. Therefore, thousands of interrupt calls generate thousands of transitions into and out of v86 mode. This gives the v86 task a substantial performance disadvantage compared to the same program running under DOS.

VME Fixes v86 Problems

Enhanced virtual-8086 mode (Ev86 mode) was designed to eliminate many of these problems, and significantly enhance the performance of v86 tasks running at all IOPL levels. When running in Ev86 mode at IOPL=3, CLI and STI still modify IF. This behavior hasn't changed.

However, running at IOPL<3 has changed. CLI, STI, and all other IF-sensitive instructions no longer unconditionally fault to the Ev86 monitor. Instead, IF-sensitive instructions clear and set a virtual version of the interrupt flag in the EFLAGS register called VIF (STI will fault when EFLAGS.VIP=1). Clearing VIF does not block external interrupts, as clearing IF does. Instead, IF-sensitive instructions clear and set a virtual version of the interrupt flag called VIF. VIF does not control external interrupts as IF does. Instead, VIF is an indicator of the interruptibility state of the Ev86 task. If a DOS program were to disable interrupts and then spin inside of an idle loop (like a JMP $ instruction), the errant program would lock up the computer. However, a DOS program running as an Ev86 task would be invulnerable to such a bug. Since, instead of IF, VIF is set and cleared by IF-sensitive instructions, the operating system kernel always has a means to regain control of the errant program. This new behavior has some substantial benefits: Performance is increased, as CLI and STI don't cause time-consuming faults. Secondly, the complexity of the monitor program is reduced, since it doesn't have to maintain its own virtual interrupt flag. When the old-style v86 monitor virtualized IF, it needed to emulate all changes to IF caused by IF-sensitive instructions (CLI, STI, PUSHF, POPF, INT, and IRET). Using Ev86 mode eliminates this complexity because the CPU automatically virtualizes IF; performance increases because IF-sensitive instructions don't fault to the Ev86 monitor.

When a timer tick, keyboard stroke, or any other external interrupt occurs, the host operating system always intercepts and services these interrupts. The v86 task (or Ev86 task) is never allowed to directly service these interrupts. Imagine the computer state when such an interrupt occurs. The computer may be in a v86 task, or it may not. Imagine that the computer is executing a v86 task, but the virtualized IF flag is clear, thus indicating that the v86 task is in an uninterruptable state. Or simply imagine that some tasks are swapped out to disk when the interrupt occurs. When this happens, the host OS must delay sending the interrupt to the v86 task until it is running and ready to accept interrupts. Other interrupts, such as keystrokes, may be intended for a specific VDM, but not all VDMs. In this case, the v86 monitor needs to send a specific interrupt to a specific VDM -- ignoring all other VDMs. Delaying and filtering interrupts in this manner is known as "interrupt virtualization." Once the VDM with a virtual interrupt pending becomes interruptible, the OS reflects the interrupt to the VDM as if a real interrupt had occurred.

Prior to Ev86 mode, the v86 monitor needed to maintain a virtual interrupt flag in software. The v86 monitor was forced to handle many exceptions that were unnecessary. For example, when a virtual interrupt was pending, further IOPL-sensitive instructions that attempted to clear IF caused undesired faults, which then caused the monitor to redundantly clear the virtual interrupt flag. This problem doesn't exist in Ev86 mode. These instructions, which redundantly attempt to clear IF, don't fault to the monitor. Therefore, the source code that clears the software virtual interrupt flag can be removed. In fact, while using Ev86 mode, all of the code needed to maintain the software virtual interrupt flag can be removed -- since the virtual interrupt flag is maintained by the CPU itself.

Prior to Ev86 mode, software interrupts (INT-n instructions) always caused a switch out of v86 mode. If IOPL=3, the transition occurs through a gate associated with the interrupt (assuming that all other protection attributes will permit the transition to occur); when IOPL<3, the transition occurs as the result of a general protection fault to the monitor. When IOPL=3, the monitor needs to determine whether or not the cause of the interrupt is a software interrupt, external interrupt or CPU-generated exception. When IOPL<3, software interrupts don't transition through their associated gates in the IDT (they transition through the #GP gate). In the case of software interrupts, the monitor must interpret the opcode to determine which interrupt number needs servicing. The monitor must then emulate the interrupt, or reflect the interrupt back to the v86 task. External interrupts and CPU-generated exceptions still transition through their associated gate in the IDT. For these cases, the monitor still needs to determine the source of the interrupt (external or CPU exception), and take the appropriate action. Using Ev86 mode can simplify this process, and enhance the performance of handling software interrupts.

Software interrupt execution is controlled by a new structure in the TSS called the Interrupt Redirection bitmap (IR bitmap). Hardware interrupts and exceptions are not handled by the IR bitmap. Each bit in this new structure controls whether or not a specific software interrupt will be invoked in a manner compatible with the Intel386, or be invoked purely within the Ev86 task. In Ev86 mode, these interrupts may be invoked and executed without ever leaving the Ev86 task. Using this new technique would reduce complexity in the monitor. Interrupts that would normally fault to the monitor no longer behave this way. Interrupts that would transition through the IDT no longer would.

Conclusion

With the introduction of the 80386 came microprocessor support for emulating multiple 8086s. These "v86-mode tasks" are most prominently used by memory managers and DOS boxes in Windows. QEMM, 386MAX, and EMM386 use v86 mode as a means to emulate expanded memory and provide the ability to load DOS programs in upper memory blocks (UMBs). Ev86 mode was introduced in the Pentium as a means to fix some of the performance deficiencies of standard v86 mode. Using Ev86 mode, the operating system can provide a substantial performance benefit to v86 tasks.

In this column, I've barely scratched the surface of a discussion of the Virtual Mode Extensions. Even though Intel finally documented VME, many details were incorrect or left out. In my next column, I'll start digging under the hood of VME and explain how it works. I'll provide source code examples of setting up Ev86 tasks and demonstrate how to redirect interrupts using the new Interrupt Redirection Bitmap.

DDJ