Running Windowed DOS Apps in Windows 3.11

Running non-Windows DOS applications within a graphical window was one of the hallmark features of Microsoft Windows 3.11. To achieve this, the operating system relied on the architectural features of the Intel 80386 processor operating in 386 Enhanced Mode, paired with 32-bit Virtual Device Drivers (VxDs) and dedicated configuration files. Instead of relinquishing complete control of the machine to the legacy program, Windows isolated each DOS application inside its own Virtual DOS Machine (VDM), virtualizing memory, video hardware, and input devices so the software could coexist alongside native graphical applications.

386 Enhanced Mode and Virtual 8086 Mode

Windows 3.11 could only run DOS programs in a window when operating in 386 Enhanced Mode. In this mode, the system leveraged the Intel 80386 processor's hardware-level Virtual 8086 (V86) mode.

V86 mode allowed the CPU to create an isolated 1 MB address space that simulated an independent, native 8086 processor. When a user launched a DOS executable, Windows created a Virtual DOS Machine (VDM) where the legacy software operated under the assumption that it had exclusive access to the computer's CPU and memory.

Virtualizing Video and Screen Output

The primary obstacle to running a DOS app in a desktop window was handling screen writes. Most DOS programs bypassed the operating system and wrote directly to video RAM addresses (such as 0xB800 for color text mode).

Windows solved this through video virtualization handled by the display driver and the virtual display device (*VDD):

  • Text-Mode Applications: When a DOS app wrote characters directly to text-mode video memory, the VDD trapped the memory writes and mirrored the content into a dedicated buffer. Windows then rendered those characters inside a standard graphical desktop window using bitmapped screen fonts (such as 8x12 or 10x18).
  • Graphics-Mode Applications: Applications utilizing graphical EGA or VGA modes demanded immense processing power to emulate inside a window. While Windows 3.11 could sometimes capture and display low-resolution graphics in a window, high-resolution or fast-updating graphical applications were typically forced into full-screen mode. Pressing Alt + Enter toggled the application between a windowed view and full-screen display.

Hardware Interception via Virtual Device Drivers (VxDs)

Because DOS programs expected direct access to physical hardware, Windows 3.11 used 32-bit protected-mode Virtual Device Drivers (VxDs) to intercept hardware interrupts and I/O port operations:

  • Keyboard Input: The virtual keyboard device (*VKD) routed keystrokes from the active Windows window directly into the DOS application's emulated keyboard buffer.
  • Mouse Integration: The virtual mouse driver (*VMD) converted standard Windows cursor coordinates into mouse interrupt signals (INT 33h) expected by the DOS program, allowing users to seamlessly move their mouse in and out of the DOS window.
  • Timers and Sound: The system intercepted hardware timer interrupts and PC speaker commands, ensuring that background applications did not disrupt system timing.

Program Information Files (PIFs)

Users controlled how each DOS application behaved in Windows using the PIF Editor, which generated .PIF files. These files dictated:

  • Display Usage: Whether the application launched full-screen or inside a window by default.
  • Memory Allocation: The amount of conventional, expanded (EMS), and extended (XMS) memory allocated to the virtual machine.
  • Multitasking Behavior: Whether the application paused when placed in the background or continued processing cycles (Background Execution).
  • Shortcut Keys: Disabling Windows system shortcuts (like Alt + Tab or Alt + Esc) so the DOS program could use those key combinations internally.

Preemptive Scheduling

While native 16-bit Windows applications operated using cooperative multitasking, DOS applications running in 386 Enhanced Mode were scheduled preemptively. The underlying Windows 3.11 kernel allocated time slices to each VDM independently. This ensured that an ill-behaved DOS application stuck in an infinite loop would not immediately freeze the entire graphical desktop, allowing the user to safely terminate the virtual machine.