From BIOS to Hypervisor: Mastering Protected Mode and Multi-Tasking
Introduction
When I was 17, switching a 286 CPU from real to protected mode and launching multiple tasks felt like wizardry.
Protected mode was first introduced with the Intel 80286 (often just called the 286). While it provided basic protection features, it was the Intel 80386 that fully exploited protected mode with a 32‑bit architecture and advanced virtual memory support.
Today, with the advent of VMMs like VMware, KVM, and Xen, much of that complexity is hidden behind virtualisation layers. In this post, we'll revisit why entering protected mode was so challenging and explore a minimal hypervisor example that enters VMM-protected mode and launches a couple of tasks.
The Challenge of Protected Mode
In real mode, the CPU starts with:
- 16-bit addressing and no memory protection.
- A flat memory model with minimal segmentation.
To switch to protected mode, you had to:
- Set up a Global Descriptor Table (GDT): Define segment descriptors for code and data.
- Enable the PE Bit in CR0: This bit switches the processor into 32-bit protected mode.
- Flush the prefetch queue: Ensure the processor begins executing code with the new settings.
Any misstep here could crash the system, which is why many early OS developers struggled with it.
However, as Willy Tarreau from HAProxy.org pointed out, the real challenge wasn't entering protected mode but leaving it. The 80286 had no built-in way to return to real mode, requiring an unusual workaround. Below is his direct quote explaining this historical quirk:
The most difficult was not to switch *to* PM, but to switch back to real mode *from* protected mode. The 80286 was completely bogus and unable to do this. That's why they had to wire one line of the 8042 keyboard controller to the reset line so you could trigger a fast CPU reset by emitting 0xFE to port 0x64 after having written the reset cause to 0x40:0x72 to tell the BIOS to *not* initialize the system and instead jump back to the pointer you had preliminary programmed. It was sort of a signal handler in a way :-) This trick was probably one of the ugliest of all computer history, but it later also allowed applications to intercept triple faults. Funny days...
This quirk was a key limitation of the 286, and it wasn't until the 386 that the CPU could seamlessly switch between modes.
The Evolution of Virtual Machine Monitors
The protected mode provided the foundation for modern hypervisors. When virtualisation extensions (like Intel VT-x) emerged, they built on these legacy mechanisms. Modern hypervisors:
- Leverage hardware-assisted virtualisation to run guest OSes in isolated environments.
- Offer advanced scheduling and task isolation compared to the primitive methods used in early protected mode demos.
A Minimal Hypervisor Example
Below is an example that demonstrates:
- Bootstrapping: Loading a simple GDT and entering protected mode.
- Task Scheduling: A rudimentary round-robin scheduler launching two tasks.
Note: This code is a simplified prototype. A production hypervisor requires extensive setup (e.g., VMX initialisation, proper interrupt handling, etc.) and hardware support checks.
Bootloader and Protected Mode Entry (Assembly)
; boot.asm - A minimal bootloader to enter protected mode
[org 0x7c00]
; --- GDT Definition ---
gdt_start:
dq 0x0000000000000000 ; Null descriptor
dq 0x00CF9A000000FFFF ; Code segment descriptor
dq 0x00CF92000000FFFF ; Data segment descriptor
gdt_end:
gdt_descriptor:
dw gdt_end - gdt_start - 1 ; Limit
dd gdt_start ; Base address
; --- Start-up Code ---
start:
cli ; Disable interrupts
xor ax, ax
mov ds, ax ; Initialize DS
lgdt [gdt_descriptor] ; Load GDT
; Enable protected mode
mov eax, cr0
or eax, 1
mov cr0, eax
; Far jump to clear prefetch queue and enter protected mode
jmp 08h:protected_mode_entry
; --- Protected Mode Entry Point ---
[bits 32]
protected_mode_entry:
mov ax, 10h ; Data segment selector
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
call hypervisor_main ; Jump to hypervisor main
cli
hlt ; Halt CPU if hypervisor_main returns
; --- Simple Hypervisor Scheduler ---
hypervisor_main:
mov bx, 0 ; Task index
schedule_loop:
cmp bx, 2
jge reset_index
cmp bx, 0
je run_task1
cmp bx, 1
je run_task2
run_task1:
call task1
jmp next_task
run_task2:
call task2
jmp next_task
next_task:
inc bx
jmp schedule_loop
reset_index:
xor bx, bx ; Reset task index
jmp schedule_loop
; --- Task 1 ---
task1:
mov cx, 0xFFFF
task1_loop:
loop task1_loop
ret
; --- Task 2 ---
task2:
mov cx, 0xAAAA
task2_loop:
loop task2_loop
ret
; Boot sector padding and signature
times 510 - ($ - $$) db 0
dw 0xAA55Putting It All Together
Assemble and link the bootloader and hypervisor code: Use your favourite assembler (NASM) to build the boot sector image.
nasm -f bin boot.asm -o boot.bin
Test in a Virtual Machine:
Load the resulting boot image in a VM (e.g., QEMU) to see the scheduler.
qemu-system-i386 -drive format=raw,file=boot.bin
This prototype shows the basic steps: setting up the GDT, switching into protected mode, and creating a minimal round-robin scheduler. While modern hypervisors are far more sophisticated, this example captures the essence of early protected mode challenges and the evolution toward virtualisation.
Further Reading:
- How to Implement Your Own "Hello, World!" Boot Loader
This tutorial guides you through creating a basic bootloader that prints "Hello, World!" to the screen using assembly language.
Read more here. - Creating a Simple Bootloader in Assembly Language
This guide offers a step-by-step approach to writing a bootloader that displays "Hello, World!" upon execution.
Read more here. - Hello World Boot Loader on Bona Fide OS Developer
This tutorial provides a walkthrough for creating a "Hello, World!" bootloader, including code examples and explanations.
Read more here.
Happy coding—and may your protected mode journeys be bug-free!