Did I enter protected mode right?

Index > OS Construction > Did I enter protected mode right?

Author

Thread

IsaacZ

Joined: 05 Dec 2025
Posts: 10

IsaacZ 08 May 2026, 14:29

I've been trying to develop a very simple, and solid 32-Bit Operating System. I am having a struggle figuring out if I entered protected mode right. Do ignore the kernel, I'm not fixing that now until the bootloader is solid. I would like to know anything I did wrong and how to fix it. I also recognise the IDT isn't finished, but like I said, I'm focusing on bootloader.

Code:

;16-bit Main registers include AX(Accumulator), BX(Base), CX(Count), DX(Data). While 8-bit values are AH(high 8-bit) and AL(low 8-bit), etc.
;16-bit Index registers include SI(Source Index), DI(Destination Index), BP(Base Pointer), and SP(Stack Pointer).
;Segment registers include CS(Code Segment), DS(Data Segment), ES(Extra Segment), FS, GS, SS(Stack Segment)
;The pointer regusters are IP(Instruction Pointer), SP(Stack Pointer), and BP(Base Pointer)
;pusha pushes in this order: AX, CX, DX, BX, SP, BP, SI, DI. And is popped (pop) in reverse
;Overflow Flag, Direction Flag, Inturrupt Flag, Trap Flag, Sign Flag, Zero Flag, Adjust Flag, Parity Flag, and Carry Flag are bits within 16-bit Status registers
;cmp compares two values, and is usually followed by a jump
;Jumps include jmp(Unconditional Jump), jne(Jump if not equal to), jg(Jump if greater than), jl(Jump if less than), jge(Jump if greater than, or equal to), jle(Jump if less than, or equal to)
;$ is the current memory adress, $$ is the start of the current memory adress.
;ah=0e(hex)and then calling Inturrupt 10(hex) switches computer to printing mode. (Turn on your digital typewriter, starring BIOS!)
;ah=0 and then calling inturrupt 16 makes computer wait for keypress.
;int(Inturrupt) is the CPU's responce to an event that needs attention
;hlt(Halt) halts the CPU until the next inturrupt
;0x* and *h are both specifying hexadecimal values
;Define diractives are db(Define Byte), dw(Define Word), dd(Define Doubleword), dq(Define Quadword), dt(Define Ten Bytes)
;inc(Increment) is like add 1 to a value, exept inc preserves Carry flag while add sets all the flags. Example(inc al == add al, 1)
;0xAA55 or 0x55, 0xAA  is the boot signature, literally makes the computer boot.

;Memory Map:
;0000:7C00 - Bootloader
;0000:7E00 = Kernel

;MUST keep DS at 0 while in bootloader, 0x1000 while in kernel.
;MUST keep ES at 0 while in bootloader, 0x1000 while reading from disk, and in kernel.
;When jump to kernel, CS becomes 0x1000

;PROTECTED MODE STRUCTURE
;[Y] CLEAR INTERRUPTS
;[Y] ENABLE A20 LINE
;[Y] LOAD IN GDT (Global Descriptor Table)
;[?] LOAD IN IDT (Interrupt Descriptor Table)

org 0x7C00
use16

bootloader:
    cli
    cld
    xor ax, ax
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov bp, 0x7000
    mov sp, bp
    mov [boot_drive], dl

;Set video mode 0x03
    mov ax, 0x0003
    int 0x10

enable_a20:
    in al, 0x92
    or al, 0x2
    out 0x92, al
    sti

read_disk:
    mov ah, 0x02
    mov al, 5
    mov ch, 0
    mov cl, 2
    mov dh, 0
    mov dl, [boot_drive]
    mov bx, 0x7E00
    int 0x13
    jc disk_error

CODE_OFFSET equ gdt_code - gdt_start ;Also a synonym to 0x08
DATA_OFFSET equ gdt_data - gdt_start ;Also a synonym to 0x10

load_gdt:
    cli
    lgdt [gdt_descriptor]
    mov eax, cr0
    or eax, 1
    mov cr0, eax
    jmp CODE_OFFSET:start_protected

use32
start_protected:
    mov ax, DATA_OFFSET
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax
    mov ss, ax
    mov esp, 0x90000

    mov edi, 0xB8000
    mov al, '?'
    mov ah, 0x0F
    mov [edi], ax

    jmp 0x08:0x7E00

gdt_start:
gdt_null:
    dd 0x0
    dd 0x0
gdt_code:
    dw 0xFFFF
    dw 0
    db 0
    db 10011010b
    db 11001111b
    db 0
gdt_data:
    dw 0xFFFF
    dw 0
    db 0
    db 10010010b
    db 11001111b
    db 0
gdt_end:
gdt_descriptor:
    dw gdt_end - gdt_start - 1
    dd gdt_start

disk_error:
    mov ah, 0x0E
    mov al, '!'
    mov bh, 0
    int 0x10

    hlt
    jmp $

boot_drive db 0x00

times 510-($-$$) db 0
dw 0xAA55

org 0x7E00
use32
kernel_start:
    mov ax, 0x10
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax
    mov ss, ax
    mov esp, 0x90000

;IDT
load_idt:
    lidt [idt_descriptor]

;MAIN FOR NOW
    mov ebx, test_msg
    call print_string32

    jmp $

print_string32:
    pusha
    mov edx, 0xB8000
.loop:
    mov al, [ebx]
    cmp al, 0
    je .done

    mov ah, 0x0F
    mov [edx], ax
    add ebx, 1
    add edx, 2
    jmp .loop
.done:
    popa
    ret

test_msg db 'PROTECTED AND 32-BITS.', 0

isr_default:
    iret
idt_start:
    dw isr_default and 0xFFFF
    dw 0x08
    db 0
    db 10001110b
    dw isr_default shr 0x10
    times 255 dq 0
idt_end:
idt_descriptor:
    dw idt_end - idt_start - 1
    dd idt_start

Thanks for reading Surprised

_________________
Developing a new Operating System...

08 May 2026, 14:29

SeproMan

Joined: 11 Oct 2009
Posts: 76
Location: Belgium

SeproMan 09 May 2026, 23:02

Looks ok, but following are some observations.

Quote:

;0000:7E00 = Kernel
;MUST keep DS at 0 while in bootloader, 0x1000 while in kernel.

Here's a conflict about where the kernel resides!

Code:

    mov ss, ax
    mov bp, 0x7000
    mov sp, bp

Always load SP directly beneath loading SS. Then you don't need those CLI and STI instructions. So better write:

Code:

    mov bp, 0x7000
    mov ss, ax
    mov sp, bp

Code:

    hlt
    jmp $

In order for HLT to stay in effect, the better choice here would be:

Code:

    cli
    hlt
    jmp $-2

Loading those 5 disk sectors could go wrong for any odd reason. Therefore it is better to give it a few tries (eg. 5) and not give up on the very first error. And to avoid some potential errors, it is more robust to read the 5 successive sectors one by one (AL=1) instead of all together (AL=5).

The code at disk_error should carry a USE16 predicate.

The code at kernel_start should not be repeating setting up the segment registers.

_________________
Real Address Mode.

09 May 2026, 23:02

IsaacZ

Joined: 05 Dec 2025
Posts: 10

IsaacZ 10 May 2026, 21:30

Thanks for bringing those problems to my attention @SeproMan, I'm currently fixing those problems. I also found another potential problem, which I fixed by adding this:

Code:

org 0x7C00
use16

jmp 0x0000:start ;Some BIOSes would jump to 07C0:0000 which would make CS be 0x07C0.

start:
    ;Setup registers

10 May 2026, 21:30

Core i7

Joined: 14 Nov 2024
Posts: 164
Location: Socket on motherboard

Core i7 11 May 2026, 06:34

IsaacZ wrote:

I would like to know anything I did wrong and how to fix it.

1. Load the kernel at the lowest available address to get more real-mode kernel space. Address 0x0600 is preferred, rather than 0x7E00, as you currently have, which is specified in int-13h.

2. There are also errors in the descriptor formatting. For example, the GDT should be aligned on a 4KB boundary in virtual memory, but since you haven't implemented paging yet, define it at least at 16 bytes.

Also, pay attention to the "Accessed" bit in the descriptor's Type field. To avoid a GP# exception, it's best to set it to 1 beforehand. Even though the CPU automatically sets this bit the first time it accesses the code/data segment and updates the descriptor, the GDT may not be writable, and then a GP# exception will occur. While you're in real mode, this isn't critical, but after switching to protected mode, the error will be difficult to detect.

For example, here are the WinDbg debugger logs for a GDT x32 table dump request - note the flags in the last column:

Code:

kd> dg 0 80
                                     P  Si  Gr  Pr  Lo
Sel     Base     Limit      Type     l  ze  an  es  ng   Flags
----  -------- --------  ----------  -  --  --  --  --  --------
0000  00000000 00000000  <Reserved>  0  Nb  By  Np  Nl  00000000
0008  00000000 ffffffff  Code RE Ac  0  Bg  Pg  P   Nl  00000c9b
0010  00000000 ffffffff  Data RW Ac  0  Bg  Pg  P   Nl  00000c93
0018  00000000 ffffffff  Code RE Ac  3  Bg  Pg  P   Nl  00000cfb
0020  00000000 ffffffff  Data RW Ac  3  Bg  Pg  P   Nl  00000cf3
0028  80042000 000020ab  TSS32 Busy  0  Nb  By  P   Nl  0000008b
0030  ffdff000 00001fff  Data RW Ac  0  Bg  Pg  P   Nl  00000c93
0038  7ffde000 00000fff  Data RW Ac  3  Bg  By  P   Nl  000004f3
0040  00000400 0000ffff  Data RW     3  Nb  By  P   Nl  000000f2
0048  00000000 00000000  <Reserved>  0  Nb  By  Np  Nl  00000000
0050  8054a100 00000068  TSS32 Avl   0  Nb  By  P   Nl  00000089
0058  8054a168 00000068  TSS32 Avl   0  Nb  By  P   Nl  00000089
0060  00022f40 0000ffff  Data RW Ac  0  Nb  By  P   Nl  00000093
0068  000b8000 00003fff  Data RW     0  Nb  By  P   Nl  00000092
0070  ffff7000 000003ff  Data RW     0  Nb  By  P   Nl  00000092
0078  80400000 0000ffff  Code RE     0  Nb  By  P   Nl  0000009a
0080  80400000 0000ffff  Data RW     0  Nb  By  P   Nl  00000092

Now let's request the value of the GDTR register, and immediately output a dump of this GDT table:

Code:

kd> r @gdtr
       gdtr = 8003f000    <--------- 4Kb padding

kd> dqs 8003f000 L5
8003f000   00000000`00000000
8003f008   00cf9b00`0000ffff    <--- Code Ring(0)
8003f010   00cf9300`0000ffff    <--- Data Ring(0)

8003f018   00cffb00`0000ffff    <--- Code Ring(3)
8003f020   00cff300`0000ffff    <--- Data Ring(3)

Here you can see that bit (A) is set to 1 (see bytes 0x9b and 0x93).
But if you decode the values from your table, bit (A) is cleared everywhere—here's the code:

Code:

format  pe64 console
include 'win64ax.inc'
entry start
;//------------------
section '.data' data readable writeable
gdtCode    dw  0xffff,0
           db  0, 10011010b, 11001111b, 0

gdtData    dw  0xffff,0
           db  0, 10010010b, 11001111b, 0
;//------------------
section '.text' code readable executable
start:  sub     rsp,8
        mov     rax,qword[gdtCode]
        mov     rbx,qword[gdtData]
       cinvoke  printf,<10,' Code desc: 0x%p',\
                        10,' Data desc: 0x%p',0>,rax,rbx
       cinvoke  getch
       cinvoke  exit,0
;//------------------
section '.idata' import data readable writeable
library  msvcrt,'msvcrt.dll'
import   msvcrt,printf,'printf',getch,'_getch',exit,'exit'


;//------- RESULT ------------//
Code desc: 0x00CF9A000000FFFF
Data desc: 0x00CF92000000FFFF

So your table should look like this:

Code:

align 16            ;<------------ Padding
gdt_start:  dq  0 
gdt_code0   dq  0x00cf9b000000ffff
gdt_data0   dq  0x00cf93000000ffff
gdt_code3   dq  0x00cffb000000ffff
gdt_data3   dq  0x00cff3000000ffff

gdt_desc:   dw  $ - gdt_start -1
            dd  gdt_start

Last edited by Core i7 on 11 May 2026, 12:40; edited 1 time in total

11 May 2026, 06:34

Core i7

Joined: 14 Nov 2024
Posts: 164
Location: Socket on motherboard

Core i7 11 May 2026, 08:16

Moreover, while you're in RM, don't rush to switch to PM, but gather as much system information as possible using interrupts. In protected mode, you won't have BIOS services, and you'll have to do everything manually.

For example, for virtual memory with PAGE_TABLE, you'll need to know the size of physical memory above 1MB. Of course, you can read the SMBIOS table in RM, but it only shows the size of installed DDR-SDRAM. However, using the int-15h AX=0xE820 interrupt, you can get a full map of free/used memory, and then create a PAGE_TABLE based on that.

In addition to memory, you need to know the number of CPU cores (if you plan on multitasking), collect information about physical devices on the motherboard using "PCI-Config-Spase", determine the hard drive type ATA/SATA/SSD/NVMe (you'll need to write a driver for PM), and much more. Once finished, collect all this data into a single structure somewhere in memory, and only then switch to protected mode. This will save you a lot of time and frustration in the future.

Here are the sources for the OS I recently wrote: https://board.flatassembler.net/topic.php?t=23907

11 May 2026, 08:16

IsaacZ

Joined: 05 Dec 2025
Posts: 10

IsaacZ 11 May 2026, 13:23

Thanks @Core i7, I'm fixing some of those problems right now, such as the GDT

Core i7 wrote:

Moreover, while you're in RM, don't rush to switch to PM, but gather as much system information as possible using interrupts. In protected mode, you won't have BIOS services, and you'll have to do everything manually.

I will, but at the moment, I'm trying to make my current code as solid as possible before moving on, Thanks again Smile

_________________
Developing a new Operating System...

11 May 2026, 13:23

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum