flat assembler
Message board for the users of flat assembler.

Index > OS Construction > hypervisor

Author
Thread Post new topic Reply to topic
Feryno



Joined: 23 Mar 2005
Posts: 509
Location: Czech republic, Slovak republic
Feryno 01 Feb 2010, 10:34
I decided to start this thread thanks to:

- f0dder who pointed me to emulators capable to emulate virtualization
http://board.flatassembler.net/topic.php?p=108769#108769

- booter who decided to work on hypervisors
http://board.flatassembler.net/topic.php?t=11085

developping an hypervisor for intel is now available to everybody, you may developp it as driver for running OS or you may load it at boot stage
development of a hypervisor is very close to constructing an OS so this thread is in OS construction

I spent few hours to put everything together, so you need to do this:

----- compiling bochs to support intel vmx -----

download bochs 2.4.2 source from http://bochs.sourceforge.net and upack it

update the file .conf.linux (or dedicated file for another platform)
the necessities are these switches
--enable-vmx (I hope there is no explanation needed why it is necessary)
--enable-smp (if your hypervisor would run at more CPUs)
--enable-sse=3 (or even higher than 3)
--enable-sse-extension (vmx requires at least SSE3+SSE_extension enabled or SSE4 or higher)
--enable-x86_64 (if you plane to work on 64 bit hypervisor)

Code:
#!/bin/sh
#
# .conf.linux
#

#which_config=normal
which_config=plugins

CC="gcc"
CXX="c++"
CFLAGS="-Wall -O3 -fomit-frame-pointer -pipe"    # for speed
#CFLAGS="-Wall -g -pipe"                         # for development
CXXFLAGS="$CFLAGS"

export CC
export CXX
export CFLAGS
export CXXFLAGS


case $which_config in

  normal)

    #######################################################################
    # configuration 1 for release binary RPMs
    # Include a few standard options, speed optimizations, X11 only.
    #######################################################################
    ./configure --enable-sb16 \
                --enable-ne2000 \
                --enable-all-optimizations \
                --enable-cpu-level=6 \
                --enable-x86-64 \
                --enable-sse=3 \
                --enable-sse-extension \
                --enable-vmx \
                --enable-smp \
                --enable-pci \
                --enable-acpi \
                --enable-clgd54xx \
                --enable-usb \
                --enable-usb-ohci \
                --enable-show-ips \
                ${CONFIGURE_ARGS}
    ;;

  plugins)
    #######################################################################
    # configuration 2 for release binary RPMs
    # Include plugins, every possible gui.
    #######################################################################
    ./configure --enable-sb16 \
                --enable-ne2000 \
                --enable-all-optimizations \
                --enable-cpu-level=6 \
                --enable-x86-64 \
                --enable-sse=3 \
                --enable-sse-extension \
                --enable-vmx \
                --enable-smp \
                --enable-pci \
                --enable-acpi \
                --enable-clgd54xx \
                --enable-usb \
                --enable-usb-ohci \
                --enable-plugins \
                --enable-show-ips \
                --with-all-libs \
                ${CONFIGURE_ARGS}
    ;;

esac    


run these commands:
./.conf.linux
make
make install


----- creating disk image -----
prepare enough space on your disk (2 GB would be enough), then run this command:
bximage

when promted type these settings:
hd
flat
2000
hd0.img


----- installing win XP -----
I strongly suggest you to use 1 cpu for installation (simulating 2 CPUs in system drops the performace about twice, so you won't install 5 hours but you will 10 hours)

edit the file bochsrc.bxrc, especially the path for XP installation source which is at the line beginning with ata1-slave
Code:
###############################################
# bochsrc.bxrc file for windows xp disk image #
###############################################

# how much memory the emulated machine will have
megs: 256

# this is the fastest emulation, very suitable for fast installation:
cpu: vendor_string=GenuineIntel, msrs=msrs.def
# Intel Core 2 Duo
#cpu: count=1:2, vendor_string=GenuineIntel, msrs="msrs.def", brand_string="Intel(R) Core(TM)2 Duo CPU     T8100  @2.10GHz"
# Intel Core i7 (count=1, cores=4, threads_per_core=2), this is very slow emulation:
#cpu: count=1:4:2, vendor_string=GenuineIntel, msrs="msrs.def", brand_string="Intel(R) Core i7"

# hard disk
ata0: enabled=1, ioaddr1=0x1f0, ioaddr2=0x3f0, irq=14
ata0-master: type=disk, path="hd0.img", mode=flat, cylinders=4063, heads=16, spt=63
ata1: enabled=1, ioaddr1=0x170, ioaddr2=0x370, irq=15
ata1-slave:  type=cdrom, path=/mnt/sda4/Big_Fucking_Gun/044/windows_xp_professional_paneuro.iso, status=inserted
# if the cdrom source is in drive, then you may use path=/dev/cdrom

# choose the boot disk.
#boot: c
boot: cdrom, disk

# where do we send log messages?
log: bochsout.txt

# enable the mouse
mouse: enabled=1    


some MSRs must be enabled and set properly also
edit the msrs.def file:
Code:
###############################################
# msrs.def file for Intel Core 2 Duo CPU #
###############################################
#
#                   ----------------------------------
#                      Bochs CPU MSRs configuration
#                   ----------------------------------
#
# LEGEND:
# ------
#
#    MSR ADDRESS - MSR address in hex (supplied in ECX register for RDMSR/WRMSR)
#    MSR TYPE    - MSR type, see below
#
#    The following fields have any meaning for MSRs with no type only:
#
#    RESET_HI    - reset value of the MSR (bits 63:32)
#    RESET_LO    - reset value of the MSR (bits 31:00)
#
#    NOTE: the value of the MSR doesn't change on INIT (software reset).
#
#    RSRVD_HI    - mask of reserved bits (bits 63:32)
#    RSRVD_LO    - mask of reserved bits (bits 31:00)
#
#    NOTE: #GP fault will be generated when trying to modify any of MSR
#          reserved bits.
#
#    IGNRD_HI    - mask of ignored bits (bits 63:32)
#    IGNRD_LO    - mask of ignored bits (bits 31:00)
#
#    NOTE: Ignored bits will keep their reset value, all writes to these 
#          bits are ignored.
#
# MSR TYPES:
# ---------
#
#    0 - No type.
#    1 - MSR contains linear address, 
#        #GP if writing non-canonical address in 64-bit mode.
#    2 - MSR contains physical address,
#        #GP if writing a value which exceeds emulated physical address size.
#


# ADDRESS  TYPE   RESET_HI   RESET_LO    RSRVD_HI   RSRVD_LO   IGNRD_HI   IGNRD_LO
# ---------------------------------------------------------------------------------
#   0x02c     0    00000000   00000000    00000000   00000000   00000000   00000000
   0x03a     0    00000000   00000005    00000000   00000000   00000000   00000000
   0x08b     0    0000060C   00000000    00000000   00000000   00000000   00000000
   0x485     0    00000000   000403C0    00000000   00000000   00000000   00000000    


run this command:
bochs -q -f bochsrc.bxrc

copying files last about half an hour and the second stage about 5 hours at an 6 years old AMD64 Athlon 3200+ running at 2 GHz with 1 GB RAM under Fedora Core 3

after you finish installation and everything is set up correctly, you may increase the count of CPUs in system (this will slow donw the performace very much, just edit the line for cpu in bochsrc.bxrc and reboot, then XP discovers more cpus, it installs drivers and kernel for SMP
don't try to perform the XP installation at SMP (more CPUs than 1) because it is very slow, increase the count of CPUs only after you finish installation)



later I'll post some screenshots about sucessful test of 32 bit hypervisor skeleton, I'm writing this at hurry, in case I missed something, don't hesitate to post here


If you read my presentation about hypervisor shutdown
http://board.flatassembler.net/topic.php?p=100097#100097
under bochs the shutdown required to insert one instruction of writing into stack, e.g. the code sequence
Code:
mov ecx,cr3 ; this is in my presentation
mov cr3,ecx ; this is in my presentation
mov [esp],eax ; this is required by bochs    
Post 01 Feb 2010, 10:34
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3175
Location: Denmark
f0dder 01 Feb 2010, 10:59
Feryno wrote:
--enable-sse=3 (or even higher than 3)
--enable-sse-extension (vmx requires at least SSE3+SSE_extension enabled or SSE4 or higher)
Hm, this sounds a bit weird? While you probably won't find a processor with VMX but without SSE3, afaik there isn't anything in VMX itself that requires SSE?

Feryno wrote:
I strongly suggest you to use 1 cpu for installation (simulating 2 CPUs in system drops the performace about twice, so you won't install 5 hours but you will 10 hours)
Smile - I assume there's no way to utilize multicore systems for emulating in bochs... an important distinction between bochs and other Virtual Machines: others try to run code natively as much as possible, whereas bochs is 100% emulation... which is both it's strength and it's weakness. Mostly strength Smile

Feryno wrote:
copying files last about half an hour and the second stage about 5 hours at an 6 years old AMD64 Athlon 3200+ running at 2 GHz with 1 GB RAM under Fedora Core 3
Ouch! Better make a backup of your freshly-installed image before playing around :-O

Anyway, good work, and glad to see you figured it out so I didn't have to Smile
Post 01 Feb 2010, 10:59
View user's profile Send private message Visit poster's website Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 509
Location: Czech republic, Slovak republic
Feryno 01 Feb 2010, 13:31
bochs requires SSE3 + SSE3_extension (or SSE4 or even higher) to enable bit 9 of ECX of CPUID_EAX=1 (I found that reading source code of bochs)

cpuid.cc
Code:
#if (BX_SUPPORT_SSE >= 4) || (BX_SUPPORT_SSE >= 3 && BX_SUPPORT_SSE_EXTENSION > 0)
  features |= (1<<9);            // support SSE3E
#endif    


without enabling that - if you strongly follow manuals then you discover that CPU support virtualization but doesn't support VMX-Capabality MSR so you will be unable to set correctly some VMCS fields (you are even unable to execute the VMXON instruction because you don't know VMCS revision identifier got by reading MSR_IA32_VMX_BASIC = 480h), you won't be either able to construct vm exit handler for reading/writing CR0/CR4 (you need some bit masks from VMX-capability MSRs to protect some bits of control registers against writing)

this is fragment from my code checking CPU:
Code:
align 4
check_CPU:
; returns 0 = CPU OK, other = CPU disliked by this hypervisor design

 push    ebx                     ; CPUID destroys EBX register so we must preserve it

; discovering GenuineIntel CPU
  xor     eax,eax
     cpuid
       sub     ebx,'Genu'
        sub     edx,'ineI'
        sub     ecx,'ntel'
        or      ebx,edx
     or      ebx,ecx
     jnz     check_CPU_wrong

; DISCOVERING SUPPORT FOR VMX
; Before system software enters into VMX operation, it must discover the presence of VMX support in the processor. System software can determine whether a processor supports VMX operation using CPUID. If CPUID.1:ECX.VMX[bit 5] = 1, then VMX operation is supported.
;       mov     eax,1
; this saves 2 bytes of code but is a bit slowlier (doesn't matter, this procedure is executed jut once...)
;      push    1
;  pop     rax
; this is fast and small, RBX=0 now
  lea     eax,[ebx+1]

     cpuid
       test    cl,1 shl 5
  jz      check_CPU_wrong

; the VMX-Capabality MSRs (480h-48Bh) may be read only if CPUID.01H:ECX.[bit 9] is set to 1
      test    ch,1 shl (9-8)
      jz      check_CPU_wrong

check_CPU_OK:
    xor     eax,eax
     pop     ebx
 ret

check_CPU_wrong:
;    mov     eax,1
; this is smaller:
     mov     al,1
        pop     ebx
 ret
    
Post 01 Feb 2010, 13:31
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 509
Location: Czech republic, Slovak republic
Feryno 01 Feb 2010, 14:41
promissed screenshots

funny how OS underestimated the CPU speed at 3 MHz...
Image
Image
Image
I measure the count of cycles for executing the CPUID instruction to validate that hypervisor is running, because the CPUID causes unconditional vm exit at Intel VMX - in bochs the change is from 5 CPU cycles to 35, in bare CPU the cycles increase about from 300 (no vmx) to 2000 (under vmx)
Post 01 Feb 2010, 14:41
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.