flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Projects and Ideas > Writing a Hypervisor - ring -1 debugger

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
smoke



Joined: 16 Jan 2006
Posts: 47
Writing a Hypervisor - ring -1 debugger
Hello everyone!

I'm thinking of writing a hypervisor based kernel mode debugger and I was wondering if anyone has any books/articles/whatsoever on writing kernel mode debuggers? Im inexperienced in writing debuggers, however I do know how to code in x86 assembly, write drivers for windows, etc. I've already used google in hope to get some information but with no success. I'm thinking of using fasm in combination with C/C++ for the project. So if anyone has any information to direct me in right way, I'd be very grateful.

PS: I didnt really know where to put this thread since it's not retated to any particular OS. So if I've putten it to the wrong section, please move it to the right one. Thanks in advance! Smile

Regards,
smoke
Post 10 Oct 2011, 16:54
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic
Hi, this team did something you would like. At the time I read their work (source code) it lacked a lot of things I would like to have there (more than year ago, maybe they already implemented that).
http://code.google.com/p/hyperdbg/
In principle, your hypervisor intercepts keyboard IO to handle keystrokes so it has some input, you scan PCI bus to find graphic adapter framebuffer, map it into hypervisor and write there (direct pixels to screen) so it has some output.
Another choice is to have 2 PCs and send data via cable (protocol for serial port is simple, for USB or firewire more complicated).
Personaly I developed something else, I have hypervisor as replacement for ms win x64 debug / memory / threads / processes system calls. It is capable to debug ring3 applications, not ring0. Calls hypervisor for freezing threads / manipulate memory / manipulate registers / handle debug events. Does not call ms win APIs for that. Something like stealth debugger.
http://fdbg.x86asm.net/hdbg/hdbg.html
here some video where it steps through HideFromDebugger (you must switch to full screen and switch to HD quality to read the small text there, else it is hardly readable !!!)
http://www.metacafe.com/watch/6104225/hdbg/

I may give you advices / correct mistakes. Cannot release source code / binaries as big part of the work is property of one company I developed some technologies for. Later I extended the work and made that debugger (hdbg) as private activity.

Cheers, Feryno
Post 11 Oct 2011, 07:19
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7110
Location: Slovakia
I did "full" hypervisor debugger few years ago, eg. one where also user interface was running in hypervisor host mode, outside of debugged OS, similar to good-old Soft-Ice, but then I had to discontinue for various reasons. I still dig the idea, and hope someone will sometimes produce something like that.
Post 11 Oct 2011, 08:09
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Thanks for the info Feryno I'll look into that, and thanks for pointing me in the right direction, i honestly didnt know where and what to look for Smile Any ideas what to start working on first? Disasm engine, hypervisor?

vid: Yeah, thats exactly what I want to do, I'm inspired by SoftICE and I hate it that we dont have a decent kernel debugger for x86-64 yet (not counting Windbg for Windows ofcourse Smile).
Post 11 Oct 2011, 14:04
View user's profile Send private message Reply with quote
ouadji



Joined: 24 Dec 2008
Posts: 1081
Location: Belgium

Quote:
we dont have a decent kernel debugger for x86-64 yet

Syser.

i use Syser (ring0 degugger) every day 18h/24h
(sometimes even at night) Razz

_________________
I am not young enough to know everything (Oscar Wilde)- Image
Post 11 Oct 2011, 14:16
View user's profile Send private message Send e-mail Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Last time I checked Syser is 32bit Smile
Post 11 Oct 2011, 14:35
View user's profile Send private message Reply with quote
ouadji



Joined: 24 Dec 2008
Posts: 1081
Location: Belgium

Indeed, you're right. Syser is a 32bits degugger, not 64bit.
For 64bit, there are only WinDbg.

WinDbg is a big shit, really !

_________________
I am not young enough to know everything (Oscar Wilde)- Image
Post 11 Oct 2011, 15:30
View user's profile Send private message Send e-mail Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7110
Location: Slovakia
smoke: It's trickier than you may think. Virtualization wasn't designed for this. Keep us posted if you start any actual work on this.
Post 12 Oct 2011, 09:25
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic

smoke wrote:
Any ideas what to start working on first? Disasm engine, hypervisor?


certainly hypervisor

which platform do you plane (AMD/Intel) ?
If you have only Intel CPU you may develop AMD hypervisor using SimNow simulator (I did it in such way)
http://developer.amd.com/tools/simnow/pages/default.aspx
for Intel hypervisor the best way is to have CPU (although some free emulators are capable to emulate intel virtualization technology)
Post 12 Oct 2011, 13:34
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
vid: yeah, I can imagine Smile I gotta ask though.. are the developer manuals enough to get a complete picture of how to implement the hypervisor the right way? Or will I need to list through the sources of open source virtualization solutions?

Feryno: I have an AMD cpu, so I'll probably start working on the AMD's version of the hypervisor and then upgrade it to Intel. Smile Which one's better anyways?

PS: I'll keep you posted on my work Smile
Post 12 Oct 2011, 16:51
View user's profile Send private message Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7110
Location: Slovakia
Manuals are 95% enough, but you need to get a very good grasp of them. There was one exceptionally badly explained thing (something like "VMMON region" I think), but otherwise they were okay. I went through them dozens of times while looking for where the F-ing error could be.
Post 12 Oct 2011, 17:56
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Hopefully AMD's documents are better. Smile Well.. talk to you when I have something!

Regards,
smoke
Post 15 Oct 2011, 18:41
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic
no worry, I'll help you
for AMD it is a bit easier to setup VMCB but the vm exit handler is much more difficult than on intel
will you start you hypervisor as driver?
Post 17 Oct 2011, 07:09
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Feryno: Thanks, I'd be very glad Smile Yeah I'm thinking of writing a driver based hypervisor for Win 7 64bit as a start
Post 17 Oct 2011, 08:37
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic
modify your boot.ini or use bcdedit depending your OS (bcdedit for win7) to run on 1 CPU instead of SMP
bcdedit /set onecpu on
then
MmAllocateContiguousMemorySpecifyCache with MmCached
http://msdn.microsoft.com/en-us/library/windows/hardware/ff554464%28v=vs.85%29.aspx
I suggest you to allocate 2 MB (you'll later support more CPUs, e.g. 64 CPUs will eat about 1 MB)
then
MmGetPhysicalAddress
http://msdn.microsoft.com/en-us/library/windows/hardware/ff554547%28v=VS.85%29.aspx
then setup VMCB according AMD manual and prepare vm exit handler also

after you succeed, I'll tell you how to run that on all CPUs in SMP (about 5 OS specific system calls)

certainly install something like windbg so you can analyze crash dump file (I got certainly more than 1000 reboots/BSODs/hangs during few years)

1 trick helped me a lot to know where I got (how far) - I put few instructions which turned on all LEDs on my keyboard (e.g. I had hang but know that something passed OK so the hang was later than the procedure turning on LEDs)

be patient enough (few months of studying manuals, programming, trying, rebooting), never resign
Post 17 Oct 2011, 12:14
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Thanks, I'll do that. Can I do all the initialization from DriverEntry? Or should I initialize the hypervisor from a dispatch routine?
Post 17 Oct 2011, 12:33
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic
better from dispatch (because later you'll add turning off hypervisor on the running OS without necessity of OS restarting)
but no problem to do it from init
let the first 1MB of mem for shared things of hypervisor (paging tables, code, ...) and split the second 1 MB into 64 parts for 64 CPUs, every CPU will have 4 kB for HSAVE, 4kb for VMCB, 8 kB for stack (at first you setup only CPU00, later you implement SMP)
setup also GS base for hypervisor so you can access per CPU private data (analogy of ms win KPRCB) using GS prefix from hypervisor, I suggest to hold at least GDT, TSS, GS base (self pointer) there
the easiest vm exit loop is something like:
http://board.flatassembler.net/topic.php?p=110033#110033
Post 18 Oct 2011, 08:08
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
smoke



Joined: 16 Jan 2006
Posts: 47
Hey Feryno, I'm sorry for the late reply but i was a bit busy with school so i couldnt work on the project as much as i'd hoped.

I have a question (maybe a stupid one). I'm kinda half way done with setting up the VMCB. When setting up the Save State area.. I can just use the values of segment registers, stack pointer, control registers, etc right?

Greets,
smoke
Post 03 Nov 2011, 15:11
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 415
Location: Czech republic, Slovak republic
Hi, I was busy the whole last week also.

here sample how to setup something:

; rsi virtual memory
; rdi physical memory
; rbp CPU number * 4kB (every CPU uses 4 pages of 4 kB, 1 page for HSAVE, 1 page for VMCB, 2 pages for stack)
; N.B. big offsets like the 00301000 because hypervisor occupies 4 MB of memory, first 2 MB are used for translation tables for memory virtualization, next 1 MB for paging tables of hypervisor, its code, ..., the last 1 MB is used for 64 CPUs, every CPU occupies 16 kB (4 pages)


0000000000010A75: lea rax,[rdi+rbp*4+00301000] ; pointer to physical memory of the VMCB of the current CPU
0000000000010A7D: vmsave ; read manual for more, the rest will be saved manually as follows:
0000000000010A80: sgdt [rsp+06]
0000000000010A85: sidt [rsp+16]
0000000000010A8A: movzx r8d,word [rsp+06]
0000000000010A90: mov rdx,[rsp+08]
0000000000010A95: movzx eax,word [rsp+16]
0000000000010A9A: mov rcx,[rsp+18]
0000000000010A9F: mov [rsi+rbp*4+00301464],r8d
0000000000010AA7: mov [rsi+rbp*4+00301468],rdx
0000000000010AAF: mov [rsi+rbp*4+00301484],eax
0000000000010AB6: mov [rsi+rbp*4+00301488],rcx
0000000000010ABE: mov ecx,es
0000000000010AC0: mov [rsi+rbp*4+00301400],es
0000000000010AC7: lar eax,cx
0000000000010ACA: jnz 0000000000010AD9
0000000000010ACC: shr eax,08
0000000000010ACF: shr ah,04
0000000000010AD2: mov [rsi+rbp*4+00301402],eax
0000000000010AD9: lsl eax,cx
0000000000010ADC: jnz 0000000000010AF3
0000000000010ADE: mov [rsi+rbp*4+00301404],eax
0000000000010AE5: call 0000000000010F00
0000000000010AEA: jc 0000000000010AF3
0000000000010AEC: mov [rsi+rbp*4+00301408],eax

....

0000000000010F00: push rcx
0000000000010F01: push rdx
0000000000010F02: push rbx
0000000000010F03: lea ebx,[r8+01]
0000000000010F07: test cl,04
0000000000010F0A: jz 0000000000010F30
0000000000010F0C: sldt word eax
0000000000010F0F: lsl ebx,ax
0000000000010F12: stc
0000000000010F13: jnz 0000000000010F4C
0000000000010F15: inc ebx
0000000000010F17: mov eax,[rdx+rcx*1+02]
0000000000010F1B: mov rdx,[rdx+rcx*1+04]
0000000000010F20: and eax,00FFFFFF
0000000000010F25: and rdx,FFFFFFFFFF000000
0000000000010F2C: or rdx,rax
0000000000010F2F: nop
0000000000010F30: and cl,F8
0000000000010F33: cmp ebx,ecx
0000000000010F35: jc 0000000000010F4C
0000000000010F37: mov eax,[rdx+rcx*1+02]
0000000000010F3B: mov ecx,[rdx+rcx*1+04]
0000000000010F3F: and eax,00FFFFFF
0000000000010F44: and ecx,FF000000
0000000000010F4A: or eax,ecx
0000000000010F4C: pop rbx
0000000000010F4D: pop rdx
0000000000010F4E: pop rcx
0000000000010F4F: ret
Post 07 Nov 2011, 10:55
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
guignol



Joined: 06 Dec 2008
Posts: 230

vid wrote:
, but then I had to discontinue for various reasons.

Gavel Republik...
Post 05 Jan 2012, 15:59
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2013, Tomasz Grysztar.