Detecting Real mode

Index > Main > Detecting Real mode

Goto page 1, 2 Next

Author

Thread

Kel

Joined: 11 Feb 2009
Posts: 8
Location: Australia

Kel 30 Jan 2011, 11:20

Does anyone know of a way to detect if the processor is in real mode using only 8086/8088 instruction set as I have been thinking of creating a cpuid routine that not only detects the type of processor but the mode the processor is running in.

The logic goes:

Detect if processor is in real mode, if in real mode test for what type of processor if not in real mode then what type of emulation and processor then save results for later

The logic I haven't really worked out yet so if someone has suggestions it would be much appreciated

Thanks in advance

30 Jan 2011, 11:20

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland

Tomasz Grysztar 30 Jan 2011, 13:20

The safe way would be to first detect whether you have 286+ processor (perhaps with something like POPF/PUSHF) and then use SMSW to check whether you are in real mode or V86. Why would you like to check for real mode first?

30 Jan 2011, 13:20

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 30 Jan 2011, 14:37

Here is some old code I have to detect five CPU modes. It does not detect 64-bit modes.

Code:

  format binary

;all jumps must be short to ensure correct operation regardless of current mode or processor

       use16
       push    sp                              ;becomes "PUSH ESP" in 32 bit modes
       pop     bp                              ;becomes "POP EBP" in 32 bit modes
        cmp     sp,bp                           ;becomes "CMP ESP,EBP" in 32 bit modes
    jnz     short SystemIsInRealMode16      ;8086 does not have protected mode
  mov     cl,16
       smsw    ax                              ;becomes "SMSW EAX" in 32 bit modes
       shr     ax,1                            ;test for PE. Becomes "SHR EAX,1" in 32 bit modes
 jc      .ProtectedMode
  ;real or unreal mode when here
  or      ax,-1                           ;becomes "OR EAX,-1" in 32 bit modes
      shr     ax,cl                           ;becomes "SHR EAX,CL" in 32 bit modes
     jz      short SystemIsInRealMode16
  jmp     short SystemIsInUnrealMode32
  ;we are in protected mode, 80286+ only
    .ProtectedMode:
 or      ax,-1                           ;becomes "OR EAX,-1" in 32 bit modes
      shr     ax,cl                           ;becomes "SHR EAX,CL" in 32 bit modes
     jnz     short SystemIsInProtectedMode32
  ;we are in a 16 bit protected mode but which type, v8086 style or 286 style?
  ;detection of 286 vs 386+ processors while in 16-bit protected mode
  sub     sp,6
        mov     bp,sp
       sidt    pword[bp]
   pop     bp bp bp
    test    bp,0ff00h                       ;286 --> bp=ffxxh, 386+ --> bp=00xxh
  jnz     short SystemIsInProtectedMode16 ;286 processor does not have V8086 mode
     pushfd                                  ;386+ processors have 32 bit registers
      pop     eax
 bt      eax,17                          ;test VM flag
;these last two jumps can now be long form (0f 8x lo hi) if needed
 jc      SystemIsInV8086Mode16           ;V8086 task on 386+ processor
       jmp     SystemIsInProtectedMode16       ;286 task on 386+ processor

SystemIsInRealMode16:
    ;this could be any processor from 8086 up
       use16
       ;code for real 16-bit mode goes here
        ret

SystemIsInProtectedMode16:
    ;this could be 80286 and up
        use16
       ;code for protected 16-bit mode goes here
   ret

SystemIsInV8086Mode16:
    ;this could be 80386 and up
    use16
       ;code for V8086 16-bit mode goes here
       ret

SystemIsInUnrealMode32:
    ;this could be 80386 and up
   use32
       ;code for unreal 32-bit mode goes here
      ret

SystemIsInProtectedMode32:
    ;this could be 80386 and up
        use32
       ;code for protected 32-bit mode goes here
   ret

30 Jan 2011, 14:37

Ninho

Joined: 07 May 2010
Posts: 16

Ninho 30 Jan 2011, 17:26

Bravo, Revolution ! some clever code you have there.

Although, as you know, VM86 mode can't be reliably detected - since, if IOPL <3 , PUSHFD is trapped & the monitor might not store the true VM flag onto the user's stack. I don't think there is a solution to this problem (I'd love to be shewn wrong...)

30 Jan 2011, 17:26

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 31 Jan 2011, 00:27

Ninho wrote:

Although, as you know, VM86 mode can't be reliably detected - since, if IOPL <3 , PUSHFD is trapped & the monitor might not store the true VM flag onto the user's stack. I don't think there is a solution to this problem (I'd love to be shewn wrong...)

In most cases it is probably safe to assume the system is in v8086 mode. Only something like the old QNX uses PM16, and if you have code running on that system then there is really no need for the code above.

And actually it is hard to imagine a situation where that code as written is actually useful, since it would be unlikely for anyone to get this code running on any random system without first knowing the OS and having a proper executable file format. But it does show some of the quirks of the x86 code and how to exploit them for detecting things.

31 Jan 2011, 00:27

Ninho

Joined: 07 May 2010
Posts: 16

Ninho 31 Jan 2011, 10:44

revolution wrote:

And actually it is hard to imagine a situation where that code as written is actually useful, since it would be unlikely for anyone to get this code running on any random system without first knowing the OS and having a proper executable file format. But it does show some of the quirks of the x86 code and how to exploit them for detecting things.

100% agreed Smile

Hence the branching "logic" that your sample uses to separate VM86 from PM is delusive and rather pointless... just call it "best effort" !

31 Jan 2011, 10:44

DOS386

Joined: 08 Dec 2006
Posts: 1900

DOS386 02 Feb 2011, 03:59

Ninho wrote:

Although, as you know, VM86 mode can't be reliably detected

Wrong.

Quote:

since, if IOPL <3 , PUSHFD is trapped & the monitor might not store the true VM flag onto the user's stack.

Right

Quote:

I don't think there is a solution to this problem (I'd love to be shewn wrong...)

There is a solution, as just above. Do the "<80286 vs >=80286 PUSH SP test" (safe about "V86"), then if >=80286 use SMSW (safe about "V86"), if SMSW says "PM" then we got >=80386 and "V86".

_________________
Bug Nr.: 12345

Title: Hello World program compiles to 100 KB !!!

Status: Closed: NOT a Bug

02 Feb 2011, 03:59

Kel

Joined: 11 Feb 2009
Posts: 8
Location: Australia

Kel 02 Feb 2011, 07:32

Thanks for the replies. The reason I asked the question was because I wanted to eventually put the code in a start up routine of an Operating System and I wanted the OS to not only recognise the processor it was working on but if it was in a virtual environment too and make adjustments as required. I also wanted to check for the existence of real mode to get better results, though as I now see it, this is not needed.

So the logic looks like it will be check for a 8086/8088 cpu first with the push pop and if detected assume real mode (I don't like to assume this but I guess it will have to do) if not a 8086/8088 cpu test for the existence of a 286+ cpu and weather in v86 or protected mode (missed out some details for simplicity)

I will try to post some code for you guys to look at.

Thanks again for your help.

02 Feb 2011, 07:32

Ninho

Joined: 07 May 2010
Posts: 16

Ninho 06 Feb 2011, 19:20

DOS386 wrote:

There is a solution, as just above. Do the "<80286 vs >=80286 PUSH SP test" (safe about "V86"), then if >=80286 use SMSW (safe about "V86"), if SMSW says "PM" then we got >=80386 and "V86".

Why, no you are wrong, sorry. You didn't even read Revo's code did you, at least he tried hard to separate VM from native PM. But as I pointed out, it's an impossible task in the general case.

Of course... if you're a DOS program in the first place and you find you're running in protected mode, the odds are you're in VM86 anyway :=)

But you could be running in true 16-bit 80286-style protected mode instead, at CPL 1,2 or 3, and I insist there are no reliable ways to test for it if your "father" (the O.S. which begat you) did its thing properly.

06 Feb 2011, 19:20

Kel

Joined: 11 Feb 2009
Posts: 8
Location: Australia

Kel 11 Feb 2011, 04:36

Ok while I do admit that I'm a novice and probably trying to run before I can walk I do also understand that if the code is run in a virtual machine there is no reliable way to tell what I'm running on and that's fair enough the OS could just default to whatever the virtual machine allows.

But I have another Question: Why should an OS not at least attempt to find out if its running in a Virtual Machine and act accordingly if it found it is? Its just a thought? May be I have got it wrong and no doubt you guys well tell me so. Its just that if malware can detect if its in a VM why not an OS ?? May be the cpu detection routine might not be the best place for the detection of the VM but I have to at least try to experiment, ask questions and learn some more.

As for Revolution's code it is a beautiful piece of art and after having tinkered with it and learned a bit more I should probably go and Read The Manual again as well as experiment some more too.

No doubt I will have more questions to ask at some stage and I really do appreciate all of your points of views and experience.

I apologise if I came across as an idiot I certainly didn't intend to offend anyone.

11 Feb 2011, 04:36

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 11 Feb 2011, 04:59

Kel wrote:

But I have another Question: Why should an OS not at least attempt to find out if its running in a Virtual Machine and act accordingly if it found it is?

I expect the main reason is because there are a gazillion VMs out there. And trying to make special code for each VM would be somewhat pointless for an OS anyway since a VM is supposed to look like ordinary hardware. As VMs improve there is less and less chance of being able to find those tiny differences that distinguish real from virtual.

A reason malware might want to detect a VM is so that it can trick the user (for a trojan) into thinking the code is safe by doing nothing wrong inside the VM and then later going crazy when installed in real OS. Perhaps another reason is if there is a way to escape the VM (because of poor implementation or bugs) and get into the main OS. An OS has no need to have either of these behaviours so detection of VM is not needed.

http://xkcd.com/395/
http://xkcd.com/505/

11 Feb 2011, 04:59

DOS386

Joined: 08 Dec 2006
Posts: 1900

DOS386 13 Feb 2011, 04:36

Ninho wrote:

Why, no you are wrong, sorry. You didn't even read Revo's code did you

WtF

Quote:

he tried hard to separate VM from native PM. But as I pointed out, it's an impossible task in the general case.

But you could be running in true 16-bit 80286-style protected mode instead, at CPL 1,2 or 3, and I insist there are no reliable ways to test for it if your "father" (the O.S. which begat you) did its thing properly.

Code:

  ; We are in PM now, but is it proper 16-bit PM or V86 ???
  ...
  mov ds, ax
  ...

Brute-force from 0 to $FFFF and poke them all into DS ... look how "your f***ther" resolves all the GPF's and how it acts for the few values that don't fire a GPF Shocked

I consider the problem of proper 16-bit PM as irrelevant, still Wink

_________________
Bug Nr.: 12345

Title: Hello World program compiles to 100 KB !!!

Status: Closed: NOT a Bug

13 Feb 2011, 04:36

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 13 Feb 2011, 04:59

DOS386 wrote:

Code:

  ; We are in PM now, but is it proper 16-bit PM or V86 ???
  ...
  mov ds, ax
  ...

Brute-force from 0 to $FFFF and poke them all into DS

This is not a solution. The OS might kill your task due to such violations.

13 Feb 2011, 04:59

DOS386

Joined: 08 Dec 2006
Posts: 1900

DOS386 13 Feb 2011, 05:18

revolution wrote:

DOS386 wrote:
Code:
  ; We are in PM now, but is it proper 16-bit PM or V86 ???
  ...
  mov ds, ax
  ...    
Brute-force from 0 to $FFFF and poke them all into DS
This is not a solution. The OS might kill your task due to such violations.

Solved!!! Outsource the brute-force attack on DS into a separate process. If your OS assassinates it, then we got proper PM (as in V86 there are no GPF's then).

13 Feb 2011, 05:18

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 13 Feb 2011, 05:26

The code can't assume there is an OS, let alone if there is an OS that supports processes, so you can't go starting other processes. Besides, if you already knew enough about the OS to be able to start processes then you don't need this code at all.

13 Feb 2011, 05:26

DOS386

Joined: 08 Dec 2006
Posts: 1900

DOS386 13 Feb 2011, 05:50

revolution wrote:

then you don't need this code at all.

Right, see subject:

Quote:

Detecting Real mode

13 Feb 2011, 05:50

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 13 Feb 2011, 09:41

test bp,0ff00h ;286 --> bp=ffxxh, 386+ --> bp=00xxh

i dont get it.

13 Feb 2011, 09:41

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 13 Feb 2011, 09:49

Read about SIDT in the Intel manual.

13 Feb 2011, 09:49

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 13 Feb 2011, 11:26

Quote:

The Intel 286 processor fills these bits with 1s; the Pentium 4,
Intel Xeon, P6 processor family, Pentium, Intel486, and Intel386 processors fill these
bits with 0s.

why would 286 put 0xff in there? just tell me, WHY ?!!

13 Feb 2011, 11:26

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 13 Feb 2011, 12:12

b1528932 wrote:

why would 286 put 0xff in there? just tell me, WHY ?!!

Why you always gotta know why? If you really need to know then ask Intel.

13 Feb 2011, 12:12

Goto page 1, 2 Next

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum