flat assembler
Message board for the users of flat assembler.

Index > Windows > Funny thing about DLL code

Author
Thread Post new topic Reply to topic
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
I just experienced this, thought I'd post it.

My driver program (I mean program that uses my library) wasn't initializing correctly, so I opened it up in Olly and -:
Code:
7727A5F0   55               PUSH    EBP
7727A5F1   8BEC             MOV     EBP,ESP
7727A5F3   56               PUSH    ESI
7727A5F4   57               PUSH    EDI
7727A5F5   53               PUSH    EBX
7727A5F6   8BF4             MOV     ESI,ESP
7727A5F8   FF75 14          PUSH    DWORD PTR SS:[EBP+14]
7727A5FB   FF75 10          PUSH    DWORD PTR SS:[EBP+10]
7727A5FE   FF75 0C          PUSH    DWORD PTR SS:[EBP+C]
7727A601   FF55 08          CALL    NEAR DWORD PTR SS:[EBP+8]
7727A604   8BE6             MOV     ESP,ESI
7727A606   5B               POP     EBX
7727A607   5F               POP     EDI
7727A608   5E               POP     ESI
7727A609   5D               POP     EBP
7727A60A   C2 1000          RETN    10
    

That was Windows Vista Kernel32 code that calls the entry point in a DLL, I had never seen it before... I studied why it faulted (esp contained value 0x100), and noticed that the person who coded this apparently stored the stack register in esi, then restored it when the code returned.

So in a DLL entry, you can modify and mess with the stack and esp register as much as you want, but if you modify esi the code will fault.

Just thought it was something interesting, now I know I have to save the esi register on all entries.
Post 05 Mar 2008, 22:56
View user's profile Send private message Visit poster's website Reply with quote
asmrox



Joined: 19 Jan 2008
Posts: 160
asmrox
i always use this in dll:

entry $
pushad
code...
popad
retn 12
Post 06 Mar 2008, 00:04
View user's profile Send private message Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
? pushad? i'll check...
Post 06 Mar 2008, 00:08
View user's profile Send private message Visit poster's website Reply with quote
Goplat



Joined: 15 Sep 2006
Posts: 181
Goplat
AlexP wrote:
That was Windows Vista Kernel32 code that calls the entry point in a DLL, I had never seen it before... I studied why it faulted (esp contained value 0x100), and noticed that the person who coded this apparently stored the stack register in esi, then restored it when the code returned.
This code is to compensate for the fact that some broken DLLs return from their entry function with plain "ret" instead of the required "ret 12". Back when compiler optimization was slim to nil, this kind of bug often wouldn't actually affect anything, because the compiler just addressed everything relative to BP/EBP and didn't try any fancy stack tricks. My guess is an earlier version of Windows was compiled with such a non-optimizing compiler, and during this time the broken DLLs were made. Later, when Microsoft made their compiler better, the code broke so they put this wrapper in to keep those DLLs working.

Don't rely on this wrapper staying the same or even existing at all in the future; Microsoft doesn't care about backwards compatibility anywhere near as much as they used to (as you have surely seen if you're using Vista). To be safe, just follow the standard calling conventions.
Quote:
So in a DLL entry, you can modify and mess with the stack and esp register as much as you want, but if you modify esi the code will fault.
No. A DLL entry point follows the calling conventions like any other callback function. You can change EAX, ECX, and EDX if you want, but EBX, ESI, EDI, and EBP *all* must be preserved, and you must remove the parameters from the stack (with "ret 12").
asmrox wrote:
i always use this in dll:

entry $
pushad
code...
popad
retn 12
A DLL entry point must return a value to indicate success/failure. If it returns 0, the DLL is immediately unloaded and LoadLibrary returns NULL. With this code, you're just returning whatever was already in EAX, which could possibly be 0.
Post 06 Mar 2008, 01:23
View user's profile Send private message Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
Quote:
With this code, you're just returning whatever was already in EAX, which could possibly be 0.

So the correct procedure is to place a value other than 0 into eax before returning. Very well written, I will be sure to remember that point in the future.
Post 06 Mar 2008, 03:53
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
It's in the doc's.

SDK - DllEntryPoint wrote:
Return Values

When the system calls the DllEntryPoint function with the DLL_PROCESS_ATTACH value, the function returns TRUE if it succeeds or FALSE if initialization fails. If the return value is FALSE when DllEntryPoint
is called because the process uses the LoadLibrary function, LoadLibrary returns NULL. If the return value is FALSE when DllEntryPoint is called during process initialization, the process terminates with an error. To get extended error information, call GetLastError.

When the system calls the DllEntryPoint function with any value other than DLL_PROCESS_ATTACH, the return value is ignored.


Do you have the SDK? If not it is available online at MSDN for anyone wishing to read it.
Post 06 Mar 2008, 04:22
View user's profile Send private message Visit poster's website Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
Which SDK are you referring to? The driver SDK? Or do you mean the online documentation?
Post 06 Mar 2008, 04:28
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
Platform SDK. It is available online for free, it has also been posted for download on the flatassembler.net website.
Post 06 Mar 2008, 04:30
View user's profile Send private message Visit poster's website Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
I've been looking around microsft download, it's nowhere!!! I can't find it in FASM site.
Post 06 Mar 2008, 04:31
View user's profile Send private message Visit poster's website Reply with quote
asmrox



Joined: 19 Jan 2008
Posts: 160
asmrox
adding
xor eax,eax
inc eax will fix it?
Post 06 Mar 2008, 15:18
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
AlexP wrote:
I've been looking around microsft download, it's nowhere!!! I can't find it in FASM site.
Hmm, I have just looked and it seems to have disappeared Sad Sorry to put on the wrong track. It used to be there.

But the MSDN online version is always the most up-to-date anyway.
Post 06 Mar 2008, 15:22
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
asmrox wrote:
adding
xor eax,eax
inc eax will fix it?
Sure, also "mov eax,1" and "or eax,1" and "or eax,-1" and "xor eax,eax|dec eax" and lots of other ways.
Post 06 Mar 2008, 15:24
View user's profile Send private message Visit poster's website Reply with quote
dap



Joined: 01 Dec 2007
Posts: 61
Location: Belgium
dap
Goplat wrote:
This code is to compensate for the fact that some broken DLLs return from their entry function with plain "ret" instead of the required "ret 12".


An interesting article BTW : http://blogs.msdn.com/oldnewthing/archive/2004/01/15/58973.aspx

_________________
(French only) http://dap.developpez.com
Post 06 Mar 2008, 16:03
View user's profile Send private message Visit poster's website Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
hmm... I'll continue using the online MSDN, but I've found a new version of the SDK for "windows server 2008". I'll try it, just selected the options for only "Win32 examples" and "Win32 documentation".
Post 06 Mar 2008, 21:45
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.