flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > [sug] Yet another PROC macro suggestion for freeing up EBP

Goto page 1, 2  Next

Would you like to integrate an ESP base proc macro into fasm?
Yes please, it is a good thing
65%
 65%  [ 19 ]
Yes, but it should operate differently
6%
 6%  [ 2 ]
Perhaps, I am unsure either way
6%
 6%  [ 2 ]
No thanks, it is evil
0%
 0%  [ 0 ]
Chocolate is yummy
20%
 20%  [ 6 ]
Total Votes : 29

Author
Thread Post new topic Reply to topic
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 01 Oct 2006, 10:47
This topic has come up on this board a few times in the past but so far no real progress has been made towards any form of permanent solution.

So this is my attempt to try and effect some long term change.

I hope to show below a practical and easy solution to the following few ideas:

Idea 1) Make a set of macros for PROC/ENDP that allows one to use EBP freely
Idea 2) Make the macros simple to use and understand
Idea 3) Integrate the macros with the existing FASM package seamlessly
Idea 4) Keep the macros completely backwards compatible with existing code

The basic premis of the existing PROC macro is this:
Code:
proc function [uses reg reg ...], argument1, argument2, ...    

My proposal is to keep this existing syntax for standard (non-free-ebp) proc's and add the following syntax:
Code:
proc function esp [uses reg reg ...], argument1, argument2, ...    

Just the one change only, the addition of "esp" in the declaration. The idea here is to clearly show that the procedure is using ESP based addressing and thus leaving EBP free. I think this fulfills idea 4 above, no existing code need be changed, it will still work fine.

Okay, now you are thinking "what's the catch?". Well there are a few catches of course, most of which are only related to how the function operates by not using EBP as a base register. For the moment I would like to ignore those issues and concentrate on getting the proc's working nicely and leave until later the higher level decisions about when (or not) to actually use and ESP based procedure.

So onwards to the actual code to show how it works in the background. There are ony a few small changes that we need to do to make this syntax possible. For the following I will assume that we are only taking the existing "proc32.inc" file and editing it in a few places, currently "proc64.inc" is not considered.

Change 1: The "define@proc" macro:
Code:
...
   match =params, params \{ params equ statement
                            flag = 0 \}
   match =esp any, params\{ params equ any
                            flag=flag+100b\}
   match =esp, params\{ params equ
                            flag=flag+100b\}
   match =uses reglist=,args, params \{ regs equ reglist
                                        params equ args \}
   match =regs =uses reglist, regs params \{ regs equ reglist
                                             params equ \}
   match =regs, regs \{ regs equ \}
   current = 4
   match y,regs\{irps x,y\\{ current=current+4 \\}\}
   if flag and 100b
    virtual at esp+current+localbytes
   else
    virtual at ebp+4+current
   end if
   match =,args, params \{ defargs@proc args \}
   match =args@proc args, args@proc params \{ defargs@proc args \}
   parmbytes = $-$$
   end virtual
...
   macro locals
   \{
    if flag and 100b
      virtual at esp+current
    else
      virtual at ebp-localbytes+current
    end if
      macro label . \\{ deflocal@proc .,:, \\}
...
      restruc rb,rw,rp,rd,rt,rq
      current = current + $-$$
      end virtual \}
...    

Above I have used bit 2 of the "flags" to store the ESP state.

Change 2: The "prologuedef" macro:
Code:
macro prologuedef procname,flag,parmbytes,localbytes,reglist
 {
   irps reg, reglist \{ push reg \}
   if ~(flag and 100b) & (parmbytes | localbytes)
    push ebp
    mov ebp,esp
   end if
   if localbytes
    add esp,-localbytes
   end if
}    

Above there are actually two changes, one to support ESP based proc's and the other to aid EBP based proc's with stack management during exit.

Change 3: The "epiloguedef" macro:
Code:
macro epiloguedef procname,flag,parmbytes,localbytes,reglist
 {
   if (flag and 100b) & localbytes
    sub esp,-localbytes
   end if
   if ~(flag and 100b) & (parmbytes | localbytes)
    leave
   end if
   irps reg, reglist \{ reverse pop reg \}
   if flag and 10000b
    retn
   else
    retn parmbytes
   end if }    

Above is the reverse of prologdef and undoes the stack and returns.

You don't need to manually edit your copy of proc32.inc, because below I have attached the entire "proc32.inc" file already edited with the changes.

So one other thing I changed, as mentioned above, is the timing of the push/pop for register saving. I push before making the stack frame and pop after releasing the frame. One good reason for doing this is to allow for aborting from the proc in the case of a error without having to worry about restoring ESP, just jump to the "ret" and all preserved registers are still restored properly. Now this change cannot work for ESP based proc's so it is only useful (an extra feature) for the standard EBP based proc's.

For those of you that may not be aware of why ESP basing is not normally used, here are some reasons why you might not use it:

1) Calls to other proc's with pushing of parameters requires special handling and the proc I have shown won't solve this for you.
2) General purpose push/pop creates a nightmare with accessing local variables and caller parameters.

An example of number one and two above:
Code:
proc console_print esp,buff,length
        invoke  GetStdHandle,STD_OUTPUT_HANDLE
        push    ecx                     ;now we are in trouble!
        mov     ecx,esp
;the following invoke is VERY ugly now
        invoke  WriteFile,eax,[buff+16],[length+12],ecx,0
        pop     ecx                     ;back to normal stack again
        sub     eax,1                   ;carry=1=failed
        sbb     eax,eax
        or      eax,ecx                 ;-1=failed, other=successfully wrote eax bytes
        ret
endp    


And here are some reasons why you would want to use ESP basing:

1) The extra register makes some alogrithms really fly.
2) Leaf procedures, or simple functions, that don't require the extra overhead of EBP base setup, saves code size and execution time.

An example of number two above:
Code:
proc crc32_byte_raw esp,crc,bite,polynomial
    ;this routine is very basic and using an EBP base is unnecessary
        mov     eax,[crc]
        not     eax
        xor     al,byte[bite]
        mov     edx,8
    .a: shr     eax,1
        sbb     ecx,ecx
        and     ecx,[polynomial]
        xor     eax,ecx
        dec     edx
        jnz     .a
        not     eax
        ret
endp    


I hope that this can be integrated into the standard fasm download package for everyone to use. I have already implemented this change into my standard products for the staff at my office, everyone so far has no complaints. Do feel free to make your vote.
Post 01 Oct 2006, 10:47
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4624
Location: Argentina
LocoDelAssembly 01 Oct 2006, 13:48
I voted the first entry, however, I wish to see invoke, stdcall, and all that working without doing adjustments by my own. Do you think that it's possible or there is some limitation here that prevents automatic stack address adjustment?
Post 01 Oct 2006, 13:48
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 01 Oct 2006, 14:11
Quote:
I wish to see invoke, stdcall, and all that working without doing adjustments by my own. Do you think that it's possible or there is some limitation here that prevents automatic stack address adjustment?
I have looked into this issue before about how one might make an automatic adder for the arguments. But the issue is VERY very complex, there are so many things that can make it go wrong, and so many cases to consider. I eventually decided to not do such automatic adjustment. Another reason is that for proc's that would require an automatic adjuster it would almost always be better to use the normal EBP based proc and not have to worry about adjusting call parameters.
Post 01 Oct 2006, 14:11
View user's profile Send private message Visit poster's website Reply with quote
okasvi



Joined: 18 Aug 2005
Posts: 382
Location: Finland
okasvi 01 Oct 2006, 20:00
I'm sure Tomasz could make us proc macro using esp w/adjusting push/pop/call/invoke/stdcall etc. that would work in most cases Smile
I think the problem is that he doesnt want to add anything which wont work in all cases Neutral
Post 01 Oct 2006, 20:00
View user's profile Send private message MSN Messenger Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 01 Oct 2006, 21:52
i like it as idea, and i think tomasz already had such macros for 64 things. maybe not that complete.

but i am afraid it misses point on cases when it should be used:
Code:
1) The extra register makes some alogrithms really fly. 
2) Leaf procedures, or simple functions, that don't require the extra overhead of EBP base setup, saves code size and execution time.    
both are things that should be done by hand, and then they are done better than with macros, when it is really important
Post 01 Oct 2006, 21:52
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 01 Oct 2006, 22:47
vid wrote:
both are things that should be done by hand, and then they are done better than with macros, when it is really important
Here I have to disagree, I think this is where macros make the job much easier and provide a standard way of doing the task. Without a macro we have to do something like this:
Code:
if used crc32_byte_raw
  crc32_byte_raw:
  virtual at esp
   .ret         dd ?
   .crc         dd ?
   .bite        dd ?
   .polynomial  dd ?
  end virtual
        mov     eax,[.crc]
        not     eax
        xor     al,byte[.bite]
        mov     edx,8
    .a: shr     eax,1
        sbb     ecx,ecx
        and     ecx,[.polynomial]
        xor     eax,ecx
        dec     edx
        jnz     .a
        not     eax
        ret
end if    
I think the above is just too much visual clutter for a simple proc. The alternative, placing in an extra "esp" into the proc declaration, keeps the code neater and allows one to concentrate on the task rather than the tiny detais of every proc.
Post 01 Oct 2006, 22:47
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2465
Location: Bucharest, Romania
Borsuc 05 Oct 2006, 10:07
What's so bad about the tiny details? You can add them after you design your algorithm. However I do agree that repeating the same code in a source code tends to be better replaced by macros
Post 05 Oct 2006, 10:07
View user's profile Send private message Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 05 Oct 2006, 11:19
revolution: also it makes it too easy to make error too hard to find
Post 05 Oct 2006, 11:19
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
MazeGen



Joined: 06 Oct 2003
Posts: 977
Location: Czechoslovakia
MazeGen 05 Oct 2006, 12:16
revolution: Petroizki (from MASM board) and me already made the same thing in MASM (called "pmacros"). I attach the latest help file which I'm aware of, maybe it can help you somehow.


Description: ! Change the extension to .chm
Download
Filename: pmacros.txt
Filesize: 45.63 KB
Downloaded: 372 Time(s)

Post 05 Oct 2006, 12:16
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 05 Oct 2006, 13:09
rev: maybe you could disable esp for that part of source, and enable it later:

Code:
_internal_esp equ esp
esp equ ___undefined___symbol___

;and internally in your macros, you will use _internal_esp, user
;can't access esp directly. same for push/pop

restore _internal_esp
restore esp
    
Post 05 Oct 2006, 13:09
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 05 Oct 2006, 20:21
Quote:
maybe you could disable esp for that part of source, and enable it later:
It wasn't my intention to have such restrictions imposed. The thought was that if one uses esp to define the macro then the user has decided that the proc is simple and thus taking responsibility for any stack fixups when required. If the stack problems become difficult then it would be best to delete the esp declaration and go back to the usual ebp based proc.

I don't imagine that using the esp proc will become the default, more likely something like 10% of procs can benefit from it.

Even the current proc macro has problems when using push/pop but people either don't notice or don't understand why it is a problem.
Post 05 Oct 2006, 20:21
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 06 Oct 2006, 06:06
what problems does current proc macro have? you say that this wouldnt work??
Code:
proc a uses ebx
mov ebx, 123
push eax
ret
endp    
Post 06 Oct 2006, 06:06
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 06 Oct 2006, 11:42
vid' wrote:
what problems does current proc macro have?
There are two problems, and I am glad you posted that example because that shows another bug that I previously overlooked. But before I show what is wrong with what you posted above, first let me modify it slightly
Code:
proc b uses ebx ,val
mov ebx, 123 
push eax 
ret 
endp    
I added the parameter val, now with the current macro it assembles to this:
Code:
;00401021    55              PUSH    EBP
;00401022    89E5            MOV     EBP,ESP
;00401024    53              PUSH    EBX
;00401025    BB 7B000000     MOV     EBX,7B
;0040102A    50              PUSH    EAX
;0040102B    5B              POP     EBX 
;0040102C    C9              LEAVE
;0040102D    C2 0400         RETN    4    
I expect you can see the problem, EBX is not restored properly, it is corrupted.

Now with the macro I posted above it assembles to this:
Code:
;00401021    53              PUSH    EBX
;00401022    55              PUSH    EBP
;00401023    89E5            MOV     EBP,ESP
;00401025    BB 7B000000     MOV     EBX,7B
;0040102A    50              PUSH    EAX
;0040102B    C9              LEAVE
;0040102C    5B              POP     EBX 
;0040102D    C2 0400         RETN    4    
Now EBX is properly restored, for as long as we don't use EBP we are safe.

Okay, now back to the code you posted. It assembles to this:
Code:
;00401011    53              PUSH    EBX
;00401012    BB 7B000000     MOV     EBX,7B
;00401017    50              PUSH    EAX
;00401018    5B              POP     EBX
;00401019    C3              RETN    
This not only destroys EBX but also tries to return to the original value of EBX. Indeed both the existing macro and my altered macros above both generate the same code. This could be easily fixed by including the prologue/epilogue code for all proc's that don't have input parameters or local variables.
Post 06 Oct 2006, 11:42
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 06 Oct 2006, 11:45
it was REALLY done that way Question Exclamation Question Exclamation
i didn't even bothered to try it... wouldn't believe

eh-eh... tomasz, tomasz Wink
Post 06 Oct 2006, 11:45
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 06 Oct 2006, 11:47
btw, what was the reason of "uses ebx ecx" if this wasn't supported?
Post 06 Oct 2006, 11:47
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 06 Oct 2006, 11:54
Quote:
btw, what was the reason of "uses ebx ecx" if this wasn't supported?
It is supported, but it gets screwed up by the extra push without the pop. If not for that it all works fine.
Post 06 Oct 2006, 11:54
View user's profile Send private message Visit poster's website Reply with quote
UCM



Joined: 25 Feb 2005
Posts: 285
Location: Canada
UCM 06 Oct 2006, 22:13
Well, if it is missing a pop, then there is an error Wink
Post 06 Oct 2006, 22:13
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 07 Oct 2006, 11:25
okasvi wrote:
I'm sure Tomasz could make us proc macro using esp w/adjusting push/pop/call/invoke/stdcall etc.
Yes, I'm sure he could, I also did it, but there are too many things that can make it do the wrong thing. The issue is not about getting the macros working, it is about how to get it working properly so that the programmer knows in advance where the macro will fail. For example, consider this:
Code:
proc abc esp uses ebx, foo
 ;...
 call .subFunction1
 call .subFunction2
 ;...
 ret
.subFunction1:
 mov eax,[foo] ;what offset goes here for foo?
 ;...
 retn
.subFunction2:
 mov eax,[foo] ;what offset goes here for foo?
 ;...
 call .subFunction1
 ;...
 retn
endp    

Inside subFunction2, above, the foo offset can be predicted (+4) for the function as shown. But inside subFunction1 there is no value that can reliably be used for foo. subFunction1 is called from two places and with different stack offsets in effect. There is no solution to this problem unless we do some modifcation like inseting a dummy push before the call to subFunction1 within the main body of abc (and later a pop of course). But such a solution is incredibly easy to get wrong during the next editing session. And any macro solution would be incredibly complex and hard to get correct. And that is just a simple example!

This is why I think it is best to not do such a thing and therefore force the programmer to take responsibilty for stack usage. If the programmer does not know how to manage the stack then the easiest solution is to delete the "esp" declaration and the problem is solved.

This is why I made sure to initially post some reasons not to use esp based procedures, because they are very easy to get wrong. The addition of esp functionality is not intended to be normal usage, just where it can be useful and simple to apply.
Post 07 Oct 2006, 11:25
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 07 Oct 2006, 11:29
MazeGen wrote:
Petroizki (from MASM board) and me already made the same thing in MASM (called "pmacros"). I attach the latest help file which I'm aware of, maybe it can help you somehow.
What format is that file in? My editor displays nonsense.
Post 07 Oct 2006, 11:29
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 07 Oct 2006, 11:34
UCM wrote:
Well, if it is missing a pop, then there is an error
Perhaps you are right, but I see it more as a matter of style. There a two ways one might use that style:

1)
Code:
proc abc uses ebx, foo
 ;...
 call .f1
 ;...
.err
 ret
.f1:
 invoke something,1,2,3
 test eax,eax
 jz .err
 ;...
 retn
endp    
2)
Code:
proc abc uses ebx, foo
 ;...
 push ecx
 invoke something,1,2,3
 test eax,eax
 jz .err
 ;...
 pop ecx
 ;...
.err
 ret
endp    
Post 07 Oct 2006, 11:34
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.