flat assembler
Message board for the users of flat assembler.

flat assembler > Linux > why the ENTER instruction fails on allocating > 1416 byte

Author
Thread Post new topic Reply to topic
fly



Joined: 27 Apr 2005
Posts: 11
Location: Salzburg, Austria
i believed that the enter instruction equals to the push/mov/sub sequence, but when i try to allocate more than 1416 bytes for the stack frame the code below produces a segfault.
first i thought, my litte knowledge in assembler is the reason for that, but i tried to replace enter/leave instructions with the corresponding push/... sequences, and... it works fine!
The question is: Why?

this one will fail:
Code:
format ELF64 executable
section readable executable

entry $
        enter 1417, 0

        leave

mov rbx, 0
mov rax, 1
int 0x80    


this one will work:
Code:
format ELF64 executable
section readable executable

entry $
        push rbp
        mov rbp, rsp
        sub rsp, 1417

        mov rsp, rbp
        pop rbp

mov rbx, 0
mov rax, 1
int 0x80    
Post 06 Jun 2005, 15:18
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 16778
Location: In your JS exploiting you and your system
My guess is that RSP must be aligned by 8. 1417 does not align the stack properly. After subtracting 1417 from RSP then first instruction that uses the stack will also fail.


Last edited by revolution on 07 Jun 2005, 09:48; edited 2 times in total
Post 07 Jun 2005, 02:00
View user's profile Send private message Visit poster's website Reply with quote
fly



Joined: 27 Apr 2005
Posts: 11
Location: Salzburg, Austria
i tried a well-aligned value, but with the same result. interestingly the 32-bit counterpart allows a value below 3005 to work.

Code:
format ELF executable
section readable executable

entry $
        ;enter 3005, 0
        enter 4096, 0

        leave

mov ebx, 0
mov eax, 1
int 0x80    


edit: also 32byte alignment (e.g. 32768) will produce a segfault
Post 07 Jun 2005, 08:09
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2019, Tomasz Grysztar.

Powered by rwasa.