MS64 COFF create message box

Index > Windows > MS64 COFF create message box

Author

Thread

Ariiio

Joined: 03 Feb 2024
Posts: 5
Location: Germany

Ariiio 07 Feb 2024, 21:16

I want to create a x64 program that opens a message box and says 'hello world'. I know I could use

Code:

format PE64 GUI

but I want to use MS64 COFF since I want to later on include other libraries (raylib) and if I understood correctly I have to use COFF for this to be able to link raylib later on. What I don't get is, I'm using correct windows x64 calling convention, and everything compiles file yet whe I run the .exe nothing happens. Here are the things I do for compilation

Code:

fasm test.asm
gcc test.obj -o test

Description:		Download
Filename:	test.asm
Filesize:	322 Bytes
Downloaded:	155 Time(s)

_________________
🔨

07 Feb 2024, 21:16

MatQuasar

Joined: 25 Oct 2023
Posts: 105

MatQuasar 08 Feb 2024, 09:39

I tested OK if add this line to the beginning:

Code:

sub rsp, 40

I am not sure why it needs msys-2.0.dll because I download gcc for Windows through MSYS2.

08 Feb 2024, 09:39

Ariiio

Joined: 03 Feb 2024
Posts: 5
Location: Germany

Ariiio 08 Feb 2024, 15:13

MatQuasar wrote:

I tested OK if add this line to the beginning:
Code:
sub rsp, 40
    
I am not sure why it needs msys-2.0.dll because I download gcc for Windows through MSYS2.

could you explain why I need to reserve space on the stack before I call my function? I dont see any reason why I should do this since the function doesnt pop anything off the stack

_________________
🔨

08 Feb 2024, 15:13

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20513
Location: In your JS exploiting you and your system

revolution 08 Feb 2024, 16:05

The FASTCALL convention dictates the stack alignment. Some of the API calls use the MOVDQA instruction to access the stack. So if RSP is not aligned then the code crashes.

08 Feb 2024, 16:05

MatQuasar

Joined: 25 Oct 2023
Posts: 105

MatQuasar 08 Feb 2024, 17:02

Ariiio wrote:

MatQuasar wrote:
I tested OK if add this line to the beginning:
Code:
sub rsp, 40
    
I am not sure why it needs msys-2.0.dll because I download gcc for Windows through MSYS2.
could you explain why I need to reserve space on the stack before I call my function? I dont see any reason why I should do this since the function doesnt pop anything off the stack

I also learned from @revolution, here is his/her extra notes:

revolution wrote:

You don't need to continually use sub/add. You can do only at entry and exit as long as you reserve enough for the largest needed for any function call.

Also proper aligned is a requirement, or you risk the code crashing. You can allocate in increments of 16-bytes only. Upon each entry the stack has a return address and is unaligned, so you have to adjust by 8 more bytes to fix it, and then in multiples of 16-bytes.

Only in x64 Windows programming, in 32-bit I don't have to reserve stack space.

Extra note:

https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-170 wrote:

Since 16 bytes is a common alignment size for XMM operations, this value should work for most code.

08 Feb 2024, 17:02

Furs

Joined: 04 Mar 2016
Posts: 2595

Furs 09 Feb 2024, 15:36

Ariiio wrote:

could you explain why I need to reserve space on the stack before I call my function? I dont see any reason why I should do this since the function doesnt pop anything off the stack

Because Microsoft added a retarded "shadow space" for the first 4 args and everyone is stuck with this suboptimal shit now. At least on Windows.

64-bit ABI on Unix OSes doesn't have it and it's perfectly sane.

09 Feb 2024, 15:36

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20513
Location: In your JS exploiting you and your system

revolution 09 Feb 2024, 15:53

Furs wrote:

64-bit ABI on Unix OSes doesn't have it and it's perfectly sane.

For the OS, yes, AFAICT.

But not for the user space SYS-V libraries. Alignment is still a requirement.

09 Feb 2024, 15:53

Furs

Joined: 04 Mar 2016
Posts: 2595

Furs 10 Feb 2024, 20:10

Alignment ≠ shadow space. Alignment is meh, but can be dealt with and not nearly as bad. You can still push args for instance (just an extra dummy push if alignment needs it, not a big deal).

10 Feb 2024, 20:10

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum