64bit COFF + GoLink + MessageBox does not open on Windows

Index > Windows > 64bit COFF + GoLink + MessageBox does not open on Windows

Author

Thread

wbi

Joined: 02 Feb 2023
Posts: 2

wbi 02 Feb 2023, 10:20

Hello Board Members,

maybe someone can help, thanks in advance.

I am trying to assemble to a 64-bit COFF, then link with GoLink.exe.
Some of the tests I tried, do work but a win32 MessageBox does not open when the .exe file is run from the command line or the windows explorer or a Makefile and I cannot figure out why. It works for 32bit COFF files but not in 64 bit.

Here is the assembler code:

Code:

format MS64 COFF

;include '%fasminc%\win64ax.inc'

extrn MessageBoxA
extrn ExitProcess

section '.text' code readable executable

 public Start

 Start:
        ;mov     r9d,0
        ;lea     r8,[_caption]
        ;lea     rdx,[_message]
        ;mov     rcx,0
        ;call    MessageBoxA
        
        push    0
        push    _caption
        push    _message
        push    0
        call    MessageBoxA
        
        mov     ecx, eax
        call    ExitProcess


section '.data' data readable writeable

 _caption db 'Win64 assembly',0
 _message db 'Coffee time!',0

and here are the commands I use to assemble and link:

Code:

FASM.EXE test64_no_macros.asm build/test64_no_macros.obj
GoLink.exe -fo build/test64_no_macros.exe build/test64_no_macros.obj kernel32.dll user32.dll

Best regards
wbi

02 Feb 2023, 10:20

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20292
Location: In your JS exploiting you and your system

revolution 02 Feb 2023, 10:24

The Windows 64-bit ABI uses FASTCALL convention. The code you posted seems to be the STDCALL converted to 64-bit. But that won't work.

You need to pass the first four parameters in RCX, RDX, R8, and R9. Plus you need to stack aligned to 16. and to reserve the shadow register space before calling the API.

The code you commented out looks like the correct method, but it is still missing the stack adjustments.

I'm not sure about how golink/coff does the imports. You might need an indirect call.

02 Feb 2023, 10:24

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20292
Location: In your JS exploiting you and your system

revolution 02 Feb 2023, 10:52

This might have a better chance:

Code:

format MS64 COFF

extrn MessageBoxA qword
extrn ExitProcess qword

section '.text' code readable executable

 public Start

Start:
        push    rbp                     ; align stack
        sub     rsp, 8 * 4              ; shadow space
        xor     r9, r9
        lea     r8, [_caption]
        lea     rdx, [_message]
        xor     ecx, ecx
        call    [MessageBoxA]

        mov     ecx, eax
        call    [ExitProcess]

section '.data' data readable writeable

 _caption db 'Win64 assembly',0
 _message db 'Coffee time!',0

Does golink look for the "Start" label for the entry point? Is that the default?

02 Feb 2023, 10:52

wbi

Joined: 02 Feb 2023
Posts: 2

wbi 02 Feb 2023, 11:07

Hi revolution,

wow, that was quick and it did in fact solve my problem!
I copy pasted your code and it worked right away.

And yes, Start seems to be the default label that golink is looking for. It can be changed on the command line using the flag /entry:<label> if needed.

For anybody interested, here are the commands to compile:

Code:

FASM.EXE fastcall_64bit.asm build/fastcall_64bit.obj
GoLink.exe -fo build/fastcall_64bit.exe build/fastcall_64bit.obj kernel32.dll user32.dll

Thank you very much, revolution!

02 Feb 2023, 11:07

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum