flat assembler
Message board for the users of flat assembler.

Index > Windows > Calling FASM dll from C# in x86-64

Author
Thread Post new topic Reply to topic
jaytea



Joined: 03 Oct 2014
Posts: 5
jaytea 03 Oct 2014, 09:21
Hello,

Thanks for reading this and any help you offer. I have finally, after intending to for so long, taken up the task of learning assembly code. Whatever assembly code I write must be useable by .NET code because of my profession and that my colleagues wouldn't take the time to use my code otherwise.

So my first task in coding assembly was to make a function in a DLL with an export and returning a value. After getting past the issues with windows considering my DLL invalid, I found that I could easily return a value passed into a 32-bit function, but the 64-bit function has never behaved except when accepting a single parameter.

I've tried a few things and found a couple problems. In 32-bit I can accept an array of doubles and return the value at any index within the array. I can also accept two double parameters and return either. In 64-bit, attempting to access the array results in an access violation, and when I receive two parameters I can only return the first no matter what I try.

I'll quit rambling and share some code:

The 32-bit code is practically identical, and since it works I won't bother sharing it now unless someone asks.

Returning second parameter in ASM (C# receives the same return value, the first parameter, if I change qword[in2] to qworc[input], even though the parameters are unequal):

proc asdf input:qword,in2:qword
fld qword[in2]
ret
endp

Returning the first element of an array, always returns 0.0 even though it works in 32-bit. The same code works fine if I pass just a double parameter and not an array, although 32-bit will accept either.

proc return input:qword

fld qword [input+0x8]
ret
endp

Finally, this code has resulted in access violations. This is almost identical to the code generated by msvc compiler except I am setting the regular floating point register while the msvc code generated from C++ code called movsd xmm0... . I tried also using movsd xmm0 but it didn't help.
proc return input:qword
mov rcx,[input]
fld qword[rcx]
ret
endp

Any advice is very much appreciated. I have stayed up late in the night trying to figure this out, so hopefully I have still provided good details in spite of insomnia.

-John Thoits
Post 03 Oct 2014, 09:21
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20451
Location: In your JS exploiting you and your system
revolution 03 Oct 2014, 09:50
The 64-bit fastcall standard says that float parameters are returned in the XMM registers.
Post 03 Oct 2014, 09:50
View user's profile Send private message Visit poster's website Reply with quote
jaytea



Joined: 03 Oct 2014
Posts: 5
jaytea 03 Oct 2014, 09:56
Thanks revolution. I'll try again in the morning, but what I have suspected is that the FPU registers coincide with the XMM registers because I have successfully used either to return from my function the first value passed into my functiom. It could be that MS code checks which I use. In any event I'm pretty sure my return method is not the issue since I can successfully (if not correctly) use FPU registers to return floating point values. If the issue was which register I used to return then why did my code ever work? What I'm personally suspicious of is the handling of the parameters. I could be too tired to see my mistake, either way thanks for your response!
Post 03 Oct 2014, 09:56
View user's profile Send private message Reply with quote
jaytea



Joined: 03 Oct 2014
Posts: 5
jaytea 03 Oct 2014, 10:20
Revolution,

Thanks again for your input, would you care to provide a link to the source of your information? I tried to google the subject and found very little new information, so that would be appreciated.

-John Thoits
Post 03 Oct 2014, 10:20
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20451
Location: In your JS exploiting you and your system
revolution 03 Oct 2014, 11:15
The x64 Application Binary Interface (ABI) is a 4 register fast-call calling convention, with stack-backing for those registers. There is a strict one-to-one correspondence between arguments in a function, and the registers for those arguments. Any argument that doesn’t fit in 8 bytes, or is not 1, 2, 4, or 8 bytes, must be passed by reference. There is no attempt to spread a single argument across multiple registers. The x87 register stack is unused. It may be used, but must be considered volatile across function calls. All floating point operations are done using the 16 XMM registers. The arguments are passed in registers RCX, RDX, R8, and R9. If the arguments are float/double, they are passed in XMM0L, XMM1L, XMM2L, and XMM3L. 16 byte arguments are passed by reference. Parameter passing is described in detail in Parameter Passing. In addition to these registers, RAX, R10, R11, XMM4, and XMM5 are volatile. All other registers are non-volatile. Register usage is documented in detail in Register Usage and Caller/Callee Saved Registers.
Post 03 Oct 2014, 11:15
View user's profile Send private message Visit poster's website Reply with quote
jaytea



Joined: 03 Oct 2014
Posts: 5
jaytea 03 Oct 2014, 23:02
Thanks again revolution, I got it working using the information you linked! It has joined the exclusive ranks of my bookmarks.

Since the macros handle the input of a double array well in 32-bits I was expecting what worked there to also work in 64-bits, but I guess I was either wrong, did it wrong or there is some bug in the proc macro in handling the parameters. Doing it manually and using the registers is working fine.

Clearly the reason I thought that I was able to return the first parameter of a call double(double) was because that parameter was already in the return register!

My 32-bit code (called as a double(double[]) and works fine):
Code:
proc return input:qword 
        fld     qword [input]  
        ret
endp    


Didn't work when translated into 64-bit code (also called the same):
Code:
proc return input
        movsd   xmm0,[input] 
        ret
endp    


But by following the guide on the 64-bit calling convention this accomplished the same as the 32-bit code:
Code:
proc return
        movsd   xmm0,[rcx] 
        ret
endp    


I did try specifying the input as input:qword in 64 even though I'm pretty sure that doesn't mean anything different, but that didn't help. I'll just program for 64 using the more manual approach to handling paramters.
Post 03 Oct 2014, 23:02
View user's profile Send private message Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2909
Location: 0x77760000
typedef 04 Oct 2014, 02:42
If you want to return a value so bad why not just pass a memory pointer and fill that space with the return value? Make it a struct and you can return more than one value. Wink
Post 04 Oct 2014, 02:42
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland
Tomasz Grysztar 04 Oct 2014, 08:29
jaytea wrote:
Since the macros handle the input of a double array well in 32-bits I was expecting what worked there to also work in 64-bits, but I guess I was either wrong, did it wrong or there is some bug in the proc macro in handling the parameters. Doing it manually and using the registers is working fine.

You may find the explanation for this in the fasm's documentation:
'1.4 Procedures (64-bit)' wrote:
It should be noted however that in the calling convention used in 64-bit Windows first four parameters are passed in registers (RCX, RDX, R8 and R9), and therefore, even though there is a space reserved for them at the stack and it is labelled with name provided in the procedure definition, those four parameters will not initially reside there. They should be accessed by directly reading the registers.

You may also note this part in the fasm's template for 64-bit application (examples/win64/template/template.asm):
Code:
proc WindowProc uses rbx rsi rdi, hwnd,wmsg,wparam,lparam

; Note that first four parameters are passed in registers,
; while names given in the declaration of procedure refer to the stack
; space reserved for them - you may store them there to be later accessible
; if the contents of registers gets destroyed. This may look like:
;       mov     [hwnd],rcx
;       mov     [wmsg],edx
;       mov     [wparam],r8
;       mov     [lparam],r9

        cmp     edx,WM_DESTROY
        je      .wmdestroy
  .defwndproc:
        invoke  DefWindowProc,rcx,rdx,r8,r9
        jmp     .finish
  .wmdestroy:
        invoke  PostQuitMessage,0
        xor     eax,eax
  .finish:
        ret

endp    
Post 04 Oct 2014, 08:29
View user's profile Send private message Visit poster's website Reply with quote
jaytea



Joined: 03 Oct 2014
Posts: 5
jaytea 04 Oct 2014, 21:57
Thanks Tomasz, now I understand my error.
Post 04 Oct 2014, 21:57
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.