Using fasm with libc seems like a fantastic idea, and it certainly makes like a little bit easier.

However, ccall reverse pushd's the arguments onto the stack, and I'm unsure how this should be modified for 64 bit. I have tried using pushing qwords instead of dwords but then arg is the wrong size.

Thanks

Mat

I haven't done any 64bit stuff before but have you tried looking at the stack in OllyDbg.

I just found something interesting though:



source:

http://eli.thegreenplace.net/2011/09/06/stack-frame-layout-on-x86-64/

Here's another important read too. I recommend you start with it.
http://msdn.microsoft.com/en-us/magazine/cc301708.aspx

I didn't know the IA-64 has 128 general purpose registers, lol

Ok, so I think the conclusion is that ccall is possibly not as much a good way to do it...

I have been looking at the disassembly from simple C code to try and figure it out, and I reached a similar conclusion to you first post: that registers are used, along a similar sort of theory to fastcall.

Thanks for the links. Will probably mean I rewrite all my procedures to use different registers so that they don't have to change them...

I highly recommend this paper http://agner.org/optimize/calling_conventions.pdf
There is everything you need to know about calling convention on any compiler and any OS.

Thanks randall. Makes for some nice bed time reading

From first readings, I am particularly interested in the idea of shadow space on windows, and the red zone in linux.

I think the main problem is that I was hoping by linking to libc I'd be able to write good cross-platform asm. There seems to be many differences between windows and linux for 64 bit calls.

Of course it is perfectly possible for me to come up with my own calling convention

Other links that may help too:

http://www.openwatcom.org/index.php/Calling_Conventions

http://en.wikipedia.org/wiki/X86_calling_conventions

Here's another important read too.
http://blogs.msdn.com/b/oldnewthing/archive/2004/01/14/58579.aspx
http://blogs.msdn.com/b/ntdebugging/archive/2009/01/09/challenges-of-debugging-optimized-x64-code.aspx