flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > [fasmg] (Almost) C style fastcall macro (on the way!)

Author
Thread Post new topic Reply to topic
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 19 May 2025, 10:05
It's been 13 days (and couting) since I started with fasmg, and, thanks to the prompt help from Tomasz itself and members like, bitRAKE, dosmancer and macomics (many thanks to you guys), I speed up the learning process a lot, and now have a feasible solution to achieve my personal most complex macro: one that emulates C style calling format. I'm opening this topic to post things and questions related to its internals development, so I can centralize everything here, and not create a flood of many topics related to many individual specifics. It's actually alpha stage, but it's already working, being a proof on how powerful and vast fasmg language really is. Below is a sample code for you to have an idea on what is already doing (don't take it too seriously, it's just tests I'm doing while achieving progress on macros' codes):

Code:
format ELF64 executable 3
entry Start

use AMD64, CET_IBT, RDRAND

include 'fastcall(dev).inc'
include 'import64.inc'

proto fputs, qword, qword
proto fprintf, qword, qword, qword, dword
proto fprintf2, qword, qword, double, double, double
proto fprintf3, qword, qword, qword, qword, qword, qword, \
        qword, qword, qword, qword, qword, qword, qword, qword, qword
proto fprintf4, qword, qword, double, double, double, double, \
        double, double, double, double, double, double, double, double, double
proto getpid, none
noreturn proto exit, dword
proto errno, none

namespace __imp
        include 'import64.inc'
        interpreter '/lib64/ld-linux-x86-64.so.2'
        needed 'libc.so.6'
        import fputs, exit, stdout, getpid, fprintf, \
                __errno_location
end namespace

        stdout  equ __imp.stdout

_rdata
        msg1            db 'This is a fastcall program!', 10, 0
        msgfmt          db 'Process ''%s'' running at PID: %u', 10, 0
        msgfl           db 'Three floats: %lf and %f and %f', 10, 0
        float1          dd 123.456
        float2          dq 567.890123

_code
        proto.exit:     jmp     [__imp.exit]
        proto.errno:    jmp     [__imp.__errno_location]
        proto.fputs:    jmp     [__imp.fputs]
        proto.getpid:   jmp     [__imp.getpid]
        proto.fprintf:  jmp     [__imp.fprintf]
        proto.fprintf2: jmp     [__imp.fprintf]
        proto.fprintf3: jmp     [__imp.fprintf]
        proto.fprintf4: jmp     [__imp.fprintf]
        
        Start:          endbr64
                        mov     r10, [stdout]
                        mov     r15, r10
                        mov     rbx, [r10]
                        getpid();
                        fprintf(rbx, &msgfmt, *rsp+8, eax);
                        fputs(&msg1, **stdout);
                @@      rdrand  rdx
                        pause
                        jnc     @b
                @@      rdrand  rcx
                        pause
                        jnc     @b
                        fprintf(*r15, <"Random value at rdx: 0x%016lX; rcx: 0x%016lX",10,0>,.,.);
                        fprintf(*r15, <"Exit address: 0x%016lX",10,0>, [__imp.exit],.);
                        fputs("These are ... ", rbx);
                        fputs(<10,"...2 anonymous strings!",10,0,0FFh>, rbx);
                        mov      eax, 123.456
                        movd     xmm13, eax
                        cvtss2sd xmm12, xmm13
                        fprintf2(**stdout, &msgfl, 3.141592, xmm12, 0.0);
                        fprintf4(rbx, <'Sequence: %f %f %f %f %f %f %f %f %f %f %f %f %f',10,0>, \
                                *float2, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0);
                        errno();
                        exit([rax]);
                        
    


It's basically what we had in Masm (proto for definition of types and sizes, and invoke for calling), but without the invoke word; just 'function(parameters)' and that's it!.

Once I reach something usable, I'll post entire code here, so, you guys can also use it if you want, and even modify it to your preferences, give suggestions, improve it, etc.. This version will be System V ABI, but it is even easier to get the concept and adapt it to Windows fastcall, for example, because it is simplier (I needed 3 counters for System V ABI, but 1 counter is enough for Windows Fastcall). This is more an idea than a final concept, and, since I'm not a C programmer myself, it could have some minor mistakes on C related concepts. I know for sure this is not for everyone, but I also know that many will like it, as much as I do.

Its features include:

- passing registers, data by value and reference, numbers, equations, even strings directly as parameters;
- optimized parameter passing based on type;
- automatic stack allocation/release and ensurance of alignment (since it's already aligned before macro use);
- vararg support with best guess method based on simple hints and/or parameter types;
- call/jmp direct/[indirect] external/internal function prototypes.

Initial concepts (subject to change):

&data = effective address of data -> lea creg, [data]
*data or [data] = data as value -> mov creg, [data]
**data = data extracted from pointer -> mov creg, [data] \ mov creg, [creg]
"string simple" = create anonymous string data, appends a 0 termination, and passes its pointer at creg
<10,10,"string complex",10,0> = create anonymous string as provided and passes pointer at creg
((12 * 8 ) / CONSTANTx) = pass this as number to creg -> mov creg, (...)
3.1415926535 = float point data will be classified as XMM/Memory class and passed accordingly, unless if prototype differs
+345 or -345 or 1717 = signed/unsigned numbers have different handling
. = parameter is skipped

I know for sure that, by discussing it here, it can become even better and, perhaps, more efficient.

_________________
jesse6


Last edited by Jessé on 19 May 2025, 10:28; edited 1 time in total
Post 19 May 2025, 10:05
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 19 May 2025, 10:23
My TODO list on it:

- create vararg support; (in progress)
- create an effective way of diferentiate: labels, CONSTANTS and numbers, so I can;

mov creg, [label]
mov creg, CONSTANT
mov creg, number
Don't crash when probing a register parameter

- make it detect import method and create entries when ext proto is used. (postponed)


Last edited by Jessé on 19 May 2025, 20:28; edited 1 time in total
Post 19 May 2025, 10:23
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 19 May 2025, 10:34
First question about it: I have the following working method as register detection for parameters:

Code:
macro isReg dest
creg_sz =: 0    ; Global register identifier variable
iterate current, rax, rbx, rcx, rdx, rdi, rsi, rbp, rsp, r8, r9, r10, r11, r12, r13, r14, r15
        match =current, dest
                creg_sz = 8
                break
        end match
end iterate
iterate current, eax, ebx, ecx, edx, edi, esi, ebp, esp, r8d, r9d, r10d, r11d, r12d, r13d, r14d, r15d
        match =current, dest
                creg_sz = 4
                break
        end match
end iterate
iterate current, ax, bx, cx, dx, di, si, bp, sp, r8w, r9w, r10w, r11w, r12w, r13w, r14w, r15w
        match =current, dest
                creg_sz = 2
                break
        end match
end iterate
iterate current, ah, al, bh, bl, ch, cl, dh, dl, dil, sil, bpl, spl, r8b, r9b, r10b, r11b, r12b, r13b, r14b, r15b
        match =current, dest
                creg_sz = 1
                break
        end match
end iterate
iterate reg, xmm0,xmm1,xmm2,xmm3,xmm4,xmm5,xmm6,xmm7,xmm8,xmm9,xmm10,xmm11,xmm12,xmm13,xmm14,xmm15
        match =reg, dest
                creg_sz = 16
                break
        end match
end iterate
iterate reg, mm0,mm1,mm2,mm3,mm4,mm5,mm6,mm7
        match =reg, dest
                creg_sz = 9   ; MMX register type
                break
        end match
end iterate
iterate reg, st0,st1,st2,st3,st4,st5,st6,st7
        match =reg, dest
                creg_sz = 10  ; FPU register type
                break
        end match
end iterate
iterate reg, ymm0,ymm1,ymm2,ymm3,ymm4,ymm5,ymm6,ymm7,ymm8,ymm9,ymm10,ymm11,ymm12,ymm13,ymm14,ymm15
        match =reg, dest
                creg_sz = 32
                break
        end match
end iterate
iterate reg, zmm0,zmm1,zmm2,zmm3,zmm4,zmm5,zmm6,zmm7,zmm8,zmm9,zmm10,zmm11,zmm12,zmm13,zmm14,zmm15
        match =reg, dest
                creg_sz = 64
                break
        end match
end iterate
iterate reg, cs, ds, es, fs, gs, ss
        match =reg, dest
                creg_sz = 48    ; Segment register type
                break
        end match
end iterate
iterate reg, cr0, cr2, cr3, cr4, cr8
        match =reg, dest
                creg_sz = 1111    ; Control register type
                break
        end match
end iterate
iterate reg, dr0, dr1, dr2, dr3, dr6, dr7
        match =reg, dest
                creg_sz = 1113    ; Debug register type
                break
        end match
end iterate
end macro
    


My unconscious (i.e., intuition) is whispering to me that there is a better method to differentiate (and classify its type and size) a register from everything else.
Is there a better technique?

Edit¹: I must mention that the day I did this I didn't know for sure if fasmg x86 includes support aliases for rXb being rXl names (e.g. r9b = r9l), now I know, so, adding it to the list sooner.
Post 19 May 2025, 10:34
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8427
Location: Kraków, Poland
Tomasz Grysztar 19 May 2025, 11:11
Jessé wrote:
My unconscious (i.e., intuition) is whispering to me that there is a better method to differentiate (and classify its type and size) a register from everything else.
Is there a better technique?
First, one could take a look at how the register symbols are defined in my implementations for fasmg (like x86-2.inc used by fasm2):
Code:
element eax? : x86.r32 + 0
element ecx? : x86.r32 + 1
element edx? : x86.r32 + 2
element ebx? : x86.r32 + 3
element esp? : x86.r32 + 4
element ebp? : x86.r32 + 5
element esi? : x86.r32 + 6
element edi? : x86.r32 + 7    
and also:
Code:
element x86.r32 : x86.reg + 4    
This allows to dig through metadata to extract all the classifications:
Code:
macro identify_register name
        display `name, " is "
        match more than, name
                display "a complex expression unsupported in this example",13,10
        else
                local info, class

                info = 1 metadataof (name)
                class = 1 metadataof info

                if class relativeto x86.reg
                        display "a general-purpose register, "
                        rept 1, size: (0 scaleof class)*8, id: 0 scaleof info
                                display `size," bits, id ",`id,13,10
                        end rept
                else if info relativeto SSE.reg
                        display "an SSE register, id "
                        rept 1, id: 0 scaleof info
                                display `id,13,10
                        end rept
                else if info relativeto x86.sreg
                        display "a segment register, id "
                        rept 1, id: 0 scaleof info
                                display `id,13,10
                        end rept
                else
                        display "an unknown entity",13,10
                end if

        end match
end macro

identify_register esi
identify_register ds
identify_register xmm11    
Another option is to call the actual operand parser, which is used internally by x86 instruction handlers. My implementation of fasm-compatible fastcall uses such approach.
Code:
macro identify_operand op
        ;x86.parse_operand@src op
        SSE.parse_operand@src op
        ;AVX_512.parse_operand@src op

        if @src.type = 'reg'
                display "a general-purpose register, "
                rept 1, size: @src.size*8, id: @src.rm
                        display `size," bits, id ",`id,13,10
                end rept
        else if @src.type = 'mmreg'
                display "a vector register, "
                rept 1, size: @src.size*8, id: @src.rm
                        display `size," bits, id ",`id,13,10
                end rept
        else if @src.type = 'sreg'
                display "a segment register, id "
                rept 1, id: @src.rm
                        display `id,13,10
                end rept
        else
                display "type ",@src.type,13,10
        end if
end macro

identify_operand esi
identify_operand ds
identify_operand xmm11
identify_operand dword [ebx]    
Post 19 May 2025, 11:11
View user's profile Send private message Visit poster's website Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 19 May 2025, 12:08
Thanks Tomasz.
I have checked x86-2.inc first, also x64.inc beforehand, but, for now, element is a cloudy concept for me.
I'll test the 3 examples you have mentioned, and try to get some learning from them.
Need to mention that I didn't know that proc64.inc has a complete fastcall example. I'm gonna check it now...

Edit: so, this is what the whispers are all about... Smile
Post 19 May 2025, 12:08
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 25 May 2025, 01:18
I have revisited that post where we had talked about differentiate numbers from labels, and experiment a little, and found that org idea will be perfect for this project, because I only (and just only) need to separate labels from number/constants to achieve this:
Code:
label     db 'Test', 0
CONSTANT := 77
CONST2 = 10 ; I know this ain't constant Wink
my POINT  ; using struc POINT .x, .y from manual here
    ....
      ; macro                 ; leads to:
    func(label)     ; --> mov rdi, [label]; call func
    func(CONSTANT)  ; --> mov edi, CONSTANT; call func
    func(CONST2)    ; --> mov edi, CONST2; call func
    func(33334)     ; --> mov edi, 33334; call func
    func(my.x)      ; --> mov edi, [my.x]; call func  ; This one fails in every test I did
    


But, when I try what auxiliary doc says, it just messed up a lot of things under elfexe.inc:

Code:
format ELF64 executable 3
entry Start

element CODEBASE
org CODEBASE + 0

segment readable executable

Start:          jmp     $

    


Quote:
❯ fasm2 -e 20 error_element.fasm2
flat assembler version g.kp60
error_element.fasm2 [7]:
segment readable executable
macro segment? [3] macro segment? [5]:
if (DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0) & SEGMENT_TYPE = PT_LOAD & OVERLAY_HEADERS
Processed: if (DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0) & SEGMENT_TYPE = PT_LOAD & OVERLAY_HEADERS
Error: values not comparable.
error_element.fasm2 [7]:
segment readable executable
macro segment? [3] macro segment? [20]:
store SEGMENT_SIZE at ELF:p_memsz+SEGMENT_INDEX*SEGMENT_HEADER_LENGTH
Processed: store SEGMENT_SIZE at ELF:p_memsz+SEGMENT_INDEX*SEGMENT_HEADER_LENGTH
Error: variable term used where not expected.
error_element.fasm2 [7]:
segment readable executable
macro segment? [3] macro segment? [22]:
if DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0
Processed: if DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0
Error: values not comparable.
error_element.fasm2 [7]:
segment readable executable
macro segment? [3] macro segment? [56]:
if DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0
Processed: if DEFINED_SEGMENT | DEFINED_SEGMENT_SIZE > 0
Error: values not comparable.
[exit=2]


Is there something that I missed or got it wrong? As I already said before, element is still a "cloudy idea" for me (but now I understand it much better than last post), so, I don't know how to properly fix this when using ELF format.
And, by using this method, I still have the mentioned problem when referencing structured labels, that it was never been detect as a label (even when using the calminstruction version to test).
So, for now, I'm still stucked with my '*label' or '[label]' idea to provide a hint to the macro that this is a data location, not a immediate number. Old fashioned, but safe (for now).

By the way, I finished the macro's vararg handler part: it is now amazing! And it is already working as expected, in a very optimized (to the code it generates, not the assembler so far). It's now alpha2, but when I finish the size override part it will be beta1, then I'll post it here. Almost there! Cool
Post 25 May 2025, 01:18
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8427
Location: Kraków, Poland
Tomasz Grysztar 25 May 2025, 07:21
Jessé wrote:
Is there something that I missed or got it wrong? As I already said before, element is still a "cloudy idea" for me (but now I understand it much better than last post), so, I don't know how to properly fix this when using ELF format.
When the ORG is already set up and maintained by some other macros, like in the case of FORMAT macros, then obviously you cannot just change it in the middle of their work without breaking them. If instead of "format ELF64 executable" you were using "format ELF64" then the ORG would already not be an absolute value, as in the case of all relocatable formats, where the information for relocation tables is generated on the fly when instructions access any movable labels.

"format ELF64 executable" only needs the addresses of segment boundaries and entry point, to set up things properly in the headers. But this means that if you change ORG, you need to change it back and/or convert all the address values every time formatter macros need them:
Code:
format ELF64 executable 3

element CODEBASE
org CODEBASE + $                ; preserve $ value, it is already non-zero here

macro segment? any
        org     $ - CODEBASE    ; restore $ before SEGMENT macro is called
        segment any
        org     CODEBASE + $    ; SEGMENT likely moved the $ value, use the updated one
end macro

macro entry? point*
        entry   point - CODEBASE
end macro

postpone                        ; the formatter had its own POSTPONE blocks, but they are executed in reverse order
        org     $ - CODEBASE    ; restore $ at the end of code, before the formatter does its things
        purge   segment?        ; also remove our wrapper macro, this time it would interfere
end postpone


entry Start

segment readable executable

Start:          jmp     Start    
Obviously, it would be easier to work with a formatter that was already designed/ modified to have the labels in a form that you need. For example with "format ELF64" sections already have non-absolute bases. The ELF executable formatter is relatively simple, you could perhaps just modify it to suit your needs.

PS. As you seem to prefer using LEA instead of MOV for addresses, you likely don't want any instructions that put absolute address values into their opcodes. It worth noting, though, that it also would be broken by a non-constant ORG would. In case you need it, know that my x86 encoders were designed to allow overriding DWORD/QWORD instructions to handle these issues:
Code:
calminstruction qword? any
        check   any relativeto CODEBASE
        jno     ok
        compute any, any - CODEBASE
     ok:
        call    qword?, any
end calminstruction

        mov     rax, Start      ; wouldn't work without the above    
This is how the relocations are gathered by formatters that support them.
Post 25 May 2025, 07:21
View user's profile Send private message Visit poster's website Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 25 May 2025, 10:57
Very good. I got the idea, tested it and it worked!

Also, after some tests, I got the "child label" (i.e., some.thing) thing problem solved, too (using match to get the 'root label' apart to be probed):

Code:
format ELF64 executable 3

element CODEBASE
org CODEBASE + $                ; preserve $ value, it is already non-zero here

macro segment? any
        org     $ - CODEBASE    ; restore $ before SEGMENT macro is called
        segment any
        org     CODEBASE + $    ; SEGMENT likely moved the $ value, use the updated one
end macro

macro entry? point*
        entry   point - CODEBASE
end macro

postpone                        ; the formatter had its own POSTPONE blocks, but they are executed in reverse order
        org     $ - CODEBASE    ; restore $ at the end of code, before the formatter does its things
        purge   segment?        ; also remove our wrapper macro, this time it would interfere
end postpone


struc POINT
    label . : .size
    .x dd ?
    .y dd ?
    .size = $ - .
end struc

entry Start

segment readable executable

Start:          jmp     Start

CONST := 335
CONST2 = 336
label0  db 90h
CONST.b = 1111
POLY1 = 16 + CODEBASE

my POINT

macro test_label what&
    if ~ defined what
        display 27, '[38;5;208m''',`what, "' is undefined.", 27, '[0m'
    else
        match root.child, what
            if root relativeto CODEBASE
                display 27, '[38;5;14m''',`what, "' is a relative.", 27, '[0m'
            else
                display '''', `what, "' is absolute."
            end if
        else
            if what relativeto CODEBASE
                display 27, '[38;5;14m''',`what, "' is a relative.", 27, '[0m'
            else
                display '''', `what, "' is absolute."
            end if
        end match
    end if
    display 10
end macro

_b = sizeof(my)
irpv val, _b
    indx %%
    display "Size of 'my': ", `val, 10
    break
end irpv

test_label Start
test_label 555043
test_label -335
test_label CONST
test_label label0
test_label CONST2
test_label (5+CONST2+1)
test_label UNDEF
test_label my.y
test_label CONST.b
test_label _b
test_label POLY1
    


It gaves me the following output:

Quote:

─ ❯ fasm2 label_test3.fasm2
flat assembler version g.kp60
Size of 'my': 8
'Start' is a relative.
'555043' is absolute.
'-335' is absolute.
'CONST' is absolute.
'label0' is a relative.
'CONST2' is absolute.
'(5+CONST2+1)' is absolute.
'UNDEF' is undefined.
'my.y' is a relative.
'CONST.b' is absolute.
'_b' is absolute.
'POLY1' is a relative.

3 passes, 0.1 seconds, 123 bytes.


This is 100% aligned with what I need to get it working.
Treating relative as 'mov creg, [data]' and absolute as 'mov creg, DATA'.
Post 25 May 2025, 10:57
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8427
Location: Kraków, Poland
Tomasz Grysztar 25 May 2025, 13:07
Jessé wrote:
Also, after some tests, I got the "child label" (i.e., some.thing) thing problem solved, too (using match to get the 'root label' apart to be probed)
This sounds like mixing two different approaches. If you change ORG, then all the labels are defined according to altered addressing, it doesn't matter whether a label is local or not. Trying to get the "root label" would make sense if you were marking your main labels in some special way (like defining "root.label" symbol). Perhaps it's really the latter that is more appropriate for your purpose.
Post 25 May 2025, 13:07
View user's profile Send private message Visit poster's website Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 29 May 2025, 15:21
Guys? I made it! And love it! Despite its - so far - slightly heavy performance on fasmg assembler, bacause my knowledge is still very premature, of course. It has much to be refined, and probably has some bugs/failure points, but it is already doing everything I basically need! Very Happy Very Happy Very Happy
This is now my permanent visa to the fasmg/fasm2 assembler, because it's all I need - together with my already done @@ multilevel macro - to deploy whatever I think I can.
I have some many tasks to accomplish on it, but, one of this tasks will be learning how to properly use GitHub, so I don't need to flood this post with every modification I do.

P.S.: I choose to not enable the label/number distinguishing support, because that thing needs to be tested furthermore. Any news I'll post here. And any ideas are also welcome.

The macro code, the user guide and a bunch of demo files are attached.

Edit: so tiny file for basically 23 days of working on it every time I'm sleepless late at night (which happens very often). Very rewarding! Wink


Description:
Download
Filename: fastcall demo.zip
Filesize: 30.36 KB
Downloaded: 82 Time(s)


_________________
jesse6
Post 29 May 2025, 15:21
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 29 May 2025, 22:39
I forgot to include this important example code, that I did when developing its vararg support:

Code:
format ELF64 executable 3
entry Start

use AMD64, CET_IBT

include 'fastcall.inc'

ext proto printf, qword, vararg
ext proto puts, qword
ext noreturn proto exit, dword
ext proto snprintf, qword, dword, qword, vararg

library 'libc.so.6'
extern puts, exit, snprintf, printf

_data
        bigfloat                dt 7777.7774
        tempbuff                rb 256
_code
        Start:                  endbr64
                                puts("Starting vararg type tests...");
                                printf(<"First call with 1 fixed parameter.",10,0>);
                                printf(<"This one has %u %u %u %u %u %u %u %u %u %s.",10,0>, 2, 3, 4, 5, \
                                 6, 7, 8, 9, 10, "parameters");
                                printf(<"Parameter double: %.4lf",10,0>, 1010.0101);
                                printf(<"Parameter long double: %.4Lf %u %u %u %u %u %u %.4Lf",10,0>,\
                                        dt 3010.0103, 1, 2, 3, 4, 5, 6, *bigfloat);
                                snprintf(&tempbuff, 256, "Buffer contents: %u %ld.", 1234567890, -513463456245733);
                                lea     r10, [tempbuff]
                                mov     [r10+rax], word 0Ah
                                puts(&tempbuff);
                                puts("... finished.");
                                exit(0);

    


This one showcases functions that are forced to be memory class, and also the long double type (which is hard to work).
Also, it exposes my design decision to use r9 as auxiliary register, and why one should care about it when using functions with many parameters.
As always, a good debugger helps with seeing the resulting code. So: 'edb --run execfile'!

P.S.: lea and mov instructions are not needed; puts() already append new line char itself. It is there only to remind that this is still assembly code. Smile
Post 29 May 2025, 22:39
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 30 May 2025, 19:05
Today, I'm working on making this macro reliable, by adding on demand imported functions capability (which means one can create a header with all the imported functions needed and library definitions, let us say, stdio.inc or gtk3.inc or libc.inc, and these functions will only appear at your executable if they are used, allowing the creation of include files that are actually reusable. The "skeleton" for it already exists in the fastcall.inc file above, and it is that 'if used function' inside ext macro. I use the following model as the default import technique (copied from fasmg package):

fasmg 'elfsym.inc':

Code:
DT_NULL    = 0
DT_NEEDED  = 1
DT_HASH    = 4
DT_STRTAB  = 5
DT_SYMTAB  = 6
DT_RELA    = 7
DT_RELASZ  = 8
DT_RELAENT = 9
DT_STRSZ   = 10
DT_SYMENT  = 11
DT_REL     = 17
DT_RELSZ   = 18
DT_RELENT  = 19

STB_LOCAL  = 0
STB_GLOBAL = 1
STB_WEAK   = 2

STT_NOTYPE  = 0
STT_OBJECT  = 1
STT_FUNC    = 2
STT_SECTION = 3
STT_FILE    = 4

R_386_NONE     = 0
R_386_32       = 1
R_386_PC32     = 2
R_386_GOT32    = 3
R_386_PLT32    = 4
R_386_COPY     = 5
R_386_GLOB_DAT = 6
R_386_JMP_SLOT = 7
R_386_RELATIVE = 8
R_386_GOTOFF   = 9
R_386_GOTPC    = 10

R_X86_64_NONE      = 0
R_X86_64_64        = 1
R_X86_64_PC32      = 2
R_X86_64_GOT32     = 3
R_X86_64_PLT32     = 4
R_X86_64_COPY      = 5
R_X86_64_GLOB_DAT  = 6
R_X86_64_JUMP_SLOT = 7
R_X86_64_RELATIVE  = 8
R_X86_64_GOTPCREL  = 9
R_X86_64_32        = 10
R_X86_64_32S       = 11
R_X86_64_16        = 12
R_X86_64_PC16      = 13
R_X86_64_8         = 14
R_X86_64_PC8       = 15
R_X86_64_DPTMOD64  = 16
R_X86_64_DTPOFF64  = 17
R_X86_64_TPOFF64   = 18
R_X86_64_TLSGD     = 19
R_X86_64_TLSLD     = 20
R_X86_64_DTPOFF32  = 21
R_X86_64_GOTTPOFF  = 22
R_X86_64_TPOFF32   = 23
R_X86_64_PC64      = 24
R_X86_64_GOTOFF64  = 25
R_X86_64_GOTPC32   = 26

macro Elf32_Sym name:0,value:0,size:0,bind:0,type:0,other:0,shndx:0
        dd name
        dd value
        dd size
        db bind shl 4 + type
        db other
        dw shndx
end macro

virtual at 0
        Elf32_Sym
        sizeof.Elf32_Sym = $
end virtual

macro Elf32_Rel offset:0,symbol:0,type:0
        dd offset
        dd symbol shl 8 + type
end macro

virtual at 0
        Elf32_Rel
        sizeof.Elf32_Rel = $
end virtual

macro Elf32_Rela offset:0,symbol:0,type:0,addend:0
        dd offset
        dd symbol shl 8 + type
        dd addend
end macro

virtual at 0
        Elf32_Rela
        sizeof.Elf32_Rela = $
end virtual

macro Elf64_Sym name:0,value:0,size:0,bind:0,type:0,other:0,shndx:0
        dd name
        db bind shl 4 + type
        db other
        dw shndx
        dq value
        dq size
end macro

virtual at 0
        Elf64_Sym
        sizeof.Elf64_Sym = $
end virtual

macro Elf64_Rel offset:0,symbol:0,type:0
        dq offset
        dq symbol shl 32 + type
end macro

virtual at 0
        Elf64_Rel
        sizeof.Elf64_Rel = $
end virtual

macro Elf64_Rela offset:0,symbol:0,type:0,addend:0
        dq offset
        dq symbol shl 32 + type
        dq addend
end macro

virtual at 0
        Elf64_Rela
        sizeof.Elf64_Rela = $
end virtual
    


fasmg 'import64.inc':

Code:
include 'elfsym.inc'

        if ELF.TYPE = ET_DYN
                element DYNAMIC
        else
                DYNAMIC := 0
        end if

macro interpreter library
        segment interpreter readable
        db library,0
end macro

macro needed libraries&
        irp library, libraries
                define needed@dynamic library
        end irp
end macro

macro import definitions&
        local strtab,strsz,symtab,rel,relsz,hash
        segment dynamic readable
        irpv library, needed@dynamic
                dq DT_NEEDED,strtab.needed#%-strtab
        end irpv
        dq DT_STRTAB,strtab
        dq DT_STRSZ,strsz
        dq DT_SYMTAB,symtab
        dq DT_SYMENT,sizeof.Elf64_Sym
        dq DT_RELA,rela
        dq DT_RELASZ,relasz
        dq DT_RELAENT,sizeof.Elf64_Rela
        dq DT_HASH,hash
        dq DT_NULL,0
        segment readable writeable
        symtab: Elf64_Sym
        local count
        count = 0
        irp name, definitions
                Elf64_Sym strtab.name-strtab,0,0,STB_GLOBAL,STT_FUNC,0,0
                count = count+1
        end irp
        rela:
        irp name, definitions
                Elf64_Rela name,%,R_X86_64_64
        end irp
        relasz = $-rela
        hash:
        dd 1,count+1
        dd 0
        repeat count
                dd %
        end repeat
        dd 0
        strtab db 0
        irp name, definitions
                strtab.name db `name,0
        end irp
        irpv library, needed@dynamic
                strtab.needed#% db library,0
        end irpv
        strsz = $-strtab
        ptrtab:
        irp name, definitions
                ?name dq 0
        end irp
end macro
    


This attempt was stucked, because I've found a very weird behavior on fasm2/fasmg, which demands strange workarounds to get it working again:

Code:
format ELF64 executable 3
entry Start

use AMD64, CET_IBT

include 'import64.inc'

interpreter '/lib64/ld-linux-x86-64.so.2'
needed 'libc.so.6'

; define IMPORT_LIST exit, puts, sleep    ; (Not OK)
; import IMPORT_LIST                      ; (Not OK)
import exit, puts, sleep              ; (OK)

segment readable executable

          Start:              endbr64
                              lea       rdi, [.msg0]
                              call      [puts]
                              xor       edi, edi
                              jmp       [exit]

               .msg0:         db "This is a test program", 0
    


The above code compiles successfully as is shown, but when I exchange the '(OK)' line for the '(Not OK)' lines, I got this:

Quote:

─ ❯ fasm2 -e 100 test3.fasm2
flat assembler version g.kp60
test3.fasm2 [13]:
import IMPORT_LIST ; (Not OK)
macro import [26] macro Elf64_Rela [1] dq? [15] qword? [1] (CALM)
Error: symbol 'exit' is undefined or out of scope.
test3.fasm2 [13]:
import IMPORT_LIST ; (Not OK)
macro import [26] macro Elf64_Rela [1] dq? [15] qword? [1] (CALM)
Error: symbol 'puts' is undefined or out of scope.
test3.fasm2 [13]:
import IMPORT_LIST ; (Not OK)
macro import [26] macro Elf64_Rela [1] dq? [15] qword? [1] (CALM)
Error: symbol 'sleep' is undefined or out of scope.
test3.fasm2 [13]:
import IMPORT_LIST ; (Not OK)
macro import [48] dq? [3]
Processed: label ?IMPORT_LIST : 8
Error: definition of 'IMPORT_LIST' in conflict with already defined symbol.
test3.fasm2 [25]:
call [puts]
call? [2] x86.parse_jump_operand@dest [21] x86.parse_operand@dest [125] (CALM)
Error: symbol 'puts' is undefined or out of scope.
test3.fasm2 [25]:
call [puts]
call? [2] x86.parse_jump_operand@dest [21] x86.parse_operand@dest [128] (CALM)
Error: symbol 'puts' is undefined or out of scope.
test3.fasm2 [25]:
call [puts]
call? [61]
Custom error: operand size not specified.
test3.fasm2 [27]:
jmp [exit]
jmp? [2] x86.parse_jump_operand@dest [21] x86.parse_operand@dest [125] (CALM)
Error: symbol 'exit' is undefined or out of scope.
test3.fasm2 [27]:
jmp [exit]
jmp? [2] x86.parse_jump_operand@dest [21] x86.parse_operand@dest [128] (CALM)
Error: symbol 'exit' is undefined or out of scope.
test3.fasm2 [27]:
jmp [exit]
jmp? [61]
Custom error: operand size not specified.
[exit=2]


I apply the simple workaround below, and it is fixed:

Code:
        ; This was applied in the final part of 'import' macro:
        irp name, definitions
                match x, name
                        ?x dq 0
                end match
        end irp
    


But I have no idea why this solves the problem.
Because, from this point on, I tested with the same scenario an full 'irpv' version of this macro, when I only pass 'IMPORT_LIST' which is now a symbol that is redefined every time a function is declared. And it works the same, nice and clean.

Modified import macro:
Code:
macro importv definitions
        local strtab,strsz,symtab,rel,relsz,hash
        segment dynamic readable
        irpv library, needed@dynamic
                dq DT_NEEDED,strtab.needed#%-strtab
        end irpv
        dq DT_STRTAB,strtab
        dq DT_STRSZ,strsz
        dq DT_SYMTAB,symtab
        dq DT_SYMENT,sizeof.Elf64_Sym
        dq DT_RELA,rela
        dq DT_RELASZ,relasz
        dq DT_RELAENT,sizeof.Elf64_Rela
        dq DT_HASH,hash
        dq DT_NULL,0
        segment readable writeable
        symtab: Elf64_Sym
        local count
        count = 0
        irpv name, definitions
                Elf64_Sym strtab.name-strtab,0,0,STB_GLOBAL,STT_FUNC,0,0
                count = count+1
        end irpv
        rela:
        irpv name, definitions
                Elf64_Rela name,%,R_X86_64_64
        end irpv
        relasz = $-rela
        hash:
        dd 1,count+1
        dd 0
        repeat count
                dd %
        end repeat
        dd 0
        strtab db 0
        irpv name, definitions
                strtab.name db `name,0
        end irpv
        irpv library, needed@dynamic
                strtab.needed#% db library,0
        end irpv
        strsz = $-strtab
        ptrtab:
        irpv name, definitions
                match x, name
                        ?x dq 0
                end match
        end irpv
end macro
    


Then, I try to incorporate this exact 'importv' macro to be used within my fastcall scenario, then the errors are back again!

After some modifications, I need now to delete 'Elf64_Rela' macro and make its work inline:

Code:
                        irpv name, definitions
                                ; Elf64_Rela name,%,R_X86_64_64
                                dq ?name        ; This ? solves the problem.
                                dq % shl 32 + R_X86_64_64
                                dq 0
                        end irpv
    


And by doing this, I not need anymore that 'match x, name' inside the last 'irpv' section:

Code:
                        irpv name, definitions
                                ?name dq 0
                        end irpv
    


This absolutely makes no sense to me, but I don't know if it is due to my lack of experience with fasmg, or it is a bug. I've found another strangeness just before I delivered the BETA1 ready here, that forced me to use a dummy match inside any of the size overrides inside the 'put_parameter2' macro (in fastcall.inc' file) just to preserve parameter attributes. There, not doing this makes parameters with register expressions (like &eax+2, which should be detected as an effective address by testtype parameter) been interpreted as numbers!
Let me know if you guys have an explanation on this.
I've solved the issue and now, using above workaround, but I don't know how fragile it is.
As soon as I am sure of this, I will upload a BETA1.1, with the above benefits, plus the completely removal of the 'extern' keyword (now, ext proto already sets everything, including the import entry, in an "on demand basis".

_________________
jesse6
Post 30 May 2025, 19:05
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8427
Location: Kraków, Poland
Tomasz Grysztar 30 May 2025, 19:32
Jessé wrote:
This attempt was stucked, because I've found a very weird behavior on fasm2/fasmg, which demands strange workarounds to get it working again:
Code:
; define IMPORT_LIST exit, puts, sleep    ; (Not OK)
; import IMPORT_LIST                      ; (Not OK)
import exit, puts, sleep              ; (OK)    

The above code compiles successfully as is shown, but when I exchange the '(OK)' line for the '(Not OK)' lines...
The macro call does not evaluate its arguments, so "import IMPORT_LIST" calls the macro with just a single argument, being a single name. When this name is passed to some command that evaluates expressions, like DQ, then the symbolic definition of IMPORT_LIST is seen and unrolled, but that's already too late for the macro to function properly.

The manual mentions use of the MATCH-based idiom to force evaluation of symbolic variable when necessary. This would fix your issue:
Code:
define IMPORT_LIST exit, puts, sleep

match list, IMPORT_LIST
        import list ; call the macro as: import exit, puts, sleep
end match    
Another option is to preprocess with help of CALM, where TRANSFORM command evaluates any symbolic variables present in given argument:
Code:
calminstruction import statement&
        transform statement
        arrange statement, =import statement
        assemble statement
end calminstruction

define IMPORT_LIST exit, puts, sleep
import IMPORT_LIST    
Post 30 May 2025, 19:32
View user's profile Send private message Visit poster's website Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 30 May 2025, 19:45
Nice, thanks for the help Tomasz.
I'll do a remake, testing the above tips.
Good to hear from you!
Knowing that what came to me as a candidate solution is a normal technique, makes things easier.
And answer my question about its stability.

Code:
format ELF64 executable 3 at 100000000h
entry Start

use AMD64, CET_IBT

include 'fastcall(dev).inc'

library 'libc.so.6'

ext noreturn proto exit, dword          ; This goes to executable's import section
ext proto puts, qword                   ; This also (both because are used below)
ext proto sleep, dword                  ; This doesn't (unused)
ext proto snprintf, qword, dword, qword, vararg ; This also does not (unused)
; extern exit, puts, sleep, snprintf  ; <-- This isn't needed anymore

_code     ; The first _rdata, _data or _code macros now define once all needed imports
          ; prior to its main purpose, that is declaring a segment with its attributes
          ;  (weird, but it works fine)

          Start:              endbr64
                              puts("This is fastcall BETA1.1 test program!");
                              exit(0);

    


By the way, this is the actual phase.
Even cleaner now, with the above benefits.
Post 30 May 2025, 19:45
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 31 May 2025, 15:19
BETA2 is finally ready!

Details of implementation are in the changelog file. Also what changes in its usage.

I have a very complicated situation there, involving, in very rare cases, function call messing up with 'iprv' directive.
The main cause appears to be the new 'importv' macro: when I use it instead of the previous 'import' model, the rare problem appears if the dummy 'irpv' block is disabled (see 'ATTENTION' lines). But this only happens with 'gtktest.fasm2' example (included), and only if the line has only 2 parameters!
I already checked every idented statement block, and they're OK, nicely close every one. Also did BETA2 twice starting from BETA1, which does not have this problem, trying to trap the exact problem, but can't.
Any words or guidance on this issue will be appreciated. It is solved, but I think the solution is not ideal (I call it a band-aid).

The error is (when correction is disabled):
Quote:

gtktest.fasm2 [103]:
g_print(<'Opacity: %lf',10,0>, xmm0); ; <- ATTENTION: this is the source of BETA2 problems
macro g_print [19] macro fastcall_ed [2] macro fastcall [154] macro put_value [1]:
irpv curr, src
Processed: irpv curr, __type_para_
Error: definition of '' in conflict with already defined symbol.


Now, the good news!

- removed extern directive: now 'ext proto' does everything and "on demand";
- I included my stdio.inc and gtk3.inc headers as example, they're not complete, and might be some errors on sizes (might not also), but they're working as expected;
- added 'alias' macro to add beauty to ugly function names (e.g., __libc_start_main);
- everything else is the same (I guess);
- there is a text file with what is changed, and some guidance; this and the modified examples might help one getting started with it.


Description: Fastcall macro set - version BETA2 - packed with examples and 'irpv' demon aboard.
Download
Filename: fastcall BETA2 demo.zip
Filesize: 33.32 KB
Downloaded: 83 Time(s)


_________________
jesse6
Post 31 May 2025, 15:19
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 31 May 2025, 15:40
As always, I forgot to include an important example, that uses the reusable header 'stdio.inc' file instead of having all defined by hand:
Code:
format ELF64 executable 3 at 0F0000000h
entry Start

use AMD64, CET_IBT

include 'fastcall_BETA2.inc'

include 'stdio.inc'

_code
          Start:              endbr64
                              fprintf(**stderr, <"An error message.",10,0>);
                              fprintf(**stdout, <"A normal message.",10,0>);
                              usleep(2068779);
                              exit(0);
    


This example, with BETA2 version, only adds to the final executable the used external functions and external data, everything else at stdio.inc stayed 'dormant', waiting for usage, not being included if not needed.
Post 31 May 2025, 15:40
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 02 Jun 2025, 09:26
If anyone is using my include files, please correct the line at 'stdio.inc' file:

Code:
; This is wrong
ext proto sprintf, qword, vararg         ; rdi = *buffer; rsi = *format; ...

;This is right
ext proto sprintf, qword, qword, vararg         ; rdi = *buffer; rsi = *format; ...
    


'vararg' type should be placed always after all well defined parameters, and you can understand this by comparing it with the manual about sprintf(), for example. The 3rd (...) is the undefined, so, the 3rd must be the vararg type.

Also, after some massive tests with this macro, I start to figure out that might be something wrong with how I handle vararg inside fastcall macro.
So, as soon as I can, I will probably try to fix it. I'll let you know.
Post 02 Jun 2025, 09:26
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 02 Jun 2025, 22:51
I made 2 corrections, but this time no need to upload a file again: '1' is mandatory repair (related to string types), and '2' is optional, but it avoids any annoyance with memory parameters that can't get its size resolved (like the ones that has a register as part of the address formation).


1: mandatory: a problem with strings at vararg type is caused by a distraction of myself while doing the macro code. Somewhere around line 1000:

Code:
                                        mov cregb, parameter
                                end if
                        else
                                err "Invalid register parameter size.", 10
                        end if
                else if __type_id_ = FUNC_PARA_TYPE_STRING
                        if psize = qword
                                put_anon_data PUT_ANON_DATA_STRINGZ, parameter ; ***
                                lea cregq, [_anon_data_curr_]
                        else if psize = dword
                                mov cregd, parameter
                        else if psize = word
                                mov cregw, parameter
                        else if psize = byte
                                mov cregb, parameter
                        else
                                err "Invalid register parameter size.", 10
                        end if
                else if __type_id_ = FUNC_PARA_TYPE_XSTRING
                        if psize = qword
                                put_anon_data PUT_ANON_DATA_STRING, parameter ; ***
                                lea cregq, [_anon_data_curr_]
                        else
                                err "Invalid size for complex string parameter", 10
                        end if
                else if __type_id_ = FUNC_PARA_TYPE_EQUATION
                        if psize = qword  ; NOTE: 0 extend 32 bit values
                                if parameter < 4294967296
                                        mov cregd, parameter
                                else
                                        mov cregq, parameter
    


Replace '__type_para_' in lines marked with ' ; ***' with 'parameter', as shown above.
As I said it is a distraction: '__type_para_' variable is not ensured to have the last definition at vararg.



2: below is one optional that ensure, for vararg, parameters whose size is undefined (informed as 0 by fasm internals) is treated by the default size, which is 'qword'. This avoid annoying errors when the provided parameter does not have a size, so it will be passed as qword instead of failing with error. Also ensure that other specified sizes follows proportional behavior (dword size assumes dword, word size assumes word) if I forgot something, or modify anything in the future:

Code:
                        ; somewhere around line 860
                        cregd = r9d
                        cregw = r9w
                        cregb = r9b
                else
                        err "Exceeded register index.", 10
                end if
                if __type_id_ = FUNC_PARA_TYPE_MEMORY | __type_id_ = FUNC_PARA_TYPE_PTR | \
                        __type_id_ = FUNC_PARA_TYPE_LABEL
                        if psize = qword
                                if __type_size_ = 8 | __type_size_ = 0  ; <--
                                        mov cregq, [parameter]
                                else if __type_size_ = 4
                                        mov cregd, [parameter]
                                else if __type_size_ = 2
                                        movzx cregd, word [parameter]
                                else if __type_size_ = 1
                                        movzx cregd, byte [parameter]
                                else
                                        err "Parameter size not supported at register parameter.", 10
                                end if
                        else if psize = dword
                                if __type_size_ = 4 | __type_size_ = 0 ; <--
                                        mov cregd, [parameter]
                                else if __type_size_ = 2
                                        movzx cregd, word [parameter]
                                else if __type_size_ = 1
                                        movzx cregd, byte [parameter]
                                else if __type_size_ = 8
                                        mov cregd, dword [parameter]
                                else
                                        err "Parameter size not supported at register parameter.", 10
                                end if
                        else if psize = word
                                if __type_size_ = 2 | __type_size_ = 0 ; <--
                                        mov cregw, [parameter]
                                else if __type_size_ = 1
                                        movzx cregw, byte [parameter]
                                else if __type_size_ = 8
                                        mov cregw, [parameter]
                                else if __type_size_ = 4
                                        mov cregw, [parameter]
                                else
                                        err "Parameter size not supported at register parameter.", 10
                                end if
                        else if psize = byte
                                mov cregb, [parameter]
                        else
                                err "Invalid register parameter size.", 10
                        end if
                else if __type_id_ = FUNC_PARA_TYPE_NUMBER32
                        if psize = qword
                                if parameter = 0
                                        xor cregd, cregd
                                else
                                        mov cregd, parameter
                                end if
                        else if psize = dword
                                if parameter = 0
                                        xor cregd, cregd
                                else
    


Add '| __type_size_ = 0' to the lines marked with a ' ; <--' to enable size inheritance for "unsizeable" parameters.
That's it.
No docs on these, because anything else if equal BETA2 above.
Post 02 Jun 2025, 22:51
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 05 Jun 2025, 11:42
I have good news: this macro is leaving beta probably within a few days!
It's now BETA3, but I'll not make it public now, because I'm still taking decisions on optimizing the code, and testing literally everything I can to ensure proper working.
I'll open a new post with the official version 1.
Everything will be compatible with BETA2, so, no need to change any code from now (except 'include', of course). And some useful features are added, and performance has improved a lot since BETA2.
Post 05 Jun 2025, 11:42
View user's profile Send private message Reply with quote
Jessé



Joined: 03 May 2025
Posts: 53
Jessé 08 Jun 2025, 12:58
Finally stable!

Take it here: https://board.flatassembler.net/topic.php?t=23884

✌️
Post 08 Jun 2025, 12:58
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.