flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > CALM implementation of STRUCT - new tricks

Goto page 1, 2, 3  Next
Author
Thread Post new topic Reply to topic
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 24 Jan 2020, 17:13
With CALM it is now possible to have some processing logic even during a definition of a macro, and this gives us completely new options for re-implementing things like STRUCT macro.

A trick that looks promising is to use labeled interceptor to tell lines that contain labeled definitions from the other ones. Here's my first experimental implementation:
Code:
include 'xcalm.inc'

define __namepool __namepool
iterate name, __a,__b,__c,__d,__e,__f,__g
        define __namepool.% name
end iterate

macro struct? name
        local   declaration, body
        define  declaration struc (instance) name

        macro ends?!
                esc end macro
                restruc ?
                match strucinstance, declaration
                        strucinstance
                                label instance: sizeof.name
                                namespace instance
                                        body
                                end namespace
                        end struc
                        virtual at 0
                                name name
                                sizeof.name := $
                        end virtual
                end match
        end macro

        calminstruction (label) ?! directive&
                local default, counter, parameter
                match , directive
                jyes plain
                arrange default,
                match directive= default, directive
                init counter
                compute counter, counter + 1
                arrange parameter, __namepool.counter
                transform parameter
                arrange directive, label directive parameter
                assemble directive
                check counter = 1
                jyes first
                arrange declaration, declaration, parameter:<default>
                exit
            first:
                arrange declaration, declaration parameter:<default>
                exit
            plain:
                assemble label
        end calminstruction

        esc macro body!
end macro    
A declaration like:
Code:
struct POINT
        x dd ?
        y dd ?
ends    
is translated into the following one:
Code:
struc (instance) POINT __a:<?>, __b:<?>
        label instance: sizeof.POINT
        namespace instance
                x dd __a
                y dd __b
        end namespace
end struc
virtual at 0
        POINT POINT
        sizeof.POINT := $
end virtual    
The names for macro parameters come from a predefined pool, but it could be made automatic with help of EVAL. As it should be obvious when looking at example generated definition above, these names must be chosen/generated in such way, that they would not conflict with anything in the structure definition.

For an actual re-implementation of STRUCT.INC I would go another route, to have more flexible treatment of arguments to structure (like the "name:value" variant, which I find more more pleasant to use), and quite possibly the structure itself might end up defined as CALM instruction, with lines kept in some kind of list instead of a simple macro body.

But I think that this simple implementation is already a quite nice demonstration how one kind of declaration can be flexibly converted into another. The generated code simulates what one could write to implement a structure manually. And because it simply translates one syntax to another, it is quite transparent and does not hinder use of any special commands inside the definition - it works nicely with things like the basic implementation of UNION, with any implementation of ALIGN, etc. etc.
Post 24 Jan 2020, 17:13
View user's profile Send private message Visit poster's website Reply with quote
Roman



Joined: 21 Apr 2012
Posts: 1823
Roman 24 Jan 2020, 17:59
CALM did posible change auto size struct in 64 bit or 32 bit ?

For example:
struct A
hwnd dq 0 ;for 64 bit
posX dd 0
posY dd 0
ends

struct A
hwnd dd 0 ;for 32 bit
posX dd 0
posY dd 0
ends

Let say da mean auto size.

struct A
hwnd da 0 ;for any bits 64 or 32
posX dd 0
posY dd 0
ends
Post 24 Jan 2020, 17:59
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 24 Jan 2020, 19:34
Roman wrote:
CALM did posible change auto size struct in 64 bit or 32 bit ?
You could do that even with plain macro implementation:
Code:
include 'cpu/x64.inc'
use64

include 'struct.inc'

struc da? values&
        if x86.mode = 64
                . dq values
        else
                . dd values
        end if
end struc

struct A
        hwnd da 0
        posX dd 0
        posY dd 0
ends


try A    
Post 24 Jan 2020, 19:34
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 24 Jan 2020, 20:32
And here comes a CALM variant that defines actual structure also as CALM instruction, and uses the syntax of "name:value" pairs in the arguments:
Code:
include 'xcalm.inc'

macro struct? name
        local   counter, pname
        counter = 0
        define pname name

        virtual at 0
        namespace name

        macro ends?!
                purge ?
                restruc ?
                sizeof.name := $
                __lines := counter
                end namespace
                label name: sizeof.name at 0
                end virtual

                calminstruction (instance) name values&
                        local tmp, val, field

                        arrange tmp, =label instance : =sizeof.pname
                        assemble tmp
                        arrange tmp, =namespace instance
                        assemble tmp

                        compute counter,1

                        match , values
                        jyes process_statement
                    process_argument:
                        match field=, values, values, <>
                        jyes got_argument
                        take field, values
                      got_argument:
                        arrange val,
                        match field:val?, field
                        arrange tmp, pname.=__default__#field
                        transform tmp
                        jyes argument_ok
                        stringify field
                        asm err `name,' has no field named ',field
                        jump next_argument
                      argument_ok:
                        arrange tmp, =__init__#field
                        publish tmp, val
                      next_argument:
                        take values, values
                        jyes process_argument

                    process_statement:
                        arrange field, pname.=__label#counter
                        transform field
                        jno plain_statement
                        arrange tmp, pname.=__definition#counter
                        transform tmp
                        arrange val, =__init__#field
                        transform val
                        jyes got_value
                        arrange val, pname.=__default__#field
                        transform val
                        jno statement_ready
                      got_value:
                        arrange tmp, tmp val
                        jump statement_ready
                      plain_statement:
                        arrange tmp, pname.=__statement#counter
                        transform tmp
                      statement_ready:
                        assemble tmp
                        compute counter, counter + 1
                        check   counter < name.__lines
                        jyes    process_statement
                        asm end namespace
                end calminstruction
        end macro

        calminstruction (label) ?! definition&
                local sym, default
                match , definition
                jyes plain
                arrange default,
                match definition= default, definition
                arrange sym, pname.=__label#counter
                publish sym:, label
                arrange definition, #label definition
                arrange sym, pname.=__definition#counter
                publish sym:, definition
                arrange sym, pname.=__default__#label
                publish sym:, default
                arrange definition, definition default
                assemble definition
                exit
            plain:
                assemble label
        end calminstruction

        calminstruction ?! statement&
                local sym
                compute counter, counter+1
                arrange sym, pname.=__statement#counter
                publish sym:, statement
                assemble statement
        end calminstruction
end macro    
Code:
; usage example:

struct RECT
  left   dd ?
  top    dd ?
  right  dd ?
  bottom dd ?
ends

area RECT left:0,top:0,right:320,bottom:200    
It has one advantage over the classic macro implementation: it does not use STORE to replace default values with ones given in arguments, but it just substitutes the arguments to the directives used in the definition. This allows it to work with all kinds of definitions, including relocatable values and various customized macros. On the other hand, it still requires much work before it can offer all the features of existing macro sets.

I used indexed constants instead of value stacks so that definitions can be forward-referenced. This is mostly just a gimmick, though.
Post 24 Jan 2020, 20:32
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 25 Jan 2020, 13:20
With a little tweaks to the above implementation I got a variant that also handles legacy syntax (with values given to consecutive fields):
Code:
include 'xcalm.inc'

macro struct? name
        local   i, j, pname
        i = 0
        j = 0
        define pname name

        macro ends?!
                purge ?
                restruc ?
                name.__lines := i
                virtual at 0
                        namespace name
                                name.__body
                        end namespace
                        sizeof.name := $
                        label name: sizeof.name at 0
                end virtual

                calminstruction (instance) name values&
                        local tmp, val, field

                        arrange tmp, =label instance : =sizeof.pname
                        assemble tmp
                        arrange tmp, =namespace instance
                        assemble tmp

                        compute i, 1
                        compute j, 0

                        match , values
                        jyes default_body
                    process_argument:
                        match field=, values, values, <>
                        jyes got_argument
                        take field, values
                      got_argument:
                        match field:val?, field
                        jyes labeled_argument
                        check j < 0
                        jyes mixed_arguments
                        compute j, j + 1
                        arrange val, field
                        arrange field, pname.=__argument#j
                        transform field
                        jyes got_argument_field
                        asm err 'excess arguments'
                        jump process_statement
                      mixed_arguments:
                        asm err 'unsupported mixing of labeled and ordered values'
                        jump process_statement
                      labeled_argument:
                        check j > 0
                        jyes mixed_arguments
                        compute j, -1
                      got_argument_field:
                        arrange tmp, pname.=__default__#field
                        transform tmp
                        jyes argument_ok
                        stringify field
                        asm err `name,' has no field named ',field
                        jump next_argument
                      argument_ok:
                        arrange tmp, =__init__#field
                        publish tmp, val
                      next_argument:
                        take values, values
                        jyes process_argument

                    process_statement:
                        check i < name.__lines
                        jno finish
                        arrange field, pname.=__label#i
                        transform field
                        jno plain_statement
                        arrange tmp, pname.=__definition#i
                        transform tmp
                        arrange val, =__init__#field
                        transform val
                        jyes got_value
                        arrange val, pname.=__default__#field
                        transform val
                        jno statement_ready
                      got_value:
                        arrange tmp, tmp val
                        jump statement_ready
                      plain_statement:
                        arrange tmp, pname.=__statement#i
                        transform tmp
                      statement_ready:
                        assemble tmp
                        compute i, i + 1
                        jump    process_statement
                    default_body:
                        asm name.__body
                    finish:
                        asm end namespace
                end calminstruction

                calminstruction name values&
                        local ic
                        init ic
                        compute ic, ic + 1
                        arrange values, pname#ic pname values
                        assemble values
                end calminstruction
        end macro

        calminstruction (label) ?! definition&
                local sym, default
                match , definition
                jyes plain
                arrange default,
                match definition= default, definition
                arrange sym, pname.=__label#i
                publish sym:, label
                arrange definition, label definition
                arrange sym, pname.=__definition#i
                publish sym:, definition
                arrange sym, pname.=__default__#label
                publish sym:, default
                arrange definition, definition default
                assemble definition
                compute j, j + 1
                arrange sym, pname.=__argument#j
                publish sym:, label
                exit
            plain:
                assemble label
        end calminstruction

        calminstruction ?! statement&
                local sym
                check i
                jno collected
                match =ends?, statement
                jyes close
                arrange sym, pname.=__statement#i
                publish sym:, statement
            collected:
                assemble statement
                compute i, i + 1
                exit
            close:
                asm end macro
                assemble statement
        end calminstruction
        esc macro name.__body
end macro    
It does not yet handle sub-structures (for the purpose of argument grouping), although I believe it should also be a relatively easy addition. It would then be not that far from becoming a replacement for compatibility STRUCT.INC.
Post 25 Jan 2020, 13:20
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 26 Jan 2020, 21:45
Another update, with several tweaks and added sub-structure handling (no UNION yet, though).
Code:
include 'xcalm.inc'

macro struct? name
        local  pname, i, j, a
        define pname name
        i = 0
        j = 0
        define a name.__argument

        macro ends?!
                purge ?
                restruc ?
                name.__lines := i
                virtual at 0
                        namespace name
                                name.__body
                        end namespace
                        sizeof.name := $
                        label name: sizeof.name at 0
                end virtual

                calminstruction (instance) name values&
                        local tmp, val, field, last

                        arrange tmp, =label instance : =sizeof.pname
                        assemble tmp
                        arrange tmp, =namespace instance
                        assemble tmp

                        compute i, 1
                        compute j, 0
                        arrange a, pname.=__argument

                        match , values
                        jyes default_body
                    process_argument:
                        match field=, values, values, <>
                        jyes got_argument
                        arrange field, values
                        arrange values,
                      got_argument:
                        match field:val?, field
                        jyes labeled_argument
                        check j < 0
                        jyes mixed_arguments
                        compute j, j + 1
                        arrange val, field
                        match <val>, val
                        arrange field, a#j
                        transform field
                        jno excess_arguments
                        match <tmp>, field
                        jno got_argument_field
                        take a, tmp
                        compute tmp, 0
                        take j, tmp
                        take values, val
                        jump process_argument
                      excess_arguments:
                        asm err 'excess arguments'
                        jump process_statement
                      mixed_arguments:
                        asm err 'unsupported mixing of labeled and ordered values'
                        jump process_statement
                      labeled_argument:
                        check j > 0
                        jyes mixed_arguments
                        compute j, -1
                        match <val>, val
                      got_argument_field:
                        arrange tmp, pname.=__default__#field
                        transform tmp
                        jyes argument_ok
                        stringify field
                        asm err `name,' has no field named ',field
                        jump next_argument
                      argument_ok:
                        arrange tmp, =__init__#field
                        publish tmp, val
                      next_argument:
                        match , values
                        jno process_argument
                        take , values
                        take , j
                        take , a
                        take values, values
                        jyes next_argument

                    process_statement:
                        check i < name.__lines
                        jno finish
                        arrange field, pname.=__label#i
                        transform field
                        jno plain_statement
                        arrange tmp, pname.=__definition#i
                        transform tmp
                        arrange val, =__init__#field
                        transform val
                        jyes got_value
                        arrange val, pname.=__default__#field
                        transform val
                        jno statement_ready
                      got_value:
                        arrange tmp, tmp val
                        jump statement_ready
                      plain_statement:
                        arrange tmp, pname.=__statement#i
                        transform tmp
                      statement_ready:
                        assemble tmp
                        compute i, i + 1
                        jump    process_statement
                    default_body:
                        asm name.__body
                    finish:
                        asm end namespace
                end calminstruction

                calminstruction name values&
                        local ic
                        init ic
                        compute ic, ic + 1
                        arrange values, pname#ic pname values
                        assemble values
                end calminstruction
        end macro

        calminstruction (label) ?! definition&
                local sym, default
                match , definition
                jyes plain
                arrange default,
                match definition= default, definition
                arrange definition, label definition
                arrange sym, pname.=__definition#i
                publish sym:, definition
                match ?label, label
                arrange sym, pname.=__label#i
                publish sym:, label
                arrange sym, pname.=__default__#label
                publish sym:, default
                arrange definition, definition default
                assemble definition
                compute j, j + 1
                arrange sym, a#j
                publish sym:, label
                exit
            plain:
                assemble label
        end calminstruction

        calminstruction ?! statement&
                local sym
                check i
                jno collected
                match =struct?, statement
                jyes open
                match =ends?, statement
                jyes close
                arrange sym, pname.=__statement#i
                publish sym:, statement
            collected:
                assemble statement
                compute i, i + 1
                exit
            open:
                local tmp
                compute j, j + 1
                arrange sym, a#j
                arrange tmp, a#j#=_
                take a, tmp
                arrange tmp, <a>
                publish sym:, tmp
                compute tmp, 0
                take j, tmp
                exit
            close:
                take , a
                take , j
                take a, a
                jyes done
                asm end macro
                assemble statement
            done:
        end calminstruction
        esc macro name.__body
end macro    
This is getting close to a nice fasm-compatibility STRUCT package.
Post 26 Jan 2020, 21:45
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 27 Jan 2020, 17:06
Almost there, here comes a version with UNION support:
Code:
include 'xcalm.inc'

macro struct? name
        local  pname, i, j, a, u
        define pname name
        i = 0
        j = 0
        define a name.__argument
        u = 0

        macro ends?!
                purge ?
                restruc ?
                virtual at 0
                        namespace name
                                name.__body
                        end namespace
                        sizeof.name := $
                        label name: sizeof.name at 0
                end virtual

                calminstruction (instance) name values&
                        local tmp, val, field

                        arrange tmp, =label instance : =sizeof.pname
                        assemble tmp
                        arrange tmp, =namespace instance
                        assemble tmp

                        compute i, 1
                        compute j, 0
                        arrange a, pname.=__argument

                        match , values
                        jyes default_body
                    process_argument:
                        match field=, values, values, <>
                        jyes got_argument
                        arrange field, values
                        arrange values,
                      got_argument:
                        match field:val?, field
                        jyes labeled_argument
                        check j < 0
                        jyes mixed_arguments
                        compute j, j + 1
                        arrange val, field
                        match <val>, val
                        arrange field, a#j
                        transform field
                        jno excess_arguments
                        match <tmp>, field
                        jno got_argument_field
                        take a, tmp
                        compute tmp, 0
                        take j, tmp
                        take values, val
                        jump process_argument
                      excess_arguments:
                        asm err 'excess arguments'
                        jump process_statement
                      mixed_arguments:
                        asm err 'unsupported mixing of labeled and ordered values'
                        jump process_statement
                      labeled_argument:
                        check j > 0
                        jyes mixed_arguments
                        compute j, -1
                        match <val>, val
                      got_argument_field:
                        arrange tmp, pname.=__default__#field
                        transform tmp
                        jyes argument_ok
                        stringify field
                        asm err `name,' has no field named ',field
                        jump next_argument
                      argument_ok:
                        arrange tmp, =__init__#field
                        publish tmp, val
                      next_argument:
                        match , values
                        jno process_argument
                        take , values
                        take , j
                        take , a
                        take values, values
                        jyes next_argument

                    process_statement:
                        arrange tmp, pname.=__statement#i
                        transform tmp
                        jno finish
                        arrange field, pname.=__label#i
                        transform field
                        jno statement_ready
                        arrange tmp, pname.=__definition#i
                        transform tmp
                        arrange val, =__init__#field
                        transform val
                        jyes got_value
                        arrange val, pname.=__default__#field
                        transform val
                        jno statement_ready
                      got_value:
                        arrange tmp, tmp val
                        jump statement_ready
                      statement_ready:
                        assemble tmp
                        compute i, i + 1
                        jump    process_statement
                    default_body:
                        asm name.__body
                    finish:
                        asm end namespace
                end calminstruction

                calminstruction name values&
                        local ic
                        init ic
                        compute ic, ic + 1
                        arrange values, pname#ic pname values
                        assemble values
                end calminstruction
        end macro

        calminstruction (label) ?! definition&
                local sym, default, tmp
                match , definition
                jyes plain
                match :tmp?, definition
                jyes ignored
                match ==tmp?, definition
                jyes ignored
                arrange default,
                match definition= default, definition
                arrange definition, label definition
                arrange sym, pname.=__definition#i
                publish sym:, definition
                match ?label, label
                arrange sym, pname.=__label#i
                publish sym:, label
                arrange sym, pname.=__default__#label
                publish sym:, default
                arrange definition, definition default
                assemble definition
                match , a
                jyes done
                compute j, j + 1
                arrange sym, a#j
                publish sym:, label
            done:
                exit
            ignored:
                arrange label, label definition
            plain:
                assemble label
        end calminstruction

        calminstruction ?! statement&
                local sym, tmp
                check i
                jyes in_body
                assemble statement
                compute i, 1
                exit
            in_body:
                match =ends?, statement
                jyes close
                check u
                jno process_definition
                check u > 1
                jno union_divider_ok
                arrange tmp, =__union_divider
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                arrange a,
            union_divider_ok:
                compute u, u + 1
            process_definition:
                match =struct?, statement
                jyes open_struct
                match =union?, statement
                jyes open_union
                arrange sym, pname.=__statement#i
                publish sym:, statement
                assemble statement
                compute i, i + 1
                exit
            open_union:
                arrange tmp, =__union_start
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                arrange tmp, a
                take a, tmp
                compute tmp, 1
                take u, tmp
                exit
            open_struct:
                match , a
                jyes unlisted_substruct
                compute j, j + 1
                arrange sym, a#j
                arrange tmp, a#j#=_
                take a, tmp
                arrange tmp, <a>
                publish sym:, tmp
                jump begin_substruct
            unlisted_substruct:
                arrange tmp,
                take a, tmp
            begin_substruct:
                compute tmp, 0
                take j, tmp
                compute tmp, 0
                take u, tmp
                exit
            close:
                check u
                jno close_struct
                arrange tmp, =__union_end
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                take , a
                take , u
                exit
            close_struct:
                take , a
                take , j
                take , u
                take a, a
                jyes done
                asm end macro
                assemble statement
            done:
        end calminstruction
        esc macro name.__body
end macro

macro __union_start
        local union
        union:
        union.i = 0
        union.size = 0
        union.initialized = 0
        macro __union_open
                union.i = union.i + 1
                if (defined union.init & union.i <> union.init) | (~ defined union.init & union.i > 1)
                        virtual at union
                end if
        end macro
        macro __union_close
                if $@ > union
                        if union.i > 1
                                union.init := union.i
                        end if
                        if union.initialized
                                err 'conflicting initialization of union'
                        else
                                union.initialized = union.i
                        end if
                end if
                if $ - union > union.size
                        union.size = $ - union
                end if
                if (defined union.init & union.i <> union.init) | (~ defined union.init & union.i > 1)
                        end virtual
                end if
        end macro
        macro __union_divider
                __union_close
                __union_open
        end macro
        macro __union_end
                __union_close
                if $ - union < union.size
                        rb union.size - ($ - union)
                end if
                purge __union_open,__union_close,__union_divider,__union_end
        end macro
        __union_open
end macro    
The union sub-package is in form of macros for now, but they can be converted to CALM very easily and perhaps I'm going to do it for the final package (which is going to replace current compatibility STRUCT.INC).

What remains to do is an option of making a structure that extends another one - another easy addition, but I am showing these intermediate stages because they may be easier to follow (and see how the framework can be extended with more abilities). Nevertheless, this seems like the most maintainable version of STRUCT that I have done - even though the final macro is going to be quite long, it has pretty straightforward internals, not a tricky mess of interwoven macros that the other implementations have been. Its only downside is that definition (not instantiation) of structure is quite heavy and therefore may be slower than previous one. I may work on improving the performance later, after I have all features ready.

I think I'm also going to add support for "full path" structure fields initialization, which is not needed for fasm 1 compatibility, but would bring additional compatibility with previous STORE-based fasmg's implementations. This requires splitting field paths and passing list of "field:value" pairs to the appropriate substructure - because this implementation wants to have all initialization performed by passing the arguments to field-defining instruction (instead of patching them with later STOREs). This is also why I had to completely rewrite the UNION for this purpose.
Post 27 Jan 2020, 17:06
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 28 Jan 2020, 11:12
All right, to complete the picture of the gradual evolution, here comes a version with full features:
Code:
include 'xcalm.inc'

define struct? struct?

namespace struct?

        calminstruction instantiate: instance*, sname*, values&

                local tmp
                arrange tmp, =label instance : =sizeof.sname
                assemble tmp
                arrange tmp, =namespace instance
                assemble tmp

                match , values
                jyes default_body

                arrange tmp, =struct?.=initialize sname, values
                assemble tmp

                local i
                compute i, 1

                local field, val
            process_statement:
                arrange field, sname.=__label#i
                transform field
                jyes labeled_statement
                arrange tmp, sname.=__statement#i
                transform tmp
                jyes statement_ready
                jump finish

            labeled_statement:
                arrange tmp, sname.=__definition#i
                transform tmp
                arrange val, =__init__#field
                transform val
                jyes got_value
                arrange val, sname.=__default__#field
                transform val
                jno statement_ready
            got_value:
                arrange tmp, tmp val

            statement_ready:

                local stack
                take stack, i
                take stack, sname
                assemble tmp
                take sname, stack
                take i, stack

                compute i, i + 1
                jump    process_statement

            default_body:
                arrange tmp, sname.=__body
                assemble tmp

            finish:
                asm end namespace
        end calminstruction

        calminstruction initialize sname*, values&

                local j, a
                compute j, 0
                arrange a, sname.=__argument

                local tmp, field, val, sub
            process_argument:
                match field=, values, values, <>
                jyes got_argument
                arrange field, values
                arrange values,
            got_argument:
                match field:val?, field, <>
                jyes labeled_argument
                check j < 0
                jyes mixed_arguments
                compute j, j + 1
                arrange val, field
                match <val>, val
                arrange field, a#j
                transform field
                jno excess_arguments
                match <tmp>, field
                jno got_argument_field
                take a, tmp
                compute tmp, 0
                take j, tmp
                take values, val
                jump process_argument

            excess_arguments:
                asm err 'excess arguments'
                jump arguments_processed
            mixed_arguments:
                asm err 'unsupported mixing of labeled and ordered values'
                jump arguments_processed

            labeled_argument:
                check j > 0
                jyes mixed_arguments
                compute j, -1
                match <val>, val
            got_argument_field:
                arrange tmp, sname.=__default__#field
                transform tmp
                jyes argument_ok
                match field.sub, field
                jno unknown_field
                arrange tmp, sname.=__default__#field
                transform tmp
                jno unknown_field
                arrange val, sub:val
                arrange tmp, =__multi__#field
                transform tmp
                jno append_value
                arrange val, tmp, val
                arrange tmp, =__multi__#field
            append_value:
                publish tmp, val
                jump argument_ok
            unknown_field:
                stringify field
                arrange tmp, sname
                stringify tmp
                asm err tmp,' has no field named ',field
                jump next_argument

            argument_ok:
                arrange tmp, =__init__#field
                publish tmp, val
            next_argument:
                match , values
                jno process_argument
            arguments_processed:
                take , values
                take , j
                take , a
                take values, values
                jyes next_argument

        end calminstruction

end namespace

macro struct? declaration*

        local pname
        define pname declaration

        macro ends?!
                purge ?
                restruc ?
                purge ends?

                match name, pname
                        virtual at 0
                                namespace name
                                        name.__body
                                end namespace
                                sizeof.name := $
                                label name: sizeof.name at 0
                        end virtual

                        calminstruction (instance) name values&
                                arrange values, =struct?.=instantiate instance, pname, values
                                assemble values
                        end calminstruction

                        calminstruction name values&
                                local ic
                                init ic
                                compute ic, ic + 1
                                arrange values, =struct?.=instantiate pname#ic, pname, values
                                assemble values
                        end calminstruction
                end match
        end macro

        local i, j, a, u
        i = 0
        define j
        define u
        define a

        calminstruction (label) ?! definition&
                local sym, default, tmp
                match , definition
                jyes plain
                match :tmp?, definition
                jyes ignored
                match ==tmp?, definition
                jyes ignored
                arrange default,
                match definition= default, definition
                arrange definition, label definition
                arrange sym, pname.=__definition#i
                publish sym:, definition
                match ?label, label
                arrange sym, pname.=__label#i
                publish sym:, label
                arrange sym, pname.=__default__#label
                publish sym:, default
                arrange definition, definition default
                assemble definition
                match , a
                jyes done
                compute j, j + 1
                arrange sym, a#j
                publish sym:, label
            done:
                exit
            ignored:
                arrange label, label definition
            plain:
                assemble label
        end calminstruction

        calminstruction ?! statement&
                local proto, sym, tmp
                check i
                jyes in_body

                compute u, 0
                compute i, 1
                compute j, 1
                match pname= proto, pname
                arrange statement, =macro pname.=__body
                assemble statement
                jno prototype_copied
            use_prototype:
                arrange a,
                arrange tmp, proto.=__statement#i
                transform tmp
                jno prototype_arguments
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                jump use_prototype
            prototype_arguments:
                arrange tmp, proto.=__argument#j
                transform tmp
                jno prototype_copied
                arrange sym, pname.=__argument#j
                publish sym:, tmp
                compute j, j + 1
                jump    prototype_arguments
            prototype_copied:
                compute j, j - 1
                arrange a, pname.=__argument
                exit

            in_body:
                match =ends?, statement
                jyes close
                check u
                jno process_definition
                check u > 1
                jno union_divider_ok
                arrange tmp, =__union_divider
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                arrange a,
            union_divider_ok:
                compute u, u + 1
            process_definition:
                match =struct?, statement
                jyes open_struct
                match =union?, statement
                jyes open_union
                arrange sym, pname.=__statement#i
                publish sym:, statement
                assemble statement
                compute i, i + 1
                exit
            open_union:
                arrange tmp, =__union_start
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                arrange tmp, a
                take a, tmp
                compute tmp, 1
                take u, tmp
                exit
            open_struct:
                match , a
                jyes unlisted_substruct
                compute j, j + 1
                arrange sym, a#j
                arrange tmp, a#j#=_
                take a, tmp
                arrange tmp, <a>
                publish sym:, tmp
                jump begin_substruct
            unlisted_substruct:
                arrange tmp,
                take a, tmp
            begin_substruct:
                compute tmp, 0
                take j, tmp
                compute tmp, 0
                take u, tmp
                exit
            close:
                check u
                jno close_struct
                arrange tmp, =__union_end
                arrange sym, pname.=__statement#i
                publish sym:, tmp
                assemble tmp
                compute i, i + 1
                take , a
                take , u
                exit
            close_struct:
                take , a
                take , j
                take , u
                take a, a
                jyes done
                asm end macro
                assemble statement
            done:
        end calminstruction
        esc ; dummy line to launch the body definition (with i = 0)
end macro

macro __union_start
        local union
        union:
        union.i = 0
        union.size = 0
        union.initialized = 0
        macro __union_open
                union.i = union.i + 1
                if (defined union.init & union.i <> union.init) | (~ defined union.init & union.i > 1)
                        virtual at union
                end if
        end macro
        macro __union_close
                if $@ > union
                        if union.i > 1
                                union.init := union.i
                        end if
                        if union.initialized
                                err 'conflicting initialization of union'
                        else
                                union.initialized = union.i
                        end if
                end if
                if $ - union > union.size
                        union.size = $ - union
                end if
                if (defined union.init & union.i <> union.init) | (~ defined union.init & union.i > 1)
                        end virtual
                end if
        end macro
        macro __union_divider
                __union_close
                __union_open
        end macro
        macro __union_end
                __union_close
                if $ - union < union.size
                        rb union.size - ($ - union)
                end if
                purge __union_open,__union_close,__union_divider,__union_end
        end macro
        __union_open
end macro    
I'm going to try some further rearrangements (already partly prepared in this version) to improve the performance and then I'm most likely going to publish the final version if the official repository, replacing the struct.inc in fasm-compatibility headers.
Post 28 Jan 2020, 11:12
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 28 Jan 2020, 16:13
The final version is now in the official struct.inc file. I have used the new MVMACRO/MVSTRUC commands to improve performance, without them it would be necessary to use proxy interceptors (calling the "collect" macros).

Also, I had to add another compatibility feature that I previously forgot about - the final macro now verifies that the field has a correct length when it is initialized with a specified value (this is needed for definitions like "cFileName TCHAR MAX_PATH dup (?)" to work correctly when initialized with string of any other length).

I believe this settles it for the compatibility STRUCT package. Next station should be INVOKE macros. Smile
Post 28 Jan 2020, 16:13
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4060
Location: vpcmpistri
bitRAKE 30 Mar 2021, 05:29
A slight modification to audit structures for amd64 assembly - it will display both tail alignment and member alignment discrepancies:
Code:
virtual at 0
        struct?.octets = 0
        struc(member) dq? line&
                struct?.octets = 1 ; struct also needs alignment
                if $ and 7
                        display 10,'member "',`member,'" not 0 mod 8 (',`name,')'
                end if
                member dq line
        end struc

        namespace name
                name.__body
        end namespace

        restruc dq?

        sizeof.name := $
        label name: sizeof.name at 0
        if 7 and sizeof name
        if struct?.octets
                display 10,'structure "',`name,'" not 0 mod 8'
        end if
        end if
end virtual    
...replace the virtual block of macro struct? declaration* - where the offsets and sizeof are defined.

Probably saving me hours of debugging. Could be expanded to audit other alignments (for SSE/AVX/etc.), and RQ as well as macro DQ/RQ use. I imagine the general case would set a bit for object sizes of each power of two (to determine structure size alignment).

I've changed the syntax to "end struct/union", too. (more fasmg-ish) Very Happy

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup
Post 30 Mar 2021, 05:29
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 30 Mar 2021, 11:43
Good idea! A quick and easy option to implement something similar would be to add this code in the "ends" macro (after the "end virtual"):
Code:
                while 1
                        match label, name.__label#%
                                if defined name.label
                                        if sizeof name.label > 0 & name.label mod (1 shl (bsf sizeof name.label)) > 0
                                                display 'warning: ',`name,'.',`label,' not aligned to its natural boundary',10
                                        end if
                                else
                                        break
                                end if
                        end match
                end while     
Post 30 Mar 2021, 11:43
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4060
Location: vpcmpistri
bitRAKE 30 Mar 2021, 12:16
In terms of designing new interfaces that is certainly the better option, but it misses one case and gives false positives for the interfaces I'm working with (certainly dependent on the alignment rules).
Code:
struct VkMemoryRequirements
        size dq ?
        alignment dq ?
        memoryTypeBits dd ?,? ; pad to *8
ends    
If there is a qword size value within the structure then the whole structure needs to be a multiple of eight.

Code:
struct VkMemoryType
        propertyFlags dd ?
        heapIndex dd ?
ends
struct VkMemoryHeap
        size dq ?
        flags dd ?,?
ends

struct VkPhysicalDeviceMemoryProperties
        memoryTypeCount dd ?
        memoryTypes VkMemoryType
        repeat VK_MAX_MEMORY_TYPES-1
                memoryTypes#% VkMemoryType
        end repeat
        memoryHeapCount dd ?
        memoryHeaps VkMemoryHeap
        repeat VK_MAX_MEMORY_HEAPS-1
                memoryHeaps#% VkMemoryHeap
        end repeat
ends    
Although VkMemoryType is 8 bytes, the dwords within it don't need to be on an 8 byte boundary. Conversely, VkMemoryHeap contains a qword and needs to start on a qword boundary.

I think this is the alignment rules used by all the compilers gcc/clang/msvc; or at least the option used on windows. The people that designed these alignment rules were idiots.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup


Last edited by bitRAKE on 30 Mar 2021, 12:29; edited 1 time in total
Post 30 Mar 2021, 12:16
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 30 Mar 2021, 12:27
bitRAKE wrote:
Although VkMemoryType is 8 bytes, the dwords within it don't need to be on an 8 byte boundary. Conversely, VkMemoryHeap contains a qword and needs to start on a qword boundary.
Yeah, I did not really consider sub-structures when preparing that quick example. For a field that is itself a structure I should have used an additional constant:
Code:
                while 1
                        match label, name.__label#%
                                if defined name.label
                                        local alignment
                                        alignment = 1
                                        if sizeof name.label > 0
                                                alignment = 1 shl (bsf sizeof name.label)
                                        end if
                                        match a= type, name.__definition#%
                                                if defined type.__alignment
                                                        alignment = type.__alignment
                                                end if
                                        end match
                                        if name.label mod alignment > 0
                                                display 'warning: ',`name,'.',`label,' not aligned to its natural boundary',10
                                        end if
                                        if % = 1 ; just a simplified POC
                                                name.__alignment := alignment
                                        end if
                                else
                                        break
                                end if
                        end match
                end while    
This is still not complete, as it just copies the alignment of the first field. The proper solution should consider data from all the iterations of the WHILE loop.
Post 30 Mar 2021, 12:27
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 30 Mar 2021, 12:48
BTW, perhaps I should refactor the base macro to not include label in the __definition constant, that would make extraction of field types more reliable:
Code:
@@ -33,7 +33,7 @@ namespace struct?
                jump finish

            labeled_statement:
-               arrange tmp, sname.=__definition#i
+               arrange tmp, sname.=__label#i sname.=__definition#i
                transform tmp
                arrange val, =__init__#field
                transform val
@@ -183,7 +183,6 @@ namespace struct?
                jyes ignored
                arrange default,
                match definition= default, definition
-               arrange definition, label definition
                arrange sym, pname.=__definition#i
                publish sym:, definition
                match ?label, label
@@ -191,7 +190,7 @@ namespace struct?
                publish sym:, label
                arrange sym, pname.=__default__#label
                publish sym:, default
-               arrange definition, definition default
+               arrange definition, label definition default
                assemble definition
                arrange definition, =__size__#label == =$ - label
                assemble definition    
And then you can get the type simply like this:
Code:
                                        match type, name.__definition#%    
Post 30 Mar 2021, 12:48
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 30 Mar 2021, 13:32
Oh, and one more thing: I also forgot that __label#% may not be present for all lines, so the WHILE loop should only break when there is no __statement#% present. This is easier with CALM, so I used a little helper instruction:
Code:
calminstruction breakifndef? sym
        transform sym
        jyes ok
        asm break
    ok:
end calminstruction    
And the correct loop goes like:
Code:
                local alignment, maxalignment
                maxalignment = 1
                while 1
                        breakifndef name.__statement#%
                        match label, name.__label#%
                                if defined name.label
                                        local alignment
                                        alignment = 1
                                        if sizeof name.label > 0
                                                alignment = 1 shl (bsf sizeof name.label)
                                        end if
                                        ; match a= type, name.__definition#%
                                        match type, name.__definition#% ; adjust for the change in the base macro
                                                if defined type.__alignment
                                                        alignment = type.__alignment
                                                end if
                                        end match
                                        if name.label mod alignment > 0
                                                display 'warning: ',`name,'.',`label,' not aligned to its natural boundary',10
                                        else if alignment > maxalignment
                                                maxalignment = alignment
                                        end if
                                end if
                        end match
                end while
                name.__alignment := maxalignment    


Last edited by Tomasz Grysztar on 30 Mar 2021, 21:29; edited 1 time in total
Post 30 Mar 2021, 13:32
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 802
Location: Russian Federation, Sochi
ProMiNick 30 Mar 2021, 20:07
Tomasz Grysztar wrote:
Good idea! A quick and easy option to implement something similar would be to add this code in the "ends" macro (after the "end virtual"):
Code:
                while 1
                        match label, name.__label#%
                                if defined name.label
                                        if sizeof name.label > 0 & name.label mod (1 shl (bsf sizeof name.label)) > 0
                                                display 'warning: ',`name,'.',`label,' not aligned to its natural boundary',10
                                        end if
                                else
                                        break
                                end if
                        end match
                end while     

Could be this by default, but not as single behavior.
I called this types of structures - structures with flag forcedalignment switched on. But for internal use and space economy structs with this flag switched off are usable too.
Interesting more type of structs:
members with negate offset to base of struct (just defined offsets not affect struct size)
structs where struct base name pointed not to start of struct but somewhere else before struct end.
top level unions with substract abilities - just for convinience.

for me struct is macroset of highest priority. It could be used with any CPU architecture macros & moreover even for producing data files, so struct is a god of macros.
I can`t say how many resources would be finaly economied in case of CALMing structs but I glad too see CALMing of them.

_________________
I don`t like to refer by "you" to one person.
My soul requires acronim "thou" instead.
Post 30 Mar 2021, 20:07
View user's profile Send private message Send e-mail Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4060
Location: vpcmpistri
bitRAKE 31 Mar 2021, 10:14
Maybe split the struct from (base + indexing) constants. I tend to associate that with the implementation rather than the definition of the struct. No less important, and the same argument could be made for alignment being implementation.

A building is a good analogy: 20 stories, but five are below ground and need different support due to the added forces. The definition of the structure is the organization of the floors. The implementation of the building/floors is based on external factors.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup
Post 31 Mar 2021, 10:14
View user's profile Send private message Visit poster's website Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 84
fabbel 31 Mar 2023, 16:27
hi (again)
what is the correct syntax to initialize sub-struct in fasmg ?

assume i have :
Code:
struct T_INNER
        dwVal1          dd ?
        dwVal2          dd ?
ends

struct T_OUTER
        Inner1          T_INNER
        Inner2          T_INNER
        dwDummy1        dd ?
        dwDummy2        dd ?
        dwDummy3        dd ?

ends
    


... I mean , I see I can create new T_INNER like this e.g. :

[code]
Inner T_INNER dwVal1:0x0, dwVal2:0xFFFFFFFF
[code]

but what about T_OUTER
.. how to specify / initialize Inner1 & Inner 2 members ??
....unsure about correct syntax ?




[/code]
Post 31 Mar 2023, 16:27
View user's profile Send private message Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4060
Location: vpcmpistri
bitRAKE 31 Mar 2023, 16:34
Code:
my_to T_OUTER \
        Inner1.dwVal2: 5,\
        Inner2.dwVal1: 7    

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup
Post 31 Mar 2023, 16:34
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 31 Mar 2023, 16:58
Assuming you use the implementation in the fasm compatibility package, it supports the classic scheme compatible with how it worked with fasm's original macro (and as documented there):
Code:
example T_OUTER <11h,12h>,<21h,22h>,1,2,3    
At the same time it also supports the new handy syntax:
Code:
example T_OUTER Inner2: <dwVal1:21h,dwVal2:22h>, dwDummy2: 2     
And you can also mix and match the two schemes:
Code:
example T_OUTER Inner2: <21h,22h>, dwDummy2: 2    
(Whatever you put in <>, it becomes exact text of arguments passed to T_INNER macro.)
Post 31 Mar 2023, 16:58
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2, 3  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.