flat assembler
Message board for the users of flat assembler.

Index > Main > [fasmg] Troubles with defined/definite

Goto page Previous  1, 2, 3  Next
Author
Thread Post new topic Reply to topic
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 22 Feb 2024, 19:22
First, as quick feedback on those operators, I do find those pretty nice and useful indeed ! tx !

Some further questions / remarks :
1/ one thing that I feel is still missing is a was to check "computability" of expression in CALM:
i mean, pretty often, when doing sthg like : compute var, val,
I wish there could be and option to have flags set to track and process or log malformed expression in val
i/o fasmg simply failing with 'invalid expression' msg (which could be anywhere in the middle of calm code...)
... maybe sthg like : compute ? var, val
-> if successful, then valid computation numerical result readily available in var
-> computation success / failure testable via jyes / jno
-> "traditional"compute behavior preserved
.. this is also a way to check if val is of numerical type
... happy to get comments


2/ back to above operators, are they going to make it to "official" distribution ?
what do you have in mind there ?
do you think they could somehow be "factorized" ? talking bout 8 new operators....
i mean esp the 3 pairs __nameofequ / __nameofpriorequ, __nameofmacro / __nameofpriormacro, __nameofstruc / __nameofpriorstruc ...
... cud somehow be "fused" with defined / definite ?
... sthg like defined _name xxx / definite __name xxx, defined __nameofmacro xxx / definite __nameofmacro xxx, ...
Post 22 Feb 2024, 19:22
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 23 Feb 2024, 08:12
Yes, I have been hesitating to integrate it into the main line, because I'm not fully satisfied with the execution. While it nicely utilizes the perks of compiled expressions, it is only fun if one understands how all the underlying mechanisms work. It would not be easy to explain this in the manual in a non-tiring way, and CALM already has enough rules that require careful description. And I don't consider it a crucial feature - keep in mind that I made the entirety of fasm2 package without ever needed anything like this. Therefore, I felt I can comfortably wait until I come up with something better.

In particular, a family of condition-checking operators cannot provide all the capabilities I would want from such addition to the language. Instead of checking whether something is a valid name or a complete expression, I would prefer to have the ability to cut such name or expression off a longer sequence of tokens. Because of that I'm now considering switching to actual ALM commands - with an added value of not cluttering the global namespace, allowing to choose cleaner names. Such commands would provide more abilities than just simple checks, while allowing to be used to purpose of said checks when needed.

The experimental branch is therefore still just that, and I'm going to be testing other solutions, without any hurry.
Post 23 Feb 2024, 08:12
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 23 Feb 2024, 10:58
A side note about COMPUTE: the main purpose and strength of this command lies in its ability to evaluate precompiled expressions (everything that follows the comma at the time when CALM instruction is defined). While it can also parse and evaluate expressions embedded in variables as a tokenized text, it is a secondary function, and not especially advantageous, because the text goes through the same processing as normally for classic fasmg's commands. That's why from my point of view COMPUTE is not a good place for language additions related to parsing tokenized text of an expressions - in fact, even MATCH might be a better home. For example, the third argument of ALM MATCH could be extended so that it could state that matched segment needs to consume a complete identifier or a maximum well-formed expression. It is in fact one of the variants I've been considering, and quite promising.
Post 23 Feb 2024, 10:58
View user's profile Send private message Visit poster's website Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 18 Sep 2024, 12:08
Hi. Any update / progress there ?
cheers
Post 18 Sep 2024, 12:08
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 06 May 2025, 09:13
fabbel wrote:
Hi. Any update / progress there ?
For a long time I could not think of a syntax for the extended MATCH that I would be happy with. But I think I finally got something (not implemented yet).

Similarly to specifying pairs of bracket characters, a third argument of CALM's MATCH could choose a special character to modify wildcard names with additional options, something like:
Code:
match name/. tail, source, /    
Here "name/." would define the wildcard, with "." being the modifier (likely meaning that the wildcard must cut complete symbol identifiers). The special character would be chosen freely, so that it would not interfere with symbols required for the pattern:
Code:
match name:. tail, source, :    
Such syntax would be extendable, as it opens up possibility of introducing more modifiers for the wildcards.

Note about the syntax of the third argument: currently it must consist of two characters, defining brackets. A single-character third argument is currently invalid syntax and can therefore be adapted to mean something else. And a three-character variant would allow to specify both options at once.
Post 06 May 2025, 09:13
View user's profile Send private message Visit poster's website Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 07 May 2025, 15:04
Hi Tomasz !
thx for feedback .. was kinda thinkin this topic was in limbo... happy to see i was wrong !
this sounds promising

quick questions :
1/ i would assume this 'new syntax' would then entail multiple such 'modifiers', typ to
allow matching vs various cases - or do u see it differently now ? :
* valid identifier of a symbol
* valid identifier of a symbol with symbolic value
* valid identifier of a symbol with symbolic value defined earlier in the source
+
same for identifier of struc / macro instruction
+
potentially also for maximum well-formed expression


2/ for potential modifier relating to 'symbol with symbolic value defined earlier in the source', how would that work vs fwd-referencing / asm passes ?
I mean would that exhibit same 'side effects' as previously discussed (typ. inducing potential multiple passes even if no fwd ref, due to case sensitity - similar to irpv, as u mentionned in some other thread)
Post 07 May 2025, 15:04
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 07 May 2025, 16:03
Yes, potentially all of my previous protype operators like "__nameofpriormacro" could be re-implemented as wildcard modifiers, and greedy expression wildcard could also be possible. As for the multi-pass process, they would likely have the same side effects as their counterparts in the previous prototype.

Adding custom wildcards is tricky, though. Because making them too flexible would break some of the assumptions that allowed me to optimize MATCH, I have to put well-defined constraints on them. I need to proceed very carefully, thus the progress is slow.
Post 07 May 2025, 16:03
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 20 May 2025, 12:11
I have a working prototype in calm4 branch:
Code:
fossil open https://flatassembler.net/fossil/repo/fasmg
fossil checkout calm4    
Plugging this new framework into existing fasmg/CALM engines has been a delicate task, with many constraints requiring attention, but once I addressed the critical issues it all gained traction. Adding modifiers is now easy and fun.

The syntax is more or less what I envisioned above, using a third argument to choose a special character that introduces a modifier after the wildcard name. The third argument is either a single character to specify modifiers, or two characters to define counted brackets, these two features cannot be mixed together. All modifiers have plain, readable names:
Code:
match label:name tail, line, :    
And I'm much more happy with this language extension than with the previous one (which remains in the "laboratory" branch). The syntax is clean and matching instead of only testing is more powerful. It's going to be a great tool for fasm2, as it allows to implement some fasm-compatible features much more cleanly, and with better compatibility. Therefore I'm pretty sure this one is going to make it into the main line.
Post 20 May 2025, 12:11
View user's profile Send private message Visit poster's website Reply with quote
dosmancer



Joined: 20 Feb 2025
Posts: 38
Location: Kingdom of Sweden
dosmancer 20 May 2025, 15:22
I ran some expriments:
Code:
iterate modifier, \
    \ ; equ matches stuff with symbolic value (the symbolic value can be empty)
    equ, \
    expression, \
    macro, \
    \ ; a valid name. does not have to have been defined or anything.
    name, \
    \ ; a "raw" number token [0-9]+
    number, \
    priorequ, \
    priormacro, \
    priorstruc, \
    \ ; "raw" quoted string token
    quoted, \
    struc

    calminstruction match_#modifier? arg
            local arg_str, tmp
            arrange arg_str, arg
            stringify arg_str
            match tmp:modifier, arg, :
            jyes yes
        ; no:
            display arg_str bappend ' <> ' bappend `modifier bappend 13 bappend 10
            exit
        yes:
            display arg_str bappend ' =  ' bappend `modifier bappend 13 bappend 10
    end calminstruction
end iterate

calminstruction test arg
    call match_equ, arg
    call match_expression, arg
    call match_macro, arg
    call match_name, arg
    call match_number, arg
    call match_priorequ, arg
    call match_priormacro, arg
    call match_priorstruc, arg
    call match_quoted, arg
    call match_struc, arg
end calminstruction

define a 5
b := 5
define c
element d
define e 5b
define f
define g "test"
define h a
define i h

iterate case,  a,b,5,c,d,e,f,hello,g,'test',h,i,&
    test case
    display 13, 10
end iterate    

The output is:
Code:
calm4 fasm_macro_tests/calm4/test.asm 
flat assembler  version g.kt77
a =  equ
a =  expression
a =  macro
a =  name
a <> number
a =  priorequ
a =  priormacro
a =  priorstruc
a <> quoted
a =  struc

b <> equ
b =  expression
b =  macro
b =  name
b <> number
b <> priorequ
b =  priormacro
b =  priorstruc
b <> quoted
b =  struc

5 <> equ
5 =  expression
5 <> macro
5 <> name
5 =  number
5 <> priorequ
5 <> priormacro
5 <> priorstruc
5 <> quoted
5 <> struc

c =  equ
c <> expression
c =  macro
c =  name
c <> number
c =  priorequ
c =  priormacro
c =  priorstruc
c <> quoted
c =  struc

d <> equ
d =  expression
d =  macro
d =  name
d <> number
d <> priorequ
d =  priormacro
d =  priorstruc
d <> quoted
d =  struc

e =  equ
e =  expression
e =  macro
e =  name
e <> number
e =  priorequ
e =  priormacro
e =  priorstruc
e <> quoted
e =  struc

f =  equ
f <> expression
f =  macro
f =  name
f <> number
f =  priorequ
f =  priormacro
f =  priorstruc
f <> quoted
f =  struc

hello <> equ
hello =  expression
hello <> macro
hello =  name
hello <> number
hello <> priorequ
hello <> priormacro
hello <> priorstruc
hello <> quoted
hello <> struc

g =  equ
g =  expression
g =  macro
g =  name
g <> number
g =  priorequ
g =  priormacro
g =  priorstruc
g <> quoted
g =  struc

'test' <> equ
'test' =  expression
'test' <> macro
'test' <> name
'test' <> number
'test' <> priorequ
'test' <> priormacro
'test' <> priorstruc
'test' =  quoted
'test' <> struc

h =  equ
h =  expression
h =  macro
h =  name
h <> number
h =  priorequ
h =  priormacro
h =  priorstruc
h <> quoted
h =  struc

i =  equ
i =  expression
i =  macro
i =  name
i <> number
i =  priorequ
i =  priormacro
i =  priorstruc
i <> quoted
i =  struc

& <> equ
& <> expression
& <> macro
& <> name
& <> number
& <> priorequ
& <> priormacro
& <> priorstruc
& <> quoted
& <> struc


1 pass, 0 bytes.    

This line puzzles me. Is it correct? And I don't understand what struc and macro should match, known macros and known strucs?
Code:
hello =  expression    

_________________
fasm68k
Post 20 May 2025, 15:22
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 20 May 2025, 15:49
dosmancer wrote:
This line puzzles me. Is it correct? And I don't understand what struc and macro should match, known macros and known strucs?
I'm soon going to commit the updates to the manual, they should answer your questions. Also, "macro"/"struc" modifiers are currently buggy, the fix is in the works.
Post 20 May 2025, 15:49
View user's profile Send private message Visit poster's website Reply with quote
dosmancer



Joined: 20 Feb 2025
Posts: 38
Location: Kingdom of Sweden
dosmancer 20 May 2025, 17:03
Tomasz Grysztar wrote:
dosmancer wrote:
This line puzzles me. Is it correct? And I don't understand what struc and macro should match, known macros and known strucs?
I'm soon going to commit the updates to the manual, they should answer your questions. Also, "macro"/"struc" modifiers are currently buggy, the fix is in the works.

Yes, it does 👌
Quote:
The "expression" modifier makes the wildcard match a complete computable expression. It ensures that the expression is correctly structured, but does not attempt computing it. An additional "defined" check may determine if it also contains no undefined values

_________________
fasm68k
Post 20 May 2025, 17:03
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 21 May 2025, 09:50
The "expression" modifier allows to implement TIMES in a more fasm-compatible way, and could even allow for this terrible multi-operand PUSH instruction:
Code:
calminstruction push? args&
        local   op, size
        arrange size,
    loop:
        match   op.expression args?, args, .
        jno     other
        match   size.name, op, .
        jno     plain
        transform size, x86
        jno     plain
        match   :size, size
        jno     plain
        match   op.expression args?, args, .
        jno     other
    plain:
        arrange op, =push size op
        assemble op
        match   , args
        jno     loop
        exit
    other:
        match   [op], args
        jyes    memory
        stringify args
        err     args bappend ' is not a valid operand' bappend 0A0Dh
        exit
    memory:
        arrange op, =push size [op]
        match   , args
        jno     loop
end calminstruction    
Code:
; Assembles just like fasm 1:
; (not that this is a good idea, it's a horrible syntax)
push 1 2        ; two pushes,
push 1 +2       ; a single push.    
The "name" wildcard should also allow for a more fasm-like IRPS implemented as macro.

And, finally back to the original topic of this thread, the IFDEF/IFDEFPRIOR examples I made for the earlier prototype can be rewritten using the new framework and I even included one of them in the updated manual:
Code:
if_true equ if 1
if_false equ if 0

calminstruction ifdef? sym*
        match   sym.name, sym, .
        jno     error
        match   sym.equ, sym, .
        jyes    true
        check   defined sym
        jno     false
    true:
        assemble if_true
        exit
    error:
        err     'not a name'
    false:
        assemble if_false
end calminstruction

calminstruction ifdefprior? sym*
        match   sym.name, sym, .
        jno     error
        match   sym.priorequ, sym, .
        jyes    true
        match   sym.equ, sym, .
        jyes    false
        check   definite sym
        jno     false
    true:
        assemble if_true
        exit
    error:
        err     'not a name'
    false:
        assemble if_false
end calminstruction

ifdef a
        display 'Yes'
else
        display 'No'
end if

ifdefprior a
        display 'Yes'
else
        display 'No'
end if

a equ 8    
Post 21 May 2025, 09:50
View user's profile Send private message Visit poster's website Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 21 May 2025, 10:30
.... sry.. trying again .. too quick to submit....


... will try this as soon as i can ... but looks definitely interesting to me...
... in the meantime, some quick questions / remarks that spring to mind:

1/ are special characters '=' and '?' allowed as modifier special symbol ?
-> doesn't that mix up the recognition of litteral token / case insensitivity ?

2/ a bit 'picky' maybe.... but in the updated doc, there is the following example:
Code:
        calminstruction (output) compute? text
                match   text/expression, text, /
                jyes    inspect
                err     'malformed expression'
                exit
            inspect:
                check   defined text
                jyes    evaluate
                err     'expression with unknown quantities'
            evaluate:
                compute output, text
        end calminstruction
    


... i wonder if the last line (compute output, text), shouldn't rather be sthg like
compute text, text
publish output, text
... to make it more useful ?

3/ another picky one...
... the updated doc reads
'(...) it instead chooses the symbol that may be used to modify wildcards (...)'
... i find sthg like the below sounds clearer / neater (... u judge ofc)
'(...) it instead chooses a symbol that may be used to modify the interpretation of wildcards (...)'

4/ how far do u think is the fix for macro"/"struc" modifiers ?
Post 21 May 2025, 10:30
View user's profile Send private message Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 21 May 2025, 10:34
... regarding 1/ .... sry i meant doesn't that mix up the recognition of litteral token / token optionality ?
Post 21 May 2025, 10:34
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 21 May 2025, 10:43
fabbel wrote:
1/ are special characters '=' and '?' allowed as modifier special symbol ?
-> doesn't that mix up the recognition of litteral token / case insensitivity ?
Yes, this would interfere with other uses of the character, that's why you have a choice and why the manual recommends to select a character that is otherwise not necessary in the pattern you're working on.
But when your pattern does not need "=" for anything, you are free do this:
Code:
calminstruction test arg
        match any=name, arg, =
        stringify any
        display any
end calminstruction    


fabbel wrote:
... i wonder if the last line (compute output, text), shouldn't rather be sthg like
compute text, text
publish output, text
... to make it more useful ?
Yes, of course! I'm going to change it.

fabbel wrote:
3/ another picky one...
... the updated doc reads
'(...) it instead chooses the symbol that may be used to modify wildcards (...)'
... i find sthg like the below sounds clearer / neater (... u judge ofc)
'(...) it instead chooses a symbol that may be used to modify the interpretation of wildcards (...)'
Does it really make much of a difference? I prefer more concise phrasing, I feel like the manual is already too dense at times.

fabbel wrote:
4/ how far do u think is the fix for macro"/"struc" modifiers ?
I uploaded the fix immediately after posting yesterday. I'm already working on other fixes, look out for fresh commits.
Post 21 May 2025, 10:43
View user's profile Send private message Visit poster's website Reply with quote
fabbel



Joined: 30 Oct 2012
Posts: 103
fabbel 21 May 2025, 11:02
tx for the (quick) update !

... reg 3/ fair enough ... I know i was playing picky .. as said Wink
(just felt those few words made it less confusing to me... not actually talking abt 'modifying' but rather interpreting the wildcards)
but again... ur the boss !
Post 21 May 2025, 11:02
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 21 May 2025, 11:31
Please let me know immediately if you get any crashes. This design has been giving me headaches and I doubt that it's airtight. I should renew friendship with afl-fuzz.
Post 21 May 2025, 11:31
View user's profile Send private message Visit poster's website Reply with quote
dosmancer



Joined: 20 Feb 2025
Posts: 38
Location: Kingdom of Sweden
dosmancer 21 May 2025, 19:02
No crashes but I noted that `name`:s can contain dots:
Code:
calminstruction test2
        local tmp
        arrange tmp, =d6.=w
        match tmp:name, tmp, :
        jyes yes
    ; no:
        display 'no' bappend 13 bappend 10
        exit
    yes:
        display 'yes' bappend 13 bappend 10
end calminstruction

test2    

Docs say:
Code:
The "name" modifier makes the wildcard match a complete symbol identifier:    

Usually in the manual "name" means a sequence of tokens not broken by special characters, if I recall correctly, so maybe `identifier` would be a better name for the modifier?
Post 21 May 2025, 19:02
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8434
Location: Kraków, Poland
Tomasz Grysztar 21 May 2025, 19:22
dosmancer wrote:
Usually in the manual "name" means a sequence of tokens not broken by special characters, if I recall correctly, so maybe `identifier` would be a better name for the modifier?
The manual uses "name" loosely in many different meanings, but is precise when using the "name token" phrase, which refers to a single alphanumeric-like token that is not a number and is most likely what you had in mind here.

A name token is, as its name suggests, always a single token, therefore it could be separated easily with the classic MATCH. But symbol names can also be specified by more complex, multi-token identifiers, and "name" wildcard provides an ability to separate them without splitting into individual tokens.
Post 21 May 2025, 19:22
View user's profile Send private message Visit poster's website Reply with quote
dosmancer



Joined: 20 Feb 2025
Posts: 38
Location: Kingdom of Sweden
dosmancer 21 May 2025, 19:45
Tomasz Grysztar wrote:
The manual uses "name" loosely in many different meanings, but is precise when using the "name token" phrase, which refers to a single alphanumeric-like token that is not a number and is most likely what you had in mind here.

Yes, I thought that was a strict definition of what was meant by `name`. Thank you for the explanation.

_________________
fasm68k
Post 21 May 2025, 19:45
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.