flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > [calm] display stringified value

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 21 Jan 2023, 02:05
This does not work:

Code:
local value
...
arrange value, value
stringify value
arrange cmd, =display value
assemble cmd    



I am kind of lost between the numeric value, the decimal token and the string token.
Post 21 Jan 2023, 02:05
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 21 Jan 2023, 09:16
ARRANGE in principle operates on symbolic values, as it composes a tokenized text and only symbolic variables provide values of this kind. There is a single exception: ARRANGE also can convert a plain positive number into a token corresponding to a sequence of decimal characters, which is a kind of value it can put into the produced tokenized text.

For other types of computed values, including negative numbers, floating-point numbers, strings, and linear polynomials, there is no such conversion to text implemented and ARRANGE does not allow them. If you need to pass such value to a directive like DISPLAY, the best way is to refer to a variable holding the value. The manual contains an example that does exactly that - look for the "calminstruction show? text&" example, which uses "initsym" to compose assembled text that refers to a local variable. Pointing to a local variable is only possible with a trick like "initsym", but you could avoid it by using a global variable as a proxy:
Code:
        define global

        calminstruction show? text&
                arrange global, text
                stringify global
                arrange text, =display =global
                assemble text
        end calminstruction

        show :)    
Post 21 Jan 2023, 09:16
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 21 Jan 2023, 12:44
I think I am lost because I don't picture the full hierarchy of fasmg object types, then I have a very approximate view of what calm instructions do.
Post 21 Jan 2023, 12:44
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 21 Jan 2023, 14:56
It really follows the principles of fasm 1 quite closely. In fasm 1 you had two layers: preprocessor, operating purely on tokenized text, which was called "symbolic value" (EQU/DEFINE were preprocessor's directives in fasm 1, hence they defined symbolic variables), and assembler, operating on computed binary objects.

With fasmg I retained this classification, but now there is a single engine, so symbolic values and computed values can be assigned to symbols on the same layer. But overall, you still have two worlds. First a symbolic one, where you operate on tokenized text, this is where you use EQU, DEFINE, and also operations like MATCH. In case of CALM it's the ARRANGE and also MATCH (there they are in fact dual to each other).

The second world is all the computed values. All the types that exist in fasmg existed in fasm 1 already and could be compared with EQTYPE - numeric values (which were an early version of fasmg's linear polynomials, allowing for up to two scaled registers and possibly relocatable base, and only 65 bits of precision), floating-point values, strings. In fasm 1 only numeric values could be held in variables, so = could not assign floating-point value or string without converting it to plain numeric value. In case of fasmg variables are capable of holding any of these three types.

In short, EQU/DEFINE is for symbolic values (snippets of source text), = is for all the other ones. This classic example from the manual illustrates the basic idea:
Code:
        numeric = 2 + 2
        symbolic equ 2 + 2
        x = numeric*3           ; x = 4*3
        y = symbolic*3          ; y = 2 + 2*3    
Note that this snippet does exactly the same thing in fasm 1 as in fasmg.

In CALM context you have ARRANGE operating almost purely in the world of text (with that single exception of being able to convert plain numeric value to a sequence of decimal digits), analogously to EQU, while COMPUTE is like = and can produce all other kinds of values.

STRINGIFY takes a symbolic value and converts it to string type. This is now in the computed world, you no longer have a snippet of source text, only a string of raw bits. There is no way to get back to source text world from here - again with the single exception of producing a decimal digits sequence, which only works for plan positive integer values. You could actually treat the string as a binary representation of a huge integer, then convert it to decimal representation, and then force DISPLAY to convert that number into binary string once again. But that implies unnecessarily converting back and forth and I do not recommend this technique:
Code:
        calminstruction show? text&
                stringify text
                compute text, +text
                arrange text, =display =string text
                assemble text
        end calminstruction

        show any text      
To see what happens there, just replace "display" with some nonsense that would cause an error:
Code:
                arrange text, =foobar =string text    
And the error message shows what is produced:
Code:
test.asm [8]:
        show any text
show? [4]
Processed: foobar string 8392569455040360033
Error: illegal instruction.    
That's why I suggest keeping the string value in a variable and then referring DISPLAY to that variable. But you can use whatever suits you best.
Post 21 Jan 2023, 14:56
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 21 Jan 2023, 15:09
A string is not a token from the tokenized world?

You mean there is a string token and a string binary value?
Post 21 Jan 2023, 15:09
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 21 Jan 2023, 15:16
sylware wrote:
A string is not a token from the tokenized world?

You mean there is a string token and a string binary value?
Yes, the string in tokenized text is a different thing, it is complete with enclosing quotes, it could even be missing the closing quote and be causing an error because of that.

The string as the value in computed context is just the bytes corresponding to the contents between quotes, and nothing else.

You could think of this distinction as: a representation of string in source text vs its run-time form.
Post 21 Jan 2023, 15:16
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 21 Jan 2023, 16:37
Allright! And calm stringify generates a string binary value, not a string token in a "tokenized" value.

Then if I am not mistaken the string operator does cast inplace the bits of a value to a a binary string. It has nothing to do with a "string" token.

This is what I meant, all this is very confusing,"string" can be a token or a binary value. This near-internals information is required for proper understanding of calm programming, and even normal fasmg programming (equ/define,=,etc).

Basically, we would need a "advanced" cheat sheet to know what types of values (tokenized/binary) a command/operator needs and produces.

-- edit

I guess, apart from classic macros, "commands" working and producing tokenized values should be somewhat rare. Then an explicit reference of those commands could be nice, and those transforming for good tokenized values to binary values (since there is no come back).
Post 21 Jan 2023, 16:37
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 21 Jan 2023, 17:08
sylware wrote:
This is what I meant, all this is very confusing,"string" can be a token or a binary value.
I generally refer to the former as "quoted string" and I think I more or less consistently did this throughout documentation. It might be worth it to add "(not a quoted string)" remark to the STRINGIFY description. Smile
Post 21 Jan 2023, 17:08
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 22 Jan 2023, 14:39
That could be a way to remove the confusion.

That said, if I picture it well, the calm "assemble" command will expect a variable containing a "tokenized" value. If this "tokenized" value does contain itself a "display" command, the "assemble" command will simply compute the "literals" token (string, numbers) arguments to a binary value, resolve the name token arguments to their corresponding binary values, then call the "display" command with all those binary values as arguments. And the "display" command will refuse any argument which is not a binary value, for instance a "tokenized" value.

Is that not too wrong and/or incomplete?
Post 22 Jan 2023, 14:39
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 22 Jan 2023, 18:39
The way I prefer to describe it, ASSEMBLE command does not compute anything, it just takes the value of a symbolic variable and passes it as a line of text to the assembly queue. I pointed to an artificially induced error message to demonstrate it: what is shown after the "Processed:" header is the raw line of text that got passed to the assembler by the ASSEMBLE command (and this text in our example is produced by ARRANGE). When the text is "display string 8392569455040360033", then the DISPLAY instruction takes over, and it is the handler of this specific instruction that decides what to do further with the text "string 8392569455040360033". In case of DISPLAY, the text that follows it is parsed as an expression, and then the expression is evaluated and the final value displayed as a string or as a single byte value, depending on the resulting type.

I should stress this further: DISPLAY, just like any other instruction or macroinstruction, sees its arguments as a pure text, a portion of the line of source that invoked it. It then parses this text and interprets it according to its own rules. I'm not sure what you had in mind when saying that DISPLAY would refuse any argument which is not a binary value, but it is false in every sense that I can think of. Not only the arguments to DISPLAY are initially just a text, they may also contain identifiers of symbolic variables, that would then be unrolled recursively to find out what the expression is:
Code:
digit equ '0'+
last equ mod 10 
display digit 678 last    
(By the way, this is another source that assembles the same with fasm 1 as well as fasm g.)
Post 22 Jan 2023, 18:39
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 22 Jan 2023, 19:38
When you said "line of text" , I guess a sequence of lexical tokens.

So in my words:

The command arguments would actually be given to the command code as a sequence of tokens, I guess all the tokens following the command name token on the line.

With your example, the display command is calling some expression evaluation code on a part of the sequence of tokens it was given as argument.

Is calling this expression evaluation code is automatic on each argument sequence of tokens (separated by the ',' token) or the command has to call explicitely this evaluation code on a selected parts of the sequence of argument tokens?
Post 22 Jan 2023, 19:38
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 22 Jan 2023, 22:25
sylware wrote:
Is calling this expression evaluation code is automatic on each argument sequence of tokens (separated by the ',' token) or the command has to call explicitely this evaluation code on a selected parts of the sequence of argument tokens?
This seems to be a question concerning specific implementation. From the point of view of language design it should not matter whether particular implementation parses the whole line in advance, or in portions just-in-time. Is there some hidden problem behind this question?
Post 22 Jan 2023, 22:25
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 23 Jan 2023, 00:10
Well, it depends, if calling the evaluation code on the argument sequence of tokens is done independently from the command code, that would remove some freedom from the command to decide the format of its arguments, then the language specifications... but it would mean the command is receiving a list of "evaluated" binary values unless there is pure token-only evaluation code?
Post 23 Jan 2023, 00:10
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 23 Jan 2023, 06:36
As I mentioned, an instruction "sees" the arguments as a line of pure text - it could turn out that DISPLAY was overridden by a macro, but it doesn't matter whether the instruction is implemented in form of macro, CALM, or a built-in handler, in every case you have the same text, like "string 8392569455040360033", and it is up to the implementation of the instruction to decide how to interpret it.

I'm not sure what the confusion is. I should stress once more: when ARRANGE produces a "display string 8392569455040360033" text and passes it to the assembly queue, it is the same as if you have simply written "display string 8392569455040360033" as a line in source file, to the letter. The decimal digits as ASCII characters. Under the hood there might be some optimizations that make representation look a bit different, just to save on processing time, but from the point of view of language user there really should be no difference how this line of text was produced.

Talking about the specifics of fasmg's internal implementation would only cloud the matters.
Post 23 Jan 2023, 06:36
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 19084
Location: In your JS exploiting you and your system
revolution 23 Jan 2023, 08:21
Can we say that arrange is similar to the eval that many other interpreted languages have?
Post 23 Jan 2023, 08:21
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 23 Jan 2023, 09:24
No, fasmg has an actual EVAL and it does something different - it takes the value of a string (possibly a computed one) and interprets it as a new text to assemble:
Code:
eval "db 1"
eval string ("cb 1" + 1)    
A string value differs from a symbolic value - the former is a binary sequence of characters extracted from between quotes or computed in some other way, the latter is a piece of source text (tokenized, but otherwise preserved, it is what you assign with EQU etc.)

Also you probably meant ASSEMBLE, not ARRANGE. ASSEMBLE takes the value of a symbolic variable and assembles it as a new line, an effect similar to this snippet in base language (not CALM):
Code:
match line, variable
    line
end match    
As for ARRANGE, it is basically the opposite of MATCH. Where MATCH takes a symbolic value and cuts it into pieces, ARRANGE can take such pieces and compose them into a single symbolic value.
Post 23 Jan 2023, 09:24
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 23 Jan 2023, 14:36
Allright, then all commands receive their arguments as a block of tokenized text.

That's why "the reference manual" I did talk about in another thread.

For instance in the case the display command, we would need to know what the command is doing with that argument of tokenized text: splitting based on the ',' token, then "expression evaluation" on each of those arguments until we end up with only "binary" values which bytes will be sent to the output device.

Well, this is the idea.
Post 23 Jan 2023, 14:36
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 23 Jan 2023, 15:15
sylware wrote:
Allright, then all commands receive their arguments as a block of tokenized text.

That's why "the reference manual" I did talk about in another thread.

For instance in the case the display command, we would need to know what the command is doing with that argument of tokenized text: splitting based on the ',' token, then "expression evaluation" on each of those arguments until we end up with only "binary" values which bytes will be sent to the output device.

Well, this is the idea.
Yes, a good idea, and this time you described the process quite well.

Anyway, the specific issue that started this thread is going to become less relevant once I finish and release CALM phase 3.
Post 23 Jan 2023, 15:15
View user's profile Send private message Visit poster's website Reply with quote
sylware



Joined: 23 Oct 2020
Posts: 227
Location: Marseille/France
sylware 23 Jan 2023, 16:14
Yep, I think this reference manual is critical for fasmg.

What do you mean by CALM phase 3?
Post 23 Jan 2023, 16:14
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8103
Location: Kraków, Poland
Tomasz Grysztar 25 Jan 2023, 11:57
sylware wrote:
What do you mean by CALM phase 3?
It's the new development branch I linked above.

With a version compiled from that branch you can do this directly in CALM:
Code:
stringify value
display value    
Post 25 Jan 2023, 11:57
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2023, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.