flat assembler
Message board for the users of flat assembler.

Index > Programming Language Design > CALM extension of fasmg

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
Ever since I released fasmg 5 years ago, I've been mentioning my intention to make some kind of "assembler construction kit" based on this engine. What I initially had in mind was a set of documentation and tools that would aid in writing native instruction handlers working with fasmg core - and I even wrote a short tutorial implementing few examples of simple x86 instructions. Near the end of that tutorial I mentioned one of the problems that actually kept me from trying to create fasm 2 this way. After I started using fasmg to customize output formats (including relocations) through macros, I realized it would be very hard to give up flexibility of this approach. I though I could perhaps make native instruction handlers that would be able to call user-defined macros for things like "emit dword", but I felt it would be clunky and not easy to implement well. On the other hand, my implementation of x86 instructions in form of macros, even though slow, turned out to be not really as slow as I feared. Honestly, it was (if only barely) fast enough that I could use it for medium-sized projects like fasmg and enjoy all the flexibility given by the macros. In addition to that, my vision of fasm 2 had some bold ideas that turned out to be much easier to implement when using fasmg's macros for the task.

In parallel (and since the very beginning) people have been suggesting another idea - to make a separate language that could be compiled into instruction handlers. Such definitions could even be processed and compiled at the time of assembly, so they might be as flexible as macro packages, while offering better performance. I liked the idea in principle, as it had something in common with a bytecode used internally by fasm 1 (which is one of the few features of fasm that were completely missing in fasmg). A potential of bringing advantages of both approaches together sounded really tempting. The problem, however, was that I had no clear idea what that language should look like. While designing the language of fasmg one of my basic principles was to keep as much familiarity as possible, retaining or at least imitating various syntactic structures of fasm 1. But here I needed to invent something distinct and I had no good idea how to make it not look like another completely new language that one needs to learn in addition to the basic language of fasm.

All these ideas have been hanging around in the corners of my mind, waiting for a good moment to get combined into something that would really speak to me. And as I continued to be reminded from time to time that awfully long assembly times with macros may sometimes be a roadblock, I had a very good incentive to keep thinking about it. And this is how I came up with the design of CALM language.

It has several features that made me feel that this is exactly what I needed. First and foremost, it utilizes the existing framework of fasmg engine in such way that it was really easy to implement (which was quite important for me, considering various constraints). As a consequence, it also uses concepts and syntactical structures that should feel familiar to anyone already knowing fasmg, even though it really is a completely new sub-language. For example, there is a MATCH command that in use feels mostly the same as such named directive of fasm and fasmg. On the other hand, there is a completely a different control flow - it uses jumps instead of structural programming (the main rationale for this was to allow for better performance), which makes it feel like a kind of assembly language. And this is where the name came from - Compiled Assembly-Like Macro.

The instructions written in this language are compiled at the time of definition into a code for a specialized VM. Unlike macros, they do not need to allocate a new namespace for local symbols every time they are called, which allows them to not only be faster, but also require much less memory for processing. And for any task that they are not able to perform themselves, they can simply construct a text and pass it to regular assembly - this allows things like x86 instruction handler calling DD macro defined by the relocation-handling macros, which one of the things I so much wanted to have.

The fact that these instructions can be transparently used in place of existing macro, and also can execute other existing macros themselves, makes it possible to write CALM replacements for individual macros without having to rewrite entire packages. For instance, I started by rewriting "x86.parse_operand" macro in 80386.INC package, while leaving all other macros that interact with it unchanged. This change alone increased the speed of assembly noticeably, while requiring substantially less effort that rewriting the whole instruction set.

The command set is very concise, at least for start. I needed just a dozen of commands to re-implement x86 macros, and whatever cannot be done with them I do by simply assembling a regular statement of fasmg's language from inside the CALM instruction. While I have many interesting ideas for additional commands, I prefer to not implement them too hastily, but first find out in practice what is really needed.

What I am especially fond of is that this extension plugs into the fasmg engine in a way that should allow to utilize it to its full potential. While it is still not going to be anywhere as efficient as the specialized and minimalistic approach of fasm 1, it brings together features of both designs into a compromise that may (I hope) be just good enough for most purposes. I am only afraid that by removing the main bottlenecks CALM instructions may expose some portions of the engine that I did not care to optimize, because previously it did not matter. Normally this should simply lead to further development and improving the code, but currently this just adds to the end of terribly long list of additional things to do that I brought onto myself by starting this project.

Also, I would like to thank MaoKo for some early testing of the extension.

Q. Why not compile the existing macros instead?

A: The macros by their very nature are not well-suited to be compiled. Because they are at their core just a substitution of text, the lines they generate may turn out to contain different commands and different syntactical structures every time a macro is called. While this allows for some fun tricks, it also prevents a pre-compilation of a kind that is available to language like CALM.

It is important to note that this basic simplicity of macros is also often an unwanted feature. For example when you write a macro like:
Code:
macro set var, val
        var = val
end macro    
you probably do not really want this macro to be able to execute an IF directive when called like "set IF 0, 1". It is a side-effect of macro being just a substitution of text and to get rid of this unwanted effect and allow the instruction to be pre-compiled, the semantics of the entire process need to change. In the language of CALM all commands have well-defined effects and the data of the arguments passed to the instruction are held and processed in variables accessed by these commands. The above macro rewritten as CALM instruction would look like:
Code:
calminstruction set var, val
        compute val, val
        publish var, val
end calminstruction    
and here semantics prevent "var" argument from being anything other than the identifier of a symbol to define.

Q. How fast is it?

A: For x86 assembly, we can get the final verdict only after implementing entirety of instruction set using CALM, although even rewriting just a single key macro in the package can already have a noticeable effect. Please do not expect miracles, though - this is supposed to somewhat counteract ridiculous slowness of implementing everything through macros, not magically make it as fast as a natively implemented assembler.

Currently I have rewritten just a few of the most frequently used macros in the basic x86 package, and this already substantially reduced the time of fasmg's self-hosting.
With 80386.INC:
Code:
4 passes, 5.1 seconds, 62464 bytes.    

With 80386.ALM (which still uses 80386.INC under the hood, and just overwrites a couple of key macros):
Code:
4 passes, 1.3 seconds, 62464 bytes.    
(the memory requirements are also similarly reduced, fasmg's macros are quite memory-hungry).

Rewriting all instructions into CALM should increase the speed even further (although it is going to have diminishing returns). However, this still would not use the potential of CALM to its fullest. I'm rewriting the macros in a way that preserves some compatibility, for example "x86.parse_instruction" can still be used the same way as with macro implementation (it is used by some external packages, like the "fastcall" macro). This requires some additional shuffling of the values between namespaces, etc. I think it might be worth it to later attempt writing another implementation completely from scratch, with architecture perhaps more reminiscent of fasm 1 internals, as it could be even faster.

Q. Does this make normal macros obsolete?

A: I think of these two ways of defining instructions as complementing each other. Someone may find it easier to write a macro for a simple task where the performance difference does not matter. Some ideas may be better conveyed in form of a macro. Also, macros may be able to achieve some things not possible with CALM and vice versa, but they can cooperate and call each other if needed.

On the other hand, it feels like many of the existing macro packages would simply become much better if rewritten using CALM. This is certainly going to be a continued process and I would expect the content of standard packages to change quite a lot over time. The good news is that all old macros are still going to work as usual, CALM just adds another way of implementing new instructions.

One type of macros that almost make no sense to use when CALM is available are the interceptor macros, used for example to detect and process nonstandard syntax. Because they are usually called for a large percentage of lines in the source text, having them as transparent and fast as possible is crucial. CALM can provide both these things. Let me show you another example that I used for testing:
Code:
struc (symbol) ? definition&
        match [index] == value, definition
                repeat 1, i:index
                        symbol#i = value
                end repeat
        else
                symbol definition
        end match
end struc

abc[1+2] = 'test'

display abc3    
Even though this interceptor is called only for lines starting with a label, it still slows down the assembly process noticeably even on medium-sized sources. However the slowdown nearly disappeared when I tried it with a CALM version of the same interceptor:
Code:
calminstruction (symbol) ? definition&
        local   index, value
        match   [index] == value, definition
        jyes    indexed_definition
        arrange definition, symbol definition
        assemble definition
        exit
    indexed_definition:
        compute index, index
        arrange definition, symbol#index
        compute value, value
        publish definition, value
end calminstruction    

On the opposite end, an example of a package that relies on the features of macros that are (at least for now) not directly available to CALM instructions is the one that provides anonymous labels.

Q. What does it mean for the prospect of fasm 2?

A: After the dust settles, I am certainly going to make another attempt at implementing my advanced x86 encoder in form of CALM instructions. I'm probably going to start it from scratch, as mentioned above, to make optimal use of the new techniques. I hope that if I succeed, this may be something deserving the name of fasm 2 - with all the customizability I wanted it to have and perhaps (hopefully!) an acceptable performance.

Q. I would like to try it. Where is it?

A: I'm taking my time to prepare a good initial release. You can, however, access the current shapshot from the public copy of fasmg's Fossil repository.
Post 01 Jan 2020, 16:54
View user's profile Send private message Visit poster's website Reply with quote
guignol



Joined: 06 Dec 2008
Posts: 724
guignol
you are bloating assembler
and confusing self-reliance (or self-indulgence, if you please) with actual tool-kit for assembly purpose

what is needed for that range of flexibility and effectiveness that you seek is a compiler that can change itself yet retaining the possibility to produce for any given platform
Post 02 Jan 2020, 03:06
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
I converted more of the x86 instructions, including all the most crucial ones, and got self-hosting down to:
Code:
4 passes, 1.0 seconds, 62464 bytes.    
(compared to 5.1 seconds with pure macro implementation). This is more or less what I expected. In my various preliminary tests CALM instructions are on average about 6 times faster than equivalent macros.

I'm going to keep working on the new instruction packages, but fasmg itself seems mostly ready to release at this point (I just need to run it through a fuzzer now). If you'd prefer me to release it early, please let me know!

It felt like a discovery to me when I realized that the engine actually allows me to customize commands used to define CALM code, not only in form of macros, but even as CALM instructions themselves. At first I thought it might seem a little crazy, but it turns out that it can be quite pleasant work with:
Code:
; INIT
; this command can be used to give an initial numeric value to local variable
; at the time when the CALM instruction is defined
calminstruction calminstruction?.init? var*, val:0
        compute val, val
        publish var, val
end calminstruction

; INITSYM
; this command can be used to give an initial symbolic value to local variable
; at the time when the CALM instruction is defined
calminstruction calminstruction?.initsym? var*, val&
        publish var, val
end calminstruction

; UNIQUE
; generates a new unique identifier and stores it in given variable
; (the identifier uses the name of said variable as a prefix)
calminstruction calminstruction?.unique? name
        local counter, buffer
        init counter
        compute counter, counter + 1
        arrange buffer, name#counter
        publish name, buffer
end calminstruction

; ASM
; generates code to assemble given line of text as-is
calminstruction calminstruction?.asm? line&
        local tmp, ln, buffer
        initsym tmp, unique ln
        assemble tmp
        publish ln, line
        arrange buffer, =assemble ln
        assemble buffer
end calminstruction    
Altering existing commands is possible as well:
Code:
; Extend the standard ASSEMBLE command with additional syntax:
; argument enclosed in braces is going to be treated as text to assemble directly
calminstruction calminstruction?.assemble? statement&
        match {text}, statement
        jyes assemble_text
        arrange statement, =assemble statement
        assemble statement
        exit
    assemble_text:
        arrange statement, =asm text
        assemble statement
end calminstruction

calminstruction tester
        local buffer
        arrange buffer, =display 'testing one',13,10
        assemble buffer
        assemble { display 'testing two',13,10 }
end calminstruction
tester    
And, going a step further, an unconditional interceptor gives even more options of customizing syntax (note that only this kind of interceptor works in CALM definitions, as they otherwise use a different assembly mode):
Code:
calminstruction ?! line&
        match var val, line
        jno default
        match == val, val
        jno default
        arrange line, =compute var, val
        assemble line
        exit
    default:
        assemble line
end calminstruction

calminstruction tester
        local a
        a = 0
    loop:
        check a = 100
        jyes done
        a = a + 1
        assemble { display '.' }
        jump loop
    done:
end calminstruction

purge ?

tester    
For now I'm just playing with it (it is really satisfying when a world of new possibilities emerges from a good design), but this shows how it might be possible to write new modules in multiple layers, allowing to sculpt definitions of an instruction set with a syntax tuned specifically for a given architecture. However, I keep writing x86 instruction handlers as mostly raw CALM code, to get a better feel of it (and see whether it needs any improvements). But when I start writing the next encoder, I may even attempt to write something that would generate CALM instructions out of a more maintainable set of definitions.

There are, in fact, so many interesting things to do with it, that I'm a bit overwhelmed, especially since I have nearly depleted my reserves of spare time for now. This is perhaps an argument for releasing CALM-powered fasmg already, and hoping that there might be others interested in joining the efforts.


Last edited by Tomasz Grysztar on 04 Jan 2020, 11:40; edited 2 times in total
Post 03 Jan 2020, 16:58
View user's profile Send private message Visit poster's website Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 364
Location: Australia
redsock
Wow this is fantastic!

My vote is definitely a +1 for CALM-powered fasmg release, though I don't know how I feel about retiring my reliance on fasm1 Smile

Is this going to become "fasm 2" then?

_________________
2 Ton Digital - https://2ton.com.au/
Post 03 Jan 2020, 22:35
View user's profile Send private message Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 38
jacobly
I was already working on rewriting my ez80 macros in CALM before reading this topic. I have found that even trivial macros are faster when translated to CALM and can be even faster if carefully rewritten. I initially assumed that compute would simplify constant subexpressions as it precompiled, but when I realized that wasn't the case, I now precompute those expressions manually instead and inject the final result with repeat 1. Lately, I have been struggling to figure out a way to create unique label names in a custom CALM command. Your unique macro sounds promising, but it doesn't seem to be useful for anything. For example:
Code:
calminstruction test
        asm display 'first', 10
        asm display 'second', 10
end calminstruction
test    

shows that unique only gets called once inside asm, just like how init only gets called once inside unique.

I think my current specific problem could be solved with . or # support in labels like so:
Code:
macro calminstruction?.do1? id
        id.first:
        id#second:
end macro
calminstruction calminstruction?.do2? id
        local temp
        arrange temp, id.=first:
        assemble temp
        arrange temp, id#=second:
        assemble temp
end calminstruction

calminstruction test
        do1 x
        do2 y
end calminstruction
test    

The only workaround I have found so far is passing them explicitly like:
Code:
do1 x, xfirst, xsecond    
Post 04 Jan 2020, 09:18
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
jacobly wrote:
For example:
Code:
calminstruction test
        asm display 'first', 10
        asm display 'second', 10
end calminstruction
test    

shows that unique only gets called once inside asm, just like how init only gets called once inside unique.
Yes, I made a mistake, UNIQUE should be evaluated at different point. I'm going to rewrite it.

As for your second problem, allowing # in CALM command/label names is certainly possible, just a bit of work. Please be patient, though - at the moment I'm having more ideas than I'm able to process.
Post 04 Jan 2020, 09:51
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
I updated the text of my example, it just needed UNIQUE to be assembled at the time when ASM is executed, not when it is defined. Note that I needed INITSYM there in order to preserve the recognition context of "ln".

As for your second problem, all I can offer for now is some kind of pre-defined name pool:
Code:
define name_pool
repeat 100
        eval 'define name_pool.',`%,' loc',`%
end repeat

calminstruction calminstruction.id? var*
        local   counter
        match   (init), var
        jno     generate
        compute counter, init
        exit
    generate:
        compute counter, counter + 1
        publish var, counter
end calminstruction

calminstruction calminstruction?.getname? var
        local   n
        asm     id n
        arrange n, =name_pool.n
        transform n
        publish var, n
end calminstruction

calminstruction calminstruction?.label? proxy
        transform proxy
        arrange proxy, proxy:
        assemble proxy
end calminstruction

calminstruction calminstruction?.jumpto? proxy
        transform proxy
        arrange proxy, =jump proxy
        assemble proxy
end calminstruction

calminstruction test
        jump    test

        id      (0)     ; restart name pool indexing
        local   first

        getname first
        label   first   ; loc1:
        asm     display '1'
        exit

        getname first
        label   first   ; loc2:
        asm     display '2'
        exit

        getname first
        label   first   ; loc3:
        asm     display '3'
        exit

    test:
        jumpto  first   ; jump loc3

        getname first
        label   first   ; loc4:
        asm     display '4'
        exit

end calminstruction
test    


UPDATE: I managed to add concatenation support to the commands/labels identification, which allows for this implementation of "getname" in above sample:
Code:
calminstruction calminstruction?.getname? var
        local   n
        asm     id n
        arrange n, =loc#n
        publish var, n
end calminstruction    
Obviously, it no longer needs the name pool then.

BTW, as it seems there are more people using version from repository than I thought, I started generating new version numbers for these snapshots.
Post 04 Jan 2020, 10:45
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 2936
Location: vpcmipstrm
bitRAKE
Intriguing, you found a middle ground to increase the flexibility even further! I was happy to find the condition flag set by MATCH/TRANSFORM/CHECK persists beyond the following line. Seems like this was by design, but not indicated in the documentation. Lest you'd just include the branch targets on the same line (i.e. CHECK {yes},{no},a=20). Hope it stays that way. Very Happy
Code:
calminstruction calminstruction?.assemble? statement&
        local tmp
        match {text}, statement
        arrange tmp, =asm text
        jyes assemble_text
        arrange tmp, =assemble statement
    assemble_text:
        assemble tmp
end calminstruction    

_________________
¯\(°_o)/¯ unlicense.org
Post 04 Jan 2020, 22:38
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
Yes, it is by design. There is this sentence near the current end of section 15:
Quote:
The result flag is modified only by some of the commands, like "check", "match" or "transform". Other commands keep it unchanged.
The ASSEMBLE even goes the extra mile in order to preserve this flag.
Post 04 Jan 2020, 22:52
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 2936
Location: vpcmipstrm
bitRAKE
Can you better explain what COMPUTE creates?
Quote:
The "compute" command allows to evaluate expressions and assign numeric results to variables.
Yet, the last example needs to convert the result to decimal and then a string. What does this extra separation offer in terms of the COMPUTE design?
Code:
define ḵ
calminstruction (var) ∑ expr*,a*,b*,Δ=0
        compute ḵ,a
  more:
        compute Δ,expr+Δ
        compute ḵ,+1
        check ḵ>b
        jno more

        arrange Δ,Δ
        stringify Δ
        publish var,Δ
end calminstruction

p ∑ <*-3*+1>,0,4 ; <> not needed
display p,13,10    
Are the symbols evaluated only during ARRANGE and just pre-compiled by COMPUTE? The auto-bracketing of sub-expressions certainly simplifies syntax. Thank you.

_________________
¯\(°_o)/¯ unlicense.org


Last edited by bitRAKE on 10 Jan 2020, 23:10; edited 2 times in total
Post 06 Jan 2020, 07:23
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
bitRAKE wrote:
Can you better explain what COMPUTE creates?
Quote:
The "compute" command allows to evaluate expressions and assign numeric results to variables.
Yet, the last example needs to convert the result to decimal and then a string. What does this extra separation offer in terms of the COMPUTE design?
This is based on nomenclature that dates back to fasm 1. A numeric result is what you put into a variable when you use "=" in regular assembly:
Code:
a = 3*7    
Floats, strings and linear polynomials are all kinds of such result. What is important here is that there is an evaluation of expression involved.

On the other hand, a symbolic value is a pre-tokenized text, this is what you assign with EQU/DEFINE:
Code:
a equ 3*7    
There is no evaluation of expression here (although EQU does symbolic replacements, in CALM this is made into separate operation with TRANSFORM), this is just a sequence of syntactical tokens "3", "*" and "7".

All values of arguments to CALM instruction are initially symbolic. When COMPUTE sees such value in an expression it calculates, it needs to parse the sequence of tokens and then evaluate it as a sub-expression. On the other hand, if you give COMPUTE an expression on variables that have numeric values, it does not need to do any parsing, because the "outer" expression has been pre-parsed during CALM compilation.

ARRANGE allows to build a new symbolic value by stitching together various sequences of tokens. When you give it a variable that has a numeric value instead of symbolic, it has to somehow make a sequence of tokens out of it. Therefore it converts it to a decimal token if possible.

Finally, when a sequence of tokens is converted into a string with STRINGIFY, it is made into a numeric value again - a kind of string that you can assign using "=" operator.
Post 06 Jan 2020, 07:59
View user's profile Send private message Visit poster's website Reply with quote
Marut



Joined: 18 Jun 2017
Posts: 12
Location: Veneto, Italy
Marut
Bug report:
the 32-bit calm version on fossil crashes under Linux when compiling.
I bisected the branch with the following results:
Code:
bisect complete
  1 BAD     2020-01-05 16:33:02 e1f4078d57182181
  4 BAD     2020-01-02 21:39:02 ea80771fdd969a94
  6 GOOD    2020-01-02 13:31:28 8a417fa7d1321a1d CURRENT
  5 GOOD    2020-01-01 17:31:39 e12b928581082ac5
  3 GOOD    2020-01-01 12:20:47 cb412820f93bfced
  2 GOOD    2019-12-29 09:25:32 07c6e39d15d67ae2    


To test each branch I did the following:

  1. Compile the branch using fasmg.is7xq
  2. chmod +x fasmg
  3. Move the new fasmg to an opportune location
  4. Compile the branch again using the new fasmg

All the versions run correctly when given no arguments, showing the help string.
With the "BAD" versions when compiling I had an output such as:
Quote:
flat assembler version g.is7xq
5 passes, 1.4 seconds, 61005 bytes.
flat assembler version g.isbt0
malloc(): invalid size (unsorted)
Aborted (core dumped)
Post 06 Jan 2020, 19:32
View user's profile Send private message Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 38
jacobly
With the new bracket detection, parsing the dup syntax manually now seems feasible, but still complicated enough I wanted to share my implementation. This reimplements various data definition macros in terms of single element emits. I am using something similar to this so that duplicated values can be relocated properly.
Code:
iterate type, b, w, d, q, dq, qq, dqq, qqq
        repeat 1, size: 1 shl (%-1)
                calminstruction d#type? data*&
                        local temp, current, count, sequence, duplicate
                        arrange current, data
                loop:
                        arrange count, current, 0
                        arrange sequence,
                        arrange data,
                        match count =dup? sequence, current, ()
                        match sequence =, data, sequence, ()
                        match ( sequence ), sequence
                        jump splitenter
                splitloop:
                        compute current, current
                        arrange temp, =emit size: current
                        assemble temp
                splitenter:
                        match current =, count, count
                        jyes splitloop
                        compute count, count
                        check count > 0
                        jyes repeatloop
                        check count
                        jno repeatend
                        arrange temp, =err 'value out of allowed range'
                        assemble temp
                        exit
                repeatloop:
                        arrange duplicate, sequence
                repeatsplitloop:
                        arrange current, duplicate
                        match current =, duplicate, duplicate
                        compute current, current
                        arrange temp, =emit size: current
                        assemble temp
                        jyes repeatsplitloop
                        compute count, count - 1
                        check count
                        jyes repeatloop
                repeatend:
                        match current, data
                        jyes loop
                end calminstruction
        end repeat
end iterate    
Post 06 Jan 2020, 19:38
View user's profile Send private message Reply with quote
Marut



Joined: 18 Jun 2017
Posts: 12
Location: Veneto, Italy
Marut
Also, I forgot, this is using libc version 2.30-8
Post 06 Jan 2020, 19:41
View user's profile Send private message Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 38
jacobly
Marut, since I can't reproduce the crash, here's my latest 32-bit linux executable to see if it still crashes for you.


Description: flat assembler version g.iskhr 32-bit linux executable
Download
Filename: fasmg.tgz
Filesize: 36.02 KB
Downloaded: 96 Time(s)

Post 06 Jan 2020, 19:53
View user's profile Send private message Reply with quote
Marut



Joined: 18 Jun 2017
Posts: 12
Location: Veneto, Italy
Marut
Your version is working fine on my end, jacobly.

EDIT: Ok, I think I got it.
It was a bootstrapping problem: to compile I need a working compiler, and the public fasmg binaries don't yet support the CALM directives, which are already integral part of the latest development version.
So I had to compile an older version first, one that had support for the new directives without requiring them to run; looks like I picked a botched version.
So the latest version is self-hosting fine.

TL;DR: self-hosting shenanigans, the compiler was the problem, not the compiled.
Post 06 Jan 2020, 20:09
View user's profile Send private message Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 38
jacobly
Ah, other ways to bootstrap are to replace the first line of selfhost.inc with include '../../examples/x86/include/80386.inc' and use the latest fasmg release, or use the latest fasm release.


Last edited by jacobly on 06 Jan 2020, 20:36; edited 1 time in total
Post 06 Jan 2020, 20:33
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
jacobly wrote:
With the new bracket detection, parsing the dup syntax manually now seems feasible, but still complicated enough I wanted to share my implementation.
Great work! I'm amazed to see how some of you are becoming more fluent in CALM than I am, even before the official release. And the bracket detection has only been implemented today (I was not even sure if it's usable enough), so I'm impressed that you already incorporated it so nicely! Smile I'm going to need something similar for my relocating formatters, and I wanted to do it a bit differently, but I like some your tricks.

jacobly wrote:
Ah, other ways to bootstrap are (...), or use the latest fasm release.
Yeah, I'm still using fasm 1 for safe bootstrapping myself.
Post 06 Jan 2020, 20:35
View user's profile Send private message Visit poster's website Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 38
jacobly
Yeah, at first I wasn't sure if not being able to match "enclosings" was going to be an issue, but it still ended up being pretty obvious how to use it to detect arguments entirely enclosed in matching parentheses like so:
Code:
        local temp
        match ( temp ), argument
        jno skip
        match temp, temp, ()
skip:
        compute argument, argument
        jyes indirect    

which is much better than before:
Code:
        local left, middle, right
        match ( middle ), argument
        compute argument, argument
        jno done
        arrange left, (
        arrange right, )
loop:
        arrange middle, left middle right
        match left ( middle ) right, middle
        jyes loop
        match middle ) right, middle
        match middle ), middle
done:
        jyes indirect    

which in turn, is way better than the messy recursive solution I had before CALM.
Post 06 Jan 2020, 20:43
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7750
Location: Kraków, Poland
Tomasz Grysztar
The first official release is out (version "ist2m"). While there remain some x86 instruction set macros that I wanted to convert, I felt that I should not withhold this any longer, as the base package seems already quite stable and way ahead of what was before. And all of you that already started working with CALM, you may now publish your work knowing that the officially downloaded fasmg is going to support it.

At the same time I also updated some of the supplementary macro packages in the GitHub repository. This includes a new XCALM.INC package which provides some of the basic additional commands for CALM that I first showed earlier in this thread. I found that I use them all the time, especially since INITSYM allows to prepare a text equipped with recognition context of CALM instruction, which in turn allows ASM to assemble instructions referring to local symbols.

I certainly should start working on some kind of tutorial for CALM, explaining the tricks like these in depth. On the other hand, I am still learning it myself, even simple things like INITSYM were something that I discovered rather that designed, and I expect there are other interesting techniques waiting to be found (and I'm pretty sure you're going to surprise me with your own!).
Post 13 Jan 2020, 11:18
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.