flat assembler
Message board for the users of flat assembler.

Index > Tutorials and Examples > Migration from fasm to fasmg

Author
Thread Post new topic Reply to topic
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 27 Jan 2019, 18:43
In this thread I'm going to post a series of guides for ones interested in migrating their projects from fasm 1 to fasmg.

0. Why or why not migrate

While fasmg offers much easier and further-reaching customization of assembly, we should acknowledge that it does not surpass fasm 1 in at least one aspect: performance. It is likely to assemble most projects a bit slower than fasm, for two reasons.

First reason is that fasmg has a unified language, with no separate preprocessor, and allows alterations to this language, like changing the meaning of various symbols (even the critical ones like MACRO) at any moment. Since everything is in flux, it is not possible to pre-compile entire source into an internal bytecode, which is what fasm 1 did. Therefore every time a line needs to be processed more than once - either because it is in a block like REPEAT, or because of multiple passes - it is going to need more time to assemble with fasmg than it would with fasm 1 (which would pre-compile it just once and then do all later iterations by interpreting its internal bytecode). And because repeating instructions and having multiple passes is the bread and butter of assembly programming with fasm, fasmg is rarely going to have an opportunity to demonstrate that when processing a raw line a single time it can be actually faster than fasm 1:
Code:
; this assembles faster with fasmg (because every consecutive line needs complete parsing)
dd 1
dd 2
; ... (this source actually has million lines)
dd 1000000    
Code:
; this assembles faster with fasm 1 (because it has line parsed just once and then repeatedly interpreted from bytecode)
repeat 1000000
        dd %
end repeat    
A similar effect can observed when assembly needs more than one pass to be resolved.

Second reason for the worse performance is that fasmg does not implement instruction sets natively. All x86 instructions can only be defined as macros (or as CALM instructions since the CALM extension has been introduced), what translates to additional layers of processing for every single instruction.

On the other hand, this is where fasmg introduces many of its advantages. Because everything is implemented through macros, everything can be customized. It is possible to deeply change (or expand) syntax of instructions, add non-standard optimizations, alter structure of the produced executable/object format, produce formats not supported by fasm, or simply implement an entirely different architecture. Take a look at the main list of resources for fasmg to see some examples of what can be done.

If you consider switching to fasmg, you should therefore consider if the increased assembly time might be a problem in your case (in most cases, especially if your code base is not very large, you might not notice it at all, unless you assemble on an old PC) and if the customization options and easier-to-use macro language offered by fasmg are something you need. Also, as demonstrated on HeavyThing, sometimes the time difference turns out to be marginal anyway.


Last edited by Tomasz Grysztar on 22 Feb 2022, 09:20; edited 2 times in total
Post 27 Jan 2019, 18:43
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 05 Feb 2019, 15:18
1. Windows-based projects

Since fasmg has only command line interface, you have to either have an IDE that can be configured to use it, or assemble from console. In the latter case you may need to add it to the PATH variable on your system, but it does not necessarily need to be made globally. You can make a command line script (batch file) that does something like:
Code:
@echo off
set path=c:\fasmg;%path%
set include=c:\fasmg\include
cmd /k cd c:\fasmg\projects    
And then use it to launch a command line window that has the right environment set up for assembling with fasmg.

In the INCLUDE variable add a path to compatibility macros, which you can get from https://github.com/tgrysztar/fasmg/tree/master/packages/x86/include. They emulate many features of fasm 1 with Windows headers.

With such environment prepared, you might be able to assemble some programs without alterations, if they were using standard headers like WIN32A.INC. Examples like HELLO, DIALOG, MINIPAD or TEMPLATE that come with fasm 1 for Windows assemble correctly as they are. In many cases it works thanks to fasmg being able to forward-reference macros - when directive like FORMAT PE or ENTRY is placed in source before including header files, it is still able to call the appropriate macro that is defined by these headers. However, to make things easier for the assembler it might be a good idea to move the INCLUDE statement to the top.

However, even in programs that seem to assemble with no issues, some places might use a little tweaks to make them more maintainable. Let's make a few changes to TEMPLATE.ASM as an example.

First, labels inside PROC block now work a bit differently, because fasmg has proper namespaces and PROC makes use of them. So labels within a procedure no longer need to start with dot. Moreover, to use local labels from outside the procedure the same way as before, they must be defined this new way:
Code:
proc WindowProc uses ebx esi edi, hwnd,wmsg,wparam,lparam
        cmp     [wmsg],WM_DESTROY
        je      wmdestroy
  defwndproc:
        invoke  DefWindowProc,[hwnd],[wmsg],[wparam],[lparam]
        jmp     finish
  wmdestroy:
        invoke  PostQuitMessage,0
        xor     eax,eax
  finish:
        ret
endp    
Now it is possible to access such local label in the same way as with fasm 1 when the label was defined with a starting dot:
Code:
        jmp     WindowProc.defwndproc    

Another place where fasmg offers something new is the initialization of structure fields. While the compatibility macro allows to do it like before:
Code:
  wc WNDCLASS 0,WindowProc,0,0,NULL,NULL,NULL,COLOR_BTNFACE+1,NULL,_class    
there is also a new option, more readable and maintainable:
Code:
  wc WNDCLASS lpszClassName: _class, style: 0, cbClsExtra: 0, \
              lpfnWndProc: WindowProc, hbrBackground: COLOR_BTNFACE+1, lpszMenuName: NULL, cbWndExtra: 0    
Only the fields that need to be initialized (or given value different from default if the structure has default values) have to be specified.

The headers by default include only a basic instruction set of either 80386 or x64-64, depending on the bitness. If in your program you need to use later instruction sets, you have to include them manually:
Code:
include 'cpu/ext/sse3.inc'
include 'cpu/ext/rdrand.inc'    


Sometimes an old source may be using a symbol that in fasmg became a keyword, for example WIN64\MANDEL from fasm for Windows package uses a label named "iterate", which is a control directive in fasmg. This breaks the entire structure of assembly and causes a cascade of errors throughout the source. A high value for "-e" switch allows to dig out the underlying error:
Code:
fasmg MANDEL.ASM -e1000    
To fix the problem, the offending label needs to be changed, for example to "iteration". But if change of the name was for some reason not a good solution, there would be an option of forcing the symbol to be interpreted as label and not instruction, by putting "?" in front of it:
Code:
?iterate:
      ; ...
        loop    iterate    


The OPENGL example uses a FIX directive to define some new types by making them aliases for directives like DD. There is no FIX in fasmg, but to make this specific application assemble we can define one this way:
Code:
struc fix? alias
        purge .
        macro . args&
                alias args
        end macro
        struc . args&
                . alias args
        end struc
end struc    


In general, whenever source uses the preprocessor of fasm 1 directly - for example defines some macros - it needs these portions to be rewritten. For instance, USECOM defines this macro:
Code:
struc GUID def
 {
   match d1-d2-d3-d4-d5, def
    \{
      .Data1 dd 0x\#d1
      .Data2 dw 0x\#d2
      .Data3 dw 0x\#d3
      .Data4 db 0x\#d4 shr 8,0x\#d4 and 0FFh
      .Data5 db 0x\#d5 shr 40,0x\#d5 shr 32 and 0FFh,0x\#d5 shr 24 and 0FFh,0x\#d5 shr 16 and 0FFh,0x\#d5 shr 8 and 0FFh,0x\#d5 and 0FFh
    \}
 }    
which after conversion to fasmg may look like:
Code:
struc GUID def
        label . : 16
        match d1-d2-d3-d4-d5, def
                .Data1 dd 0x#d1
                .Data2 dw 0x#d2
                .Data3 dw 0x#d3
                .Data4 db 0x#d4 shr 8,0x#d4 and 0FFh
                .Data5 db 0x#d5 shr 40,0x#d5 shr 32 and 0FFh,0x#d5 shr 24 and 0FFh,0x#d5 shr 16 and 0FFh,0x#d5 shr 8 and 0FFh,0x#d5 and 0FFh
        else
                err 'incorrect GUID syntax'
        end match
end struc    
It nicely demonstrates how the language of fasm's preprocessor maps to the new syntax of fasmg.

You can also find official conversions of several classic examples in the GitHub repository.
Post 05 Feb 2019, 15:18
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 06 Feb 2019, 03:03
2. Statement translation

Let's compare some snippets of fasm's syntax with their translations to fasmg, starting with less problematic ones.
Code:
;       fasm 1                                  fasm g    
Code:
        repeat 10                               repeat 10
            a = a + %                               a = a + %
            if a > 20                               if a > 20
                break                                   break
            end if                                  end if
        end repeat                              end repeat

        while a <= 20                           while a <= 20
            a = a + %                               a = a + %
            if % = 10                               if % = 10
                break                                   break
            end if                                  end if
        end while                               end while    
Control directives work as usual, no changes needed there.
Code:
        times 5 nop                             repeat 5
                                                    nop
                                                end repeat    
TIMES is no longer a built-in feature, its syntax was problematic anyway (because of ill-defined boundaries between syntactic elements). Depending on one's needs it is easy to make a macro that would implement it, at least for syntactically unambiguous cases.
Code:
        load a byte from 0                      load a byte from 0      ; legacy syntax

                                                load a:byte from 0      ; recommended syntax

        store byte a at 0                       store byte a at 0       ; legacy syntax

                                                store a:byte at 0       ; recommended syntax    
Some directives even support legacy fasm-compatible syntax, but a new one should be used if possible.
Code:
        rept 5 i:0 {                            rept 5 i:0              ; legacy syntax
            var#i db ?                              var#i db ?
        }                                       end rept

                                                repeat 5, i:0           ; recommended syntax
                                                    var#i db ?
                                                end repeat    
The new REPEAT directive combines abilities of fasm's REPEAT and REPT. In fasmg REPT is just an alias for REPEAT.
Code:
        irp reg, ebx,esi,edi {                  irp reg, ebx,esi,edi
            push reg                                push reg
        }                                       end irp

                                                iterate reg, ebx,esi,edi
                                                    push reg
                                                end iterate    
For symmetry, IRP also has a longer synonym, ITERATE.
Code:
        macro let expr* {                       macro let expr*
            define MATCHED 0
            match =0 a+==b, MATCHED expr \{         match a+==b, expr
                add a,b                                 add a,b
                redefine MATCHED 1
            \}
            match =0 a==b, MATCHED expr \{          else match a==b, expr
                mov a,b                                 mov a,b
                redefine MATCHED 1
            \}
            match =0 any, MATCHED expr \{           else
                err 'unrecognized syntax'               err 'unrecognized syntax'
            \}                                      end match
        }                                       end macro    
In fasmg MATCH behaves a lot like IF and this makes its usage much simpler. The patterns work very similarly to how they used to in fasm, with some additions. A dot, question mark and exclamation mark are now special characters that can be matched, it is also possible to match whitespace by putting equality sign followed by space in the pattern.
Code:
        macro append item {                     macro append item
            match any, list \{                      match any, list
                list equ list,item \}                   list equ list,item
            match , list \{                         else
                list equ item \}                        list equ item
        }                                           end match
                                                end macro    
Accessing values of symbolic variables with MATCH and constructing lists of item works analogously as in fasm.
Code:
        macro tokens expr {                     macro tokens expr
            irps sym, expr \{                       local buffer
                display `sym,13,10                  define buffer expr:
            \}                                      while 1
        }                                               match sym tail, buffer
                                                            display `sym,13,10
                                                            redefine buffer tail
                                                        else
                                                            break
                                                        end match
                                                    end while
                                                end macro    
There is no native IRPS in fasmg, although there is a macro that emulates it. Cutting out individual tokens out of symbolic value can be done with MATCH (in fasm IRPS was just a convenience feature).
Code:
        ENDM fix }                              macro ENDM!
                                                        esc end macro
                                                end macro    
There is no FIX in fasmg, but some of its uses can be mimicked with unconditional macros. In fasm there was also a trick of putting additional instruction on the same line as closing brace, with fasmg such instructions can be simply put in added lines of an unconditional macro.
Code:
        macro strings [label,value] {           macro strings definitions&
            common                                  strtbl:
                strtbl:                             iterate <label,value>, definitions
            forward                                     label db value,0
                label db value,0                    end iterate
        }                                       end macro    
In fasmg there is only the new syntax for variable-length list of arguments for macros (the one with ampersand, which was also back-ported to fasm 1) and iterating over them needs to be done with generic IRP/ITERATE.
Code:
        macro pascall proc,[arg] {              macro pascall proc,args&
            reverse                                 iterate arg, args
                push arg                                indx 1+%%-%
            common                                      push arg
                call proc                           end iterate
        }                                           call proc
                                                end macro    
A new INDX directive allows traversing list in any different order, not necessarily just in reverse. This ends up being much more verbose than classic syntax of fasm 1, but adds more overall flexibility.
Code:
        macro strtbl name,[string] {            macro strtbl name,strings&
            common                                      label name: dword
                label name dword                        local label
            forward                                     iterate string, strings
                local label                                     dd label.%
                dd label                                end iterate
            forward                                     iterate string, strings
                label db string,0                               label.% db string,0
        }                                               end iterate
                                                end macro    
With macro argument grouping gone, fasmg's LOCAL only creates a single local symbol. However, in fasmg every symbol can have its own descendant namespace, accessed through dot operator. Even a plain number of iteration can be used for a name of a child symbol, as in the above translation.
Let's take a look at another variation of the same macro:
Code:
                                                macro strtbl name,strings&
                                                        iterate string, strings
                                                                label name: dword
                                                                local label
                                                                repeat %%
                                                                        dd label.%
                                                                end repeat
                                                                repeat %%
                                                                        indx %
                                                                        label.% db string,0
                                                                end repeat
                                                                break
                                                        end iterate
                                                end macro    
INDX can be used multiple times within a single iteration, each time changing the visible value of iterated parameter. This allows to process all required loops entirely within the first iteration and then simply BREAK out of ITERATE to avoid repeating the process needlessly.
Code:
        alias equ a                             alias equ a
        dd alias                                dd alias
        alias = 1                               match name, alias
        match name,  alias {                        name = 1
            name equ eax                        end match
        }                                       match name, alias
                                                    name equ eax
                                                end match    
In fasm 1 symbolic variables were replaced with their values during the preprocessing and then the assembler only saw substituted text everywhere. Therefore you could define an assembly-time variable or label with name that was replaced during preprocessing. But to define a symbolic variable with a name being extracted from another one, you had to use MATCH trick.
In fasmg there is no separate preprocessing stage and symbolic variables are evaluated as any other ones, only in the expressions (arguments to instructions in general). Therefore a simple "alias = 1" line would be treated as redefinition of "alias" symbol. Therefore you need to use the MATCH trick for all such cases when you want to define a symbol with an extracted name. This illustrates the fact that fasmg has just one unified language.
Code:
        v equ 'abc'                             v equ 'abc'
        if v eqtype ''                          if v eqtype ''
            db v                                    db v
        else if v eqtype 1.0                    else if v eqtype 1.0
            dq v                                    dq v
        else                                    else
            dd v                                    dd v
        end if                                  end if    
While EQTYPE is no longer a general parsing-related tool as it was in fasm (in fasmg it can only compare results of valid numeric expressions), it can still tell strings from floating-point or regular numeric values. It cannot, however, be used to detect more complex syntactical structures, like addresses in square brackets - MATCH needs to be employed instead. Moreover, detection of architecture-specific entities like registers depends on how they are defined and handled by a specific macro package. In case of standard x86 macros that come with fasmg it is possible to call "x86.parse_operand" macro to detect various types of instruction operands.


Last edited by Tomasz Grysztar on 24 Feb 2019, 10:18; edited 3 times in total
Post 06 Feb 2019, 03:03
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 802
Location: Russian Federation, Sochi
ProMiNick 06 Feb 2019, 18:52
It would be nice if each fasmg element would be described in manner of describing CPU instructions (not only action, but list of (classes of) allowed parameters, any other limitations). For example I didn`t know that `buffer would not work.
I understand needance of buffer, because redefine of expr would be...
... incorrect, but I can say why... and so on.
Post 06 Feb 2019, 18:52
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 06 Feb 2019, 19:08
The manual contains a complete description of all the features of language that are there by design. In case of fasmg design of language mostly preceded the implementation. There shouldn't be any "hidden" features - what has been implemented is what was designed and written down.

The fact that ` applies to parameters is documented in section about macros. Other instances where parameters are present in the language are described later when discussing consecutive directives that use them - like section about REPEAT and family, where information about ` being a modifier for parameters is given again.

Note that the manual has been written in a way that introduces information gradually, never using terms that have not been fully introduced earlier, but at the same time assuming that what was in preceding sections is already known. Therefore to get a full picture you should read it all consecutively. Yeah, I'm aware that (almost) no one does that... Wink
Post 06 Feb 2019, 19:08
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4054
Location: vpcmpistri
bitRAKE 07 Feb 2019, 03:39
My approach has never been one of wanting fasm/fasmg to be "assembler X" - they are their own languages to be leveraged in a myriad of ways. Maybe this is not typical. I've read the manual several times and search it frequently as well as this board for examples. Getting comfortable with how to access symbols and evaluate them takes time. Kind of similar to dereferencing values in assembler - I know I have a pointer, but is it at a memory address?
Code:
match any,buffer
  display `any
end match    
An index of the manual would not be a bad thing. ":" what does it do? ...in the context of a macro? Missing a key sentence can have one rereading large portions, or not understanding why an error happened.

I feel fasmg is superior in every way - assemble time has never been a hindrance to development - despite how much it might be talked about. Fasmg holds the same comparative nuiance that assembler has when compared to HLLs, imho. Which makes it both concise and cumbersome, but that is the cost of such fine grained flexibility. I will continue to struggle with it in the hopes of better navigating the transforms that it provides.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup
Post 07 Feb 2019, 03:39
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 07 Feb 2019, 06:47
bitRAKE wrote:
An index of the manual would not be a bad thing.
A good point!
Post 07 Feb 2019, 06:47
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 08 Feb 2019, 08:23
I keep adding small examples as I come up with ones. Please let me know if there is something important for you that seems missing.


After the latest batch of fixes to the compatibility macros, it is now possible to assemble fasmw with fasmg out of the box, using the setup from the above guide. The resulting binaries are identical except for some header fields, timestamps and alignment fillers.

I have cleaned up a couple of places in fasm 1 sources where the problematic syntax of PUSH/POP with space-separated operands other than registers was used, to avoid having to use a special macro just to assemble these sources with fasmg. Although with help of CALM even an emulation of that troublesome syntax might be possible without a huge performance loss, making the fasm source use a cleaner syntax is probably a good thing anyway.
Post 08 Feb 2019, 08:23
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 12 Jul 2023, 12:51
The new fasm 2 header set pushes the compatibility even further, allowing in many cases to assemble legacy sources with no changes, while also giving some new features like "format MachO" or "use" settings to choose target CPU. It should mostly work out of the box, so it is likely the easiest option right now for migration onto the new engine.
Post 12 Jul 2023, 12:51
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.