flat assembler
Message board for the users of flat assembler.

flat assembler > Macroinstructions > fasmg as a preprocessor

Author
Thread Post new topic Reply to topic
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6873
Location: Kraków, Poland
While the macro language of fasmg is much more capable than that of fasm 1, we may still prefer to assemble larger projects with the latter, especially for performance reasons. An interesting question is whether we could use fasmg as a preprocessor for the source before passing it to fasm 1 for the final assembly.

It is easy to process all the lines of source file with "macro ?!" feature of fasmg, but it is normal for assembly programs to span multiple files, joined by the INCLUDE directives. A preprocessor macro-script would need to process them accordingly and collect the contents of all included files for processing.

Relatively recently I discovered a trick that uses some of the obscure features of fasmg to process source text in an environment where none or almost none of the usual instructions/directives are defined. This allows to use a simple "macro ?" to catch all the lines except the ones with INCLUDE, by defining INCLUDE as an unconditional instruction within that environment. And because INCLUDE itself is not intercepted by "macro ?", the latter is then applied to the lines inside the included file.

Here I present a simple template of such preprocessing macro-script:
Code:
PREPROCESSOR := __FILE__ namespace PREPROCESSOR define symlinks include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR match include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR, symlinks macro preprocess file local empty macro empty.include?! file include file end macro namespace empty macro ? line& if __FILE__ = PREPROCESSOR purge ? line else namespace PREPROCESSOR db `line,13,10 end namespace end if end macro include file end namespace end macro end match preprocess source end namespace
It simply passes to the output all the tokenized lines without any further changes. The source becomes solidified into a single file, with comments and excess whitespace removed, as this happens during the tokenization process.

The script requires "source" variable to be defined with a quoted path to the main source file to preprocess. This can be done by adding a definition on top of the script, or by defining it directly from the command line:
Code:
fasmg preprocess.asm -isource='hello.asm' preprocessed.asm
The above works in Windows command line, in Linux console the quotes need to be escaped:
Code:
fasmg preprocess.asm -isource=\'hello.asm\' preprocessed.asm


There are several tricks used here. First, the namespace of a local variable is used as a context for reading and interpreting the source files. This namespace is detached from the main symbol tree, therefore inside of it no defined symbols exist, including no instructions/directives. We define a single macroinstruction inside that namespace - INCLUDE - and make it call the original INCLUDE from the main symbol tree.

This works because of the outer MATCH that assigns the symbolic links to various symbols from the main tree (stored in "symlinks" variable) to parameters that are replaced with these symbolic links everywhere inside that block, which included the definition of "preprocess" macro. Therefore the body of "preprocess" macro contains symbolic links instead of just raw names of directives like INCLUDE or IF, so they are recognized and called correctly even from inside the "empty" namespace. When INCLUDE macro is processed, its body contains a symbolic link to the original INCLUDE.

The DB directive is not among those replaced with symbolic links, but "namespace PREPROCESSOR" block is used to temporarily switch context back to where all the normal instructions are defined. Inside this block we may put some more interesting preprocessing later.

The preprocessing ends when the value of __FILE__ signal that we are back in the file containing the preprocessing script, which means that everything inside the INCLUDE has been processed.


Now that we have a template ready, we can try doing something more interesting with it. I have chosen a simple task - scan all the function labels (assumed to be defined as "function:") and the CALL instructions in the source and then generate EXTRN declarations for any function that is seen to be called but not defined anywhere. A script doing this may look like this:
Code:
PREPROCESSOR := __FILE__ namespace PREPROCESSOR define symlinks include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR match include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR, symlinks macro preprocess file local empty macro empty.include?! file include file end macro namespace empty macro ? line& if __FILE__ = PREPROCESSOR purge ? line else namespace PREPROCESSOR match name:, line if ~ defined local.name restore local.name ; force variable status to disable forward-referencing local.name = 1 end if else match =call? name, line match any any, name ; ignore multi-token arguments else if ~ defined called.name restore called.name called.name = 1 define called name ; collect for IRPV end if end match end match db `line,13,10 end namespace end if end macro include file end namespace end macro end match preprocess source irpv name, called if ~ defined local.name db 13,10,'extrn ',`name end if end irpv end namespace
This still does not alter any of the source lines, it just appends some EXTRN definitions at the end.

Now we might prefer to put the EXTRN definition on top of the file. This can be done easily by keeping the preprocessed source temporarily in a VIRTUAL block. Since fasm 1 does not allow EXTRN definitions before the FORMAT directive we need to look for FORMAT and only then start buffering the further lines in a VIRTUAL block so we can later dump them all after the EXTRN definitions are generated:
Code:
PREPROCESSOR := __FILE__ namespace PREPROCESSOR define symlinks include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR match include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR, symlinks VIRTUAL_BLOCK = 0 macro preprocess file local empty macro empty.include?! file include file end macro namespace empty macro ? line& if __FILE__ = PREPROCESSOR purge ? line else namespace PREPROCESSOR match name:, line if ~ defined local.name restore local.name ; force variable status to disable forward-referencing local.name = 1 end if else match =call? name, line match any any, name ; ignore multi-token arguments else if ~ defined called.name restore called.name called.name = 1 define called name ; collect for IRPV end if end match end match db `line,13,10 match =format? any, line VIRTUAL_BLOCK = 1 virtual VirtualBlock:: end match end namespace end if end macro include file end namespace end macro end match preprocess source if VIRTUAL_BLOCK end virtual end if irpv name, called if ~ defined local.name db 'extrn ',`name,13,10 end if end irpv virtual VirtualBlock load contents:$-$$ from $$ end virtual db contents end namespace

That's all for now. I may post more when I prepare another interesting example. If you do something interesting with this template, please let us know!
Post 06 Jul 2018, 18:40
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6873
Location: Kraków, Poland
What if one would prefer to just write a regular macros that would get preprocessed, even if just for clarity? We can make a framework that would allow something like that.

Let's say we have the following macro that we would like to apply in our source:
Code:
macro mov? args& match [destaddr]=,[srcaddr], args push [srcaddr] pop [destaddr] else mov args end match end macro
A modified preprocessing script allowing to use the above macro with relatively little changes may look like this:
Code:
macro emit? line& db `line,13,10 end macro define MACROS define symlinks macro:end match macro:end, symlinks macro MACROS.macro?! definition& local NAME rawmatch name parameters, definition define NAME name else define NAME definition end rawmatch match name, NAME define name +name macro end?.macro?! esc end macro end match purge end?.macro? end macro esc macro definition end macro end match namespace MACROS ; The macros to be preprocessed go here macro mov? args& match [destaddr]=,[srcaddr], args emit push [srcaddr] emit pop [destaddr] else emit mov args end match end macro end namespace PREPROCESSOR := __FILE__ namespace PREPROCESSOR define symlinks include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR match include:macro:purge:if:else:end:namespace:__FILE__:PREPROCESSOR, symlinks macro preprocess file local empty macro empty.include?! file include file end macro namespace empty macro ? line& if __FILE__ = PREPROCESSOR purge ? line else namespace PREPROCESSOR match +command, MACROS.line command else emit line end match end namespace end if end macro include file end namespace end macro end match preprocess source end namespace
The EMIT macro is needed to distinguish the lines that would go into preprocessed source from the ones processed locally by the preprocessing script.
Post 07 Jul 2018, 11:33
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 2004-2018, Tomasz Grysztar.

Powered by rwasa.