flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Windows > fasm compiler programming tutorial?

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
kerr



Joined: 24 Feb 2016
Posts: 124
fasm compiler programming tutorial?
hello friends
you have fasm compiler programming turorial?
or you have studied its source code?

_________________
I hope we will be good friends.
Post 25 Sep 2017, 14:36
View user's profile Send private message Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
Is this question about how fasm/fasmg itself works or do you want to create windows-programmes with fasm/fasmg?
Post 26 Sep 2017, 13:25
View user's profile Send private message Visit poster's website Reply with quote
kerr



Joined: 24 Feb 2016
Posts: 124

_shura wrote:
Is this question about how fasm/fasmg itself works or do you want to create windows-programmes with fasm/fasmg?



of course is this question about fasm itself works.

_________________
I hope we will be good friends.
Post 05 Oct 2017, 12:59
View user's profile Send private message Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
then better move the topic to ›compiler internals‹. Anyway, the core file is assembler.inc, best way is to start there, but I must confess, it is complicated, because Tomasz does not comment a lot.


Last edited by _shura on 05 Oct 2017, 16:15; edited 1 time in total
Post 05 Oct 2017, 14:33
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland
There is a guide to fasm's internals, it also had some tiny updates later but I never finished it.
Post 05 Oct 2017, 15:44
View user's profile Send private message Visit poster's website Reply with quote
kerr



Joined: 24 Feb 2016
Posts: 124

_shura wrote:
then better move the topic to ›compiler internals‹. Anyway, the core file is assembly.inc, best way is to start there, but I must confess, it is complicated, because Tomasz does not comment a lot.



yes
i want to analyze it .
but it is too difficult.

_________________
I hope we will be good friends.
Post 05 Oct 2017, 15:45
View user's profile Send private message Reply with quote
kerr



Joined: 24 Feb 2016
Posts: 124

Tomasz Grysztar wrote:
There is a guide to fasm's internals, it also had some tiny updates later but I never finished it.



thank you。
are you studying this compiler?

_________________
I hope we will be good friends.
Post 05 Oct 2017, 16:01
View user's profile Send private message Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
So there is no current overview about fasm itself and none for fasmg?
PS: Tomasz is the author of this compiler. he wrote it.
Post 05 Oct 2017, 16:04
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland

_shura wrote:
So there is no current overview about fasm itself and none for fasmg?

I never finished the aforementioned guide and all my design notes for fasm 1 were only on paper (I'd been showing my notepad to the audience of fasmcon 2006), though these were mainly layouts of data structures. Many of them have later become parts of FAS format specification and it is there where you can find the most up-to-date description of many of fasm's internal data formats.

As for the fasmg, all my design notes were digital since the beginning and you can find them in the Fossil repository. They focus more on the design of the language (note that they mostly predate an actual implementation) and have only little information on the internal data structures, but the fasmg sources themselves are more self-descriptive compared to fasm 1. At least fasmg sources use named structure fields and flags (in fasm 1 all of them were hard-coded numbers that I was looking up on my paper notes every time I worked with fasm's source).
Post 05 Oct 2017, 16:27
View user's profile Send private message Visit poster's website Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
Yeah, the good old magic numbers of fasm Razz Anyway, it would be more helpful, if I know, what piece of code does what. E.g. where I have to start, if i want to add a feature to source-processing. I am currently program a compiler with fasmg-macros and I had the idea to replace the token-based system of fasmg with some kind of bytecode, that may speed up the compilation by fasmg.
Post 05 Oct 2017, 16:33
View user's profile Send private message Visit poster's website Reply with quote
kerr



Joined: 24 Feb 2016
Posts: 124

Tomasz Grysztar wrote:

_shura wrote:
So there is no current overview about fasm itself and none for fasmg?

I never finished the aforementioned guide and all my design notes for fasm 1 were only on paper (I'd been showing my notepad to the audience of fasmcon 2006), though these were mainly layouts of data structures. Many of them have later become parts of FAS format specification and it is there where you can find the most up-to-date description of many of fasm's internal data formats.

As for the fasmg, all my design notes were digital since the beginning and you can find them in the Fossil repository. They focus more on the design of the language (note that they mostly predate an actual implementation) and have only little information on the internal data structures, but the fasmg sources themselves are more self-descriptive compared to fasm 1. At least fasmg sources use named structure fields and flags (in fasm 1 all of them were hard-coded numbers that I was looking up on my paper notes every time I worked with fasm's source).



hello about fasm author
What order is the step of the FASM compiler source code?

_________________
I hope we will be good friends.
Post 05 Oct 2017, 16:40
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland

_shura wrote:
Yeah, the good old magic numbers of fasm Razz Anyway, it would be more helpful, if I know, what piece of code does what. E.g. where I have to start, if i want to add a feature to source-processing. I am currently program a compiler with fasmg-macros and I had the idea to replace the token-based system of fasmg with some kind of bytecode, that may speed up the compilation by fasmg.

This would be like the bytecode used by fasm 1 to make quick passes on the parsed source. I mentioned this as an idea of a "the best of the both worlds" assembler when I first announced fasmg, but it would require some compromises. The principles of architecture of fasmg are such that everything is in flux and the meaning of the same token may change even between consecutive executions of a loop that contains it (and of course between the passes). This prevents any viable caching in the name resolver, and this in turn makes it impossible to add a bytecode step like in fasm 1 without substantially altering the rules of the language.
Post 05 Oct 2017, 16:52
View user's profile Send private message Visit poster's website Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
depends on the design of the bytecode. But you said, that long macros are slow, because the assembler has to process e.g. the code in an if-statement, even if this path is not used in an specific macro-call. But with bytecode, you just can put the content of it in another memory and it does not get touched, if the statement is false:

Code:

macro foo bar
  if ( bar = 0 )
    ;some long code
  else if ( bar = 1 )
    ;also long code
  else
    ;even longer code
  end if
  ;always do this
end macro



can be transformed to some bytecode like this:

Code:

_0:
  ;some long code
  done
_1:
  ;also long code
  done
_2:
  ;even longer code
  done
_3:
  push 0 ;argument 0
  pushArgument
  push 1
  compare
  push _1
  push _2
  branch
  done
_macro_foo:
  push 0 ;argument 0
  pushArgument
  push 0
  compare
  push _0
  push _3
  branch
  ;always do this
  done



at least it may speed up huge macros
Post 05 Oct 2017, 17:06
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland

_shura wrote:
depends on the design of the bytecode. But you said, that long macros are slow, because the assembler has to process e.g. the code in an if-statement, even if this path is not used in an specific macro-call. But with bytecode, you just can put the content of it in another memory and it does not get touched, if the statement is false:

You do not know the boundaries of the IF block until you execute it, because everything in the language may be altered, including directives like ELSE or END. This is one the things I meant when I said that any such alteration would require modification of the rules of the language - in fact the ones that give fasmg so much flexibility. I went to the extreme with this language, but that was exactly my intention since the beginning. I wanted to demonstrate what would it look like if I implemented a macro language combining most if not all the capabilities that people have been requesting for fasm 1 during the years of its development.

What you proposed sounds very much like that "middle ground" assembler I mentioned earlier. At the time I finished the core features of fasmg I believed it would be possible to design and implement one, but it would be nowhere near as flexible as fasmg is.
Post 05 Oct 2017, 17:10
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland
BTW, the optimization that you proposed is possible to do simply at the level of macro definition in fasmg. If you rewrite the macro this way:

Code:
macro foo bar 
  if ( bar = 0 ) 
    _1
  else if ( bar = 1 ) 
    _2
  else 
    _3
  end if 

and then define "_1", "_2" and "_3" as regular (not unconditional) macros, the long blocks inside the skipped macro calls are not going to have any impact. Thus a modularization of macros pays off doubly in case of fasmg.
Post 05 Oct 2017, 17:29
View user's profile Send private message Visit poster's website Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
Well, thats true of course, did not think about that.
But what about allowing some kind of preprocessed files with limited capabilities. The changes, that must be done in assembl_pass are obvious, I just have to add a check for a magic-number at the beginning of the includes source-file, but where can I add macros and strucs to the current/root namespace? this does not need access to other symbols than macros/strucs, because setting internal variables/flags of the included file can simply be done with macro-calls. So in the end, there are 3 different types of macros: internal macros like ›display‹ (fast), custom macros (slow) and preprocessed macros, that do not have the full capabilities, but are good enough for most purposes, like encoding x86-instructions.
The addition of an ›format preprocessed‹ to create such files should be trivial.


Last edited by _shura on 05 Oct 2017, 17:37; edited 1 time in total
Post 05 Oct 2017, 17:32
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland

_shura wrote:
But what about allowing some kind of preprocessed files with limited capabilities. The changes, that must be done in assembl_pass are obvious, I just have to add a check for a magic-number at the beginning of the includes source-file, but where can I add macros and strucs to the current/root namespace? this does not need access to other symbols than macros/strucs, because setting internal variables/flags of the included file can simply be done with macro-calls. So in the end, there are 3 different types of macros: internal macros like ›display‹ (fast), custom macros (slow) and preprocessed macros, that do not have the full capabilities, but are good enough for most purposes, like encoding x86-instructions.

Yes, I considered something like that - macros that would "freeze" the meaning of the instructions inside and therefore could become pre-compiled. But I never could convince myself that this would be worth the effort. Might try it one day, though only in a simplified form perhaps, to avoid adding too many additional layers.
Post 05 Oct 2017, 17:36
View user's profile Send private message Visit poster's website Reply with quote
_shura



Joined: 22 May 2015
Posts: 47
Well, you just could explain more about the internals of fasmg, so I can try a first attempt.
Post 05 Oct 2017, 17:41
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland

_shura wrote:
Well, you just could explain more about the internals of fasmg, so I can try a first attempt.

That might be easier said than done. I could try answering specific questions, though.

As for pre-compiling macro contents, the easiest first try would be to replace initial symbol in macro line with "." enclosed within a context linking directly to the symbol that the name meant at the time when macro was defined. The same trick is currently used by IRPV to create a symbolic value representing a link to a hidden proxy symbol. This would require relatively small additions to current code but could nonetheless give some improvement in speed, though probably not that much.
The token that switches name recognition context is described in the design notes (I gave a link to it earlier). I later changed it so that it simply contains RecognitionContext structure after the initial byte 40h. The "current_label" field can be used to point to a pre-identified symbol and then "." token is going to point to this symbol directly, with no name lookup involved. You could try to change most of the symbols inside macro to such links, but you'd have to be careful to not touch parameters.

Recently I was working on something very different in the fasmg internals - I made a branch in the repository called "lowmem" as it has a much smaller memory footprint when using macros that do not have local parameters (like my x86 macros). The memory requirements for self-hosting became 12 MB instead of 70 MB, but unfortunately this new variant is slightly slower. I managed to bring its performance to be almost on par with the trunk, but there is still a small difference. Nonetheless I invite anyone willing to try it on other machines to check it out. It may remain more of a curiosity though, just a special variant for self-hosting on my DOS machine with little RAM. Wink
Post 05 Oct 2017, 19:38
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6606
Location: Kraków, Poland
I have an even simpler idea for the first approach to the symbol linking I mentioned above. We could tweak the ESC directive and create a variant that would identify all symbols and replace them with links and then append such line to the macro. This would be completely self-contained (would require no modifications to other parts of fasmg, just a new directive handler analogous to the "escape_directive" one). Then we could insert this directive at the start of lines in macros we would like to have "pre-compiled" and see what kind of a boost this provides.
Post 05 Oct 2017, 20:41
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.