flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > [unrealistic idea]Decrease FASM memory usage

Author
Thread Post new topic Reply to topic
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 02 May 2007, 23:29
i was thinking about memory usage of FASM.

I believe it reaches peak after preprocessing, before parsing. Entire preprocessed source is stored as text (even "worse" - as tokens). After preprocessing all textual names are converted into hashes or pointers, so memory usage is decreased.

In theory, it could be possible to parse every line immediately after preprocessing it, and store only result of parsing.

Wouldn't this increase speed and decrease memory usage? (and increase complexity of sources Wink )
Post 02 May 2007, 23:29
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
Octavio



Joined: 21 Jun 2003
Posts: 366
Location: Spain
Octavio 05 May 2007, 08:56
vid wrote:
i was thinking about memory usage of FASM.

I believe it reaches peak after preprocessing, before parsing. Entire preprocessed source is stored as text (even "worse" - as tokens). After preprocessing all textual names are converted into hashes or pointers, so memory usage is decreased.

In theory, it could be possible to parse every line immediately after preprocessing it, and store only result of parsing.

Wouldn't this increase speed and decrease memory usage? (and increase complexity of sources Wink )

I think that fasm has a multipass preprocessor ,so it has to use text format to the end.
Post 05 May 2007, 08:56
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 05 May 2007, 09:44
Ocatvio: no, fasm has single-pass preprocessor
Post 05 May 2007, 09:44
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
Madis731



Joined: 25 Sep 2003
Posts: 2139
Location: Estonia
Madis731 11 May 2007, 07:37
in some cases where you have tons (weigh unit Wink ) of null-data and FASM mistakens it to be useful and take tons of RAM accordingly and "tons of time" Smile

Usually I'll give FASM more memory until in finishes, but there are other limits, like my 2Gigs or RAM + some swap Wink

Damn, I forgot the example I wanted to post. I hope you understand without.
Post 11 May 2007, 07:37
View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger Reply with quote
Octavio



Joined: 21 Jun 2003
Posts: 366
Location: Spain
Octavio 12 May 2007, 22:48
vid wrote:
Ocatvio: no, fasm has single-pass preprocessor

perhaps it needs the sources for the macros,i donĀ“t know, but memory should not be a problem even if the source is loaded, i have no notice about big assembly sources (discarding stupid sources).
>cases where you have tons (weigh unit ) of null-data and FASM
Then it is a programmer mistake.
Post 12 May 2007, 22:48
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 13 May 2007, 01:18
Octavio:
problem with memory usage pops out if you want to define headers (imports, externs, symbols, argcounts) with macros. If such macros contains conditional statements ("if"), both parts get preprocessed, and resulting text for both branches must be produced.

In practice, with lot of headers you can end up with 30-60MB of RAM used, with lot of preprocessed headers.
Post 13 May 2007, 01:18
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
MCD



Joined: 21 Aug 2004
Posts: 602
Location: Germany
MCD 29 Oct 2007, 16:12
I really think the preprocessor's memory usage should be optimized.
Symbolic names should be "deallocated" after they get restored and macros after they get purged.
I mean, you can't just decrement the memory end pointers cause multiple symbols are somewhere in the middle of the memory pool.
And even if you manage to merk some memory as undeclared, you will get bad memory fragmentation, and I guess that everyone here knows what this means.
(sloth)
And I doubt this will be easy to implement in current fasm with it's hashed name space.
Post 29 Oct 2007, 16:12
View user's profile Send private message Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4353
Location: Now
edfed 29 Oct 2007, 17:14
decreasing memory usage can be pretty good
it can permitts to write more code lines
but for the moment it's not needed, because we can compile big programs without any problem
Post 29 Oct 2007, 17:14
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 29 Oct 2007, 17:48
Quote:
but for the moment it's not needed, because we can compile big programs without any problem

not really. Depends on how macroed is your program, and how many headers you use
Post 29 Oct 2007, 17:48
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4353
Location: Now
edfed 29 Oct 2007, 22:16
if all macros are preprocessed first, can they be converted into binary?
or only text copy is good?
it's really a strnge way to code in asm
for the moment i don't have many headers
i try to implement all functions of the same class with the same header
Post 29 Oct 2007, 22:16
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 29 Oct 2007, 22:53
Quote:
if all macros are preprocessed first, can they be converted into binary?

that is how it works now:
1. source is preprocessed (macros converted, etc...)
2. source is parsed to custom binary syntax
3. that one is assembled.

problem is that memory usage is highest after preprocessing before parsing, when everything is in textual form. So my idea is to immediately convert every line to binary form after it is preprocessed.

Quote:
for the moment i don't have many headers

don't you use 'win32a.inc' or 'win32ax.inc'. because if yes, then you DO have many headers.

PS: Could you please use normal way to separate sentences, eg end sentence with "." and start next sentence with capital letter? This way your posts are quite ugly to read.
Post 29 Oct 2007, 22:53
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4353
Location: Now
edfed 29 Oct 2007, 23:29
Sorry, i am used to T9, and thinking about making a function that put capitals letters at the begin of a sentence, and ending with "." before CR.

Habitually, i never use capital letters while coding.

About fasm, if you do that, if someone do that, it will become the so wanted FASM 2.0. No? Smile
Post 29 Oct 2007, 23:29
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 29 Oct 2007, 23:58
Quote:
About fasm, if you do that, if someone do that, it will become the so wanted FASM 2.0. No?

no Smile

And I am not so sure FASM2 is so wanted... Wink

Quote:
Sorry, i am used to T9

you write to board from your mobile phone? Cool
Or is it just that strong custom?
Post 29 Oct 2007, 23:58
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri
bitRAKE 30 Oct 2007, 04:35
If the sources are not monolithic then memory usage is not an issue, imho.

format COFF
Post 30 Oct 2007, 04:35
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 30 Oct 2007, 14:21
format COFF in FASM is hard to maintain because lack of control from command line. Circumventing this via some "config file" which is included from every object is possible, but not very neat.
Post 30 Oct 2007, 14:21
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri
bitRAKE 30 Oct 2007, 15:29
vid wrote:
format COFF in FASM is hard to maintain because lack of control from command line. Circumventing this via some "config file" which is included from every object is possible, but not very neat.
It would seem as if a project editor could manage both external and internal dependancies to minimize non-neatness; but then FASM is religated to a 'back-end' - albeit a powerful one. I think this could be done with a minimal control loss (I'm not talking about a HLA type thing - more like MAKE without the headache of a MAKEFILE).
Post 30 Oct 2007, 15:29
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.