flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > Estimating memory required by preprocessor

Author
Thread Post new topic Reply to topic
Liebranca



Joined: 14 Dec 2021
Posts: 3
Location: Buenos Aires, Argentina
Liebranca 23 Sep 2023, 01:35
Hello,

I wrote this awful bit forever ago to conditionally include files. It works, but it uses a lot of memory, and much more for each level of recursion.

So, my question: can you estimate the memory cost of a complex chain of directives? I'm confident I can optimize this solely using intuition, but if there's any hard rules that would also help ;>

Cheers.
Post 23 Sep 2023, 01:35
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 19871
Location: In your JS exploiting you and your system
revolution 23 Sep 2023, 09:05
There are no "hard rules" that I am aware of. It is very dependant upon what the macros contain and how they are used.

If you remove blank lines, extra spaces and other pretty formatting things then you can reduce the memory usage a small amount.

Another minor thing I discovered is that using purge increased memory usage slightly, but not significantly. That is, the purge doesn't recover used memory allocations, it just unlinks the memory or marks it invalid, or similar, so the purge line itself uses the extra space in memory.
Post 23 Sep 2023, 09:05
View user's profile Send private message Visit poster's website Reply with quote
Liebranca



Joined: 14 Dec 2021
Posts: 3
Location: Buenos Aires, Argentina
Liebranca 24 Sep 2023, 15:35
I wound up fixing the problem a few hours after making this post, the issue was (for the better part) caused by nested matches, within rept, within a forward block.

That bit was needed to pop from the list of files to include, but since I rewrote most of the importer macros, I was left with shorter code and lists that took less steps to break up, so I was able to simplify the loop as well.

Now it's light as a feather. I tested by forcing fasm to assemble some files using only 256~ KB, done with no trouble. Moral of the story: it's always best to do less work ;>

On an unrelated note, I believe I was using purge a lot in another file that's been giving me problems lately, so maybe I should look into that next.
Post 24 Sep 2023, 15:35
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8266
Location: Kraków, Poland
Tomasz Grysztar 24 Sep 2023, 16:43
revolution wrote:
Another minor thing I discovered is that using purge increased memory usage slightly, but not significantly. That is, the purge doesn't recover used memory allocations, it just unlinks the memory or marks it invalid, or similar, so the purge line itself uses the extra space in memory.
A relevant and important detail of fasm 1 architecture is that it keeps the entire preprocessed source in memory, and all kinds of structures point to it, so it needs to remain available (for example, when an error is reported, the assembly-time bytecode points to its original line in the preprocessed source, and the line also has pointer to its "parent lines", etc.)

A definition of a macro is a simple structure that contains a pointer into the preprocessed source - to the text of the macro definition. So when you "purge" such macro, you actually free the structure that contains the pointer, but this recovers very little memory, while the preprocessed text that was pointed to stays where it was, as it is just a section of the complete preprocessed source text that is kept indefinitely.

This is why adding a "purge" does not decrease memory usage - because the amount of memory that is recovered is small compared to the length of the line containing "purge" command being added to preprocessed source. The header of a preprocessed line is 4 dwords already, and it's followed by the preprocessed tokens, giving another dozen bytes at the very least.

PS. To be precise, the memory recovered by "purge" is not actually freed in a general sense, it only becomes available to be re-used when you define the macro again. But here this is a less consequential detail - even if this memory was truly made available to be used for any purpose, it would still be not enough to offset the loss caused by adding another line to the preprocessed text.
Post 24 Sep 2023, 16:43
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 19871
Location: In your JS exploiting you and your system
revolution 24 Sep 2023, 18:17
I needed 10kB more memory to assemble with purge than without.

With purge:
Code:
~ cat purge.asm ; echo ; fasm purge.asm -m 73
rept 999 { purge a
macro a \{\} }
flat assembler  version 1.73.31  (73 kilobytes memory)
error: out of memory.

~ cat purge.asm ; echo ; fasm purge.asm -m 74
rept 999 { purge a
macro a \{\} }
flat assembler  version 1.73.31  (74 kilobytes memory)
1 passes, 0 bytes.    
No purge:
Code:
~ cat no_purge.asm ; echo ; fasm no_purge.asm -m 63
rept 999 { macro a \{\} }
flat assembler  version 1.73.31  (63 kilobytes memory)
error: out of memory.

~ cat no_purge.asm ; echo ; fasm no_purge.asm -m 64
rept 999 { macro a \{\} }
flat assembler  version 1.73.31  (64 kilobytes memory)
1 passes, 0 bytes.    
It isn't a big deal, only minor. But just to be aware that using purge is not a path towards reducing memory usage.
Post 24 Sep 2023, 18:17
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2023, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.