flat assembler
Message board for the users of flat assembler.

Index > Macroinstructions > Compiler memory usage by redefined EQUs

Author
Thread Post new topic Reply to topic
Devel99



Joined: 26 Oct 2015
Posts: 30
Devel99
Does anyone know FASM compiler internals?
Should I rewind all my EQUs constants and lists in irpvs and irps? Are those EQU values memory consuming? Can I afford create lists and skip restore them back in order? Does "restore" really releases much (from EQU stack) or allocates few more?

Thank you!
Post 28 Nov 2015, 03:06
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
You can test this by simply testing code with and without "rewind"ing. Use the console version and the -m switch and see which one will assemble with the lower value.
Post 28 Nov 2015, 15:04
View user's profile Send private message Visit poster's website Reply with quote
Devel99



Joined: 26 Oct 2015
Posts: 30
Devel99
turns out that commenting out intermediate EQU unwind does not play any significant role in the FASM memory allocaton.

both compiled with m=10000 (10mb) and failed while m is 9000 due to the lack of memory. Form this point I suspect that restoring nested EQUs does not interfere with the memory involved by FASM. Case closed.

Code:
format PE console
include 'win32ax.inc'
start:

   item1 EQU LongListItem_11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
   list1 EQU

   repeat 1000
     repeat 1000
       list1 EQU list1 item1
     end repeat

     ; let's try to release compiler memory
     ;repeat 1000
     ; restore list1
     ;end repeat
   end repeat

   invoke MessageBox, HWND_DESKTOP, "test1", "test2", MB_OK
   invoke  ExitProcess, 0
.end start      
Post 28 Nov 2015, 23:42
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
You need to use REPT, not "repeat", to test this. Case still open.
Post 29 Nov 2015, 00:04
View user's profile Send private message Visit poster's website Reply with quote
Devel99



Joined: 26 Oct 2015
Posts: 30
Devel99
revolution
Thanks for reopening that. Cool I realized that in previos implementation resulting code was just a set of subsequnt EQUs, not a list

Eventually case closed. Rolling Eyes On restoring EQU definition FASM releases almost all resources associated with that EQU statement which no longer used. Please have a look:

M=1165536 (fasmW.exe UI) 8000 - out of memory
Quote:

format PE console
include 'win32ax.inc'
start:
item1 EQU LongListItem
list1 EQU

rept 7000
{
list1 EQU list1 item1
}
rept 7000
{
restore list1
}
rept 7000
{
list1 EQU list1 item1
}
rept 7000
{
restore list1
}

invoke MessageBox, HWND_DESKTOP, "test1", "test2", MB_OK
invoke ExitProcess, 0

.end start


If I comment out EQU unwind sections I'm able to cycle 5600 times each

M=1165536 (fasmW.exe UI) 5700 - out of memory
Quote:

format PE console
include 'win32ax.inc'
start:
item1 EQU LongListItem
list1 EQU

rept 5600
{
list1 EQU list1 item1
}
;rept 7000
;{
; restore list1
;}
rept 5600
{
list1 EQU list1 item1
}
;rept 7000
;{
; restore list1
;}

invoke MessageBox, HWND_DESKTOP, "test1", "test2", MB_OK
invoke ExitProcess, 0

.end start

Very Happy
so if we build a list with a long trailer we should keep in mind that it would be better to cleanup it when it is no longer needed for more efficient memory usage
Post 29 Nov 2015, 03:21
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Devel99
This is interesting, cause your results are not compatible with mine.

As for your example, I indeed get 7959 with restore and 5630 without. But if you modify the test to use non-growing stack elements, then the result will be opposite:
Code:
struc reequ [val]
{
    common
        tmp equ val 
            restore .
            . equ tmp 
        restore tmp 
}
struc equexpand [vals] { common match expanded,vals \{ . reequ expanded \} }

x equexpand x x 
x equexpand x x 
x equexpand x x 
x equexpand x x 

rept rcount/2 { list1 equ x } 
;rept rcount/2 { restore list1 }
rept rcount { list1 equ x } 
;rept rcount { restore list1 }    

In this example (with -m 1165536) I get 6066 with restore and 6068 without (maximum rcount values). The lower the size of the stack element is, the worse gets the relative behaviour of the restore .

_________________
Faith is a superposition of knowledge and fallacy
Post 29 Nov 2015, 18:46
View user's profile Send private message Reply with quote
Devel99



Joined: 26 Oct 2015
Posts: 30
Devel99
l_inc yes, I have the same on my side. Okay, maybe your sample code restores EQUs right after peak memory load ... so you don't see the gain in maximum repts available for you Wink maybe something else holds allocated EQUs preventing much memory to be actually released
Post 08 Dec 2015, 03:46
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17665
Location: In your JS exploiting you and your system
revolution
So the moral of the tests here is: Test with your own code and then decide what works best (for that code).

Don't simply assume. Don't simply read about someone else's experience and assume it will apply to your situation. Test it. Check it. Verify it.
Post 08 Dec 2015, 04:17
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Devel99
Quote:
maybe your sample code restores EQUs right after peak memory load

This can't happen, because I initially allocate only the half of the peak stack size. Uncommenting the first restore loop increases memory consumption, and uncommenting the second restore loop increases the memory consumption even more.

Quote:
maybe something else holds allocated EQUs preventing much memory to be actually released

There's no obvious reason for this "something else" to be present in my modification while not being present in your original test. For that reason I continue to insist on the point I expressed in my linked post, although in a bit relaxed way: in most cases restore'ing consumes more memory than you could gain from freeing it, but it shouldn't discourage you from using the restore to clean up after yourself. And don't build lists as growing stack elements: this makes your memory consumption quadratic. Use the whole stack as a list instead.

revolution
You're teaching bad morals. It's OK to rely on others' work to avoid reinventing bicycles with square wheels. Just pay attention, what wheels the others' bicycle has. And although I appreciate gaining some more insight from additional testing, even in this case it would be relatively safe to assume increase in memory consumption on a restore.

_________________
Faith is a superposition of knowledge and fallacy
Post 08 Dec 2015, 13:25
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.