flat assembler
Message board for the users of flat assembler.

flat assembler > IDE Development > Memhog (Was "Memleak BUG") in S&R editcore

Author
Thread Post new topic Reply to topic
DOS386



Joined: 08 Dec 2006
Posts: 1904
.

FASM both DOS and Win32 IDE's have problems with Search&Replace in bloated files (at least cca 1/10 of RAM capacity). An attempt to replace a frequent char ("a" for example) with a long string (20 chars are enough) results in abort in DOS and quasi-dead loop in Win32, while other editors accomplish the task well within less than 30 seconds.

PS: above link has a 2.3 MiB 7-ZIP uncompressing to 15 MiB text Shocked

EDIT : updated subject


Last edited by DOS386 on 29 Mar 2009, 04:31; edited 2 times in total
Post 15 Mar 2009, 03:05
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7283
Location: Kraków, Poland
That's a problem with the undo info storing mechanism - with doing large changes in huge files you can easily run out of memory, which gets filled with stored line segments for undo (or begins awful swapping in Win32). I suspect there might be some problem with search+replace doing a multiple copies of the same segment - I will find out, if that can be improved.
Post 15 Mar 2009, 09:13
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7283
Location: Kraków, Poland
I tried adding a quick scan if the given segment was already stored for undo recently - it reduces the memory usage with no noticeable slow down of the operation. It is be possible to even further reduce memory waste by doing full scan of the stored segments, however at the cost of quite a noticeable speed reduction.

DOS386 wrote:
while other editors accomplish the task well within less than 30 seconds.

Just tried it with PSPad and Windows Notepad - the first one was able to accomplish the task quite fast, however did drop the undo support on it (and did inform me about it with message box). As for the Notepad... enough to say I didn't see a point in waiting till it finishes. And it even didn't allow full undo. :>
DOS Navigator's editor is quite fast on this, and allows full undo (though, interestingly, this undo is quite slow) - and it's actually the one, which was my inspiration, when designing asmedit. It got better performance partially because of the limit of 256 characters per line - the limitation, which I got rid of in asmedit.
And FASMW's replace is slower than FASMD's because of constant updating of the scrollbars, and other Windows overhead. I'd perhaps be able to improve it by making a dedicated subroutine that wouldn't do any Windows call until it finishes the replacing process.
Post 15 Mar 2009, 10:25
View user's profile Send private message Visit poster's website Reply with quote
DOS386



Joined: 08 Dec 2006
Posts: 1904
Tomasz Grysztar wrote:
tried adding a quick scan if the given segment was already stored for undo recently - it reduces the memory usage with no noticeable slow down of the operation.


Very good Smile Works much better now Smile

Quote:
DOS386 wrote:
while other editors accomplish the task well within less than 30 seconds.
Just tried it with PSPad and Windows Notepad - the first one was able to accomplish the task quite fast, however did drop the undo support on it (and did inform me about it with message box). As for the Notepad... enough to say I didn't see a point in waiting till it finishes. And it even didn't allow full undo.


IIRC Notepad won't even load files > cca 60 KiB ... while cca 50 KiB ... 60 KiB will load but you can't edit them anymore Laughing "close some other applications and retry then" ... not to talk that it doesn't work in DOS, so it might be extremely hard to beat this editor Laughing

Also, with "other editors" I meant those carefully picked from a huge bunch (actually 1 or 2 at best: KINESICS and maybe INFOPAD), 99.9999% of editors simply suck heavily Sad

Quote:
DOS Navigator's editor is quite fast on this, and allows full undo (though, interestingly, this undo is quite slow)


Probably you won't revert huge S&R's frequently ...

Quote:
which was my inspiration, when designing asmedit. It got better performance partially because of the limit of 256 characters per line - the limitation, which I got rid of in asmedit.


Now it works very well (and also can be reverted) in both KINESICS and FASM ... FASM is slightly faster.

Quote:
And FASMW's replace is slower than FASMD's because of constant updating of the scrollbars, and other Windows overhead. I'd perhaps be able to improve it by making a dedicated subroutine that wouldn't do any Windows call until it finishes the replacing process.


Good idea. Drop the scrollbars and add a progress indicator instead, telling how much text processed and how many changes done, and keep it when done as summary ... in both Win32 and DOS. Idea

_________________
Bug Nr.: 12345

Title: Hello World program compiles to 100 KB !!!

Status: Closed: NOT a Bug
Post 22 Mar 2009, 02:40
View user's profile Send private message Reply with quote
DOS386



Joined: 08 Dec 2006
Posts: 1904
I wrote:

Quote:
Very good Smile Works much better now


S&R is very good now (also single confirm mode has been added), but there is a regression in search without replace: [F3] causes a PageFault (DOS only, not Win32 ?) Sad

Code:
nope
nope
    


And search for the 2 o's to reproduce

Also, a small related feature request: if [F3] is pressed while no search is defined, "redirect" to CTL-F (ask for string) rather than ignore it.
Post 26 Mar 2009, 01:41
View user's profile Send private message Reply with quote
Madis731



Joined: 25 Sep 2003
Posts: 2145
Location: Estonia
1.67.35 used to take 1min 40sec and 500MB of RAM
1.67.37 takes 1min 18sec and 460MB of RAM

The text was that 15MB LOONIX.txt and replace done to 'a' with 'TERE'

Undo works instantly though Very Happy

1) A progress-bar would be helpful as DOS386 suggested
2) The window flashing those scrollbars is awful and not very user-friendly / helpful. In addition you, Tomasz, also said they take significant resources. I hope we can do something about them.

Actually I do run into occasions where large code needs to be refactored or simply FASMW IDE is used to read some non-ASM code Smile
Post 26 Mar 2009, 07:25
View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7283
Location: Kraków, Poland
DOS386 wrote:
[F3] causes a PageFault (DOS only, not Win32 ?) Sad

Can you give me the EIP of page fault?
Post 26 Mar 2009, 08:28
View user's profile Send private message Visit poster's website Reply with quote
DOS386



Joined: 08 Dec 2006
Posts: 1904
Tomasz Grysztar wrote:
the EIP of page fault


Regrettably it seems pretty random (depends from FASM load address ?) ... partially > 2 GiB , in the shot a bit lower but still sufficiently invalid Shocked Also, in some cases some "incomplete restart" (very similar symptoms to the recently resolved BUG with invalid volume letter) occurs.

OOps revealed EIP just 4 minutes before fix Shocked Thanks Smile I'm going to test ASAP.


Description:
Filesize: 6.17 KB
Viewed: 2838 Time(s)

fasmpf.png




Last edited by DOS386 on 26 Mar 2009, 14:36; edited 4 times in total
Post 26 Mar 2009, 14:07
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7283
Location: Kraków, Poland
It's OK, I've managed to hunt down the bug already. I'm going to upload the fixed package in a few hours, when I get back to my primary machine.
Post 26 Mar 2009, 14:11
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7283
Location: Kraków, Poland
Madis731 wrote:
1.67.35 used to take 1min 40sec and 500MB of RAM
1.67.37 takes 1min 18sec and 460MB of RAM

The text was that 15MB LOONIX.txt and replace done to 'a' with 'TERE'

Undo works instantly though Very Happy

1) A progress-bar would be helpful as DOS386 suggested
2) The window flashing those scrollbars is awful and not very user-friendly / helpful. In addition you, Tomasz, also said they take significant resources. I hope we can do something about them.

Actually I do run into occasions where large code needs to be refactored or simply FASMW IDE is used to read some non-ASM code Smile


OK, please try with the updated version. I've removed all the scrollbar flashing etc. - and added a new feature to asmedit control that allowed easy external implementation of this feature.
There is no progress indication there right now, but you've got the full-speed s&r instead.
Post 26 Mar 2009, 17:08
View user's profile Send private message Visit poster's website Reply with quote
Madis731



Joined: 25 Sep 2003
Posts: 2145
Location: Estonia
Same machine, same file, same condition - 8 seconds and 450MB of RAM.

Thumbs-up Very Happy
Post 27 Mar 2009, 07:38
View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger Reply with quote
DOS386



Joined: 08 Dec 2006
Posts: 1904
Madis731 wrote:
1.67.35 used to take 1min 40sec and 500MB of RAM
1.67.37 takes 1min 18sec and 460MB of RAM
Same machine, same file, same condition - 8 seconds and 450MB of RAM.
The text was that 15MB LOONIX.txt and replace done to 'a' with 'TERE'


My test in DOS:

old: 520 MiB
new: 290 MiB

Takes 3 to 7 seconds (fastest to slowest PC I got) Smile

The new string seems very irrelevant, replacing a with TERESPOL&FlatAssembler gives very same results. Laughing

I wrote:

Very good Smile Works much better now

I was a bit overoptimistic ... but it is still "average-level-of" better now.

Of course hogging 20 times more memory for S&R undo than full text size is not ideal ... backuping the original text in plain would be better here. I don't think there is a urgent need to fix this further for now (the world is waiting for 1.68 major release and I don't want to find regressions in it's DOS IDE then), but after, there could be following possibilities:

- Tomasz's idea see above (how much would it improve ?)

- Add a S&R checkbox "No Undo"

- When out of memory in S&R: give choice to revert or continue irreversibly

- When S&R undo exceeds plain size, revert, backup plain text, re-apply without bunching Undo patches

- Do 2 passes, decide whether to use pain backup or per-hit Undo patches

Also, the [F3] bug is fixed Smile ... seems to have been a "typical" RETURN-without-GOSUB-BUG Shocked
Post 29 Mar 2009, 04:28
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2019, Tomasz Grysztar.

Powered by rwasa.