flat assembler
Message board for the users of flat assembler.

Index > Heap > Window manager for Linux

Goto page Previous  1, 2, 3  Next
Author
Thread Post new topic Reply to topic
ManOfSteel



Joined: 02 Feb 2005
Posts: 1154
ManOfSteel
Azu wrote:
I doubt any amount of RPM will get you ~2ns latency..

I never said it replaced RAM, otherwise we would be plugging ATA/SCSI cables in our RAM slots. I said it mitigates thrashing.

And saying a 15,000 RPM HDD won't ease thrashing significantly compared to a 4,500 RPM one (or even a 7,200 vs. 10,000), is just like saying it would take as much time to cross the distance from point A to point B in both a glider and an SR-71.

Anyway, I've always been short on RAM and never really suffered from it.
Post 27 Aug 2009, 16:21
View user's profile Send private message Reply with quote
ManOfSteel



Joined: 02 Feb 2005
Posts: 1154
ManOfSteel
Borsuc wrote:
I disabled disk paging completely.

You can disable the swap momentarily on a *nix/*nix-like system. But it is always highly recommended to have a swap partition and to keep swapping enabled as it is used on most (all?) systems for core dumps.
Post 27 Aug 2009, 16:24
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17338
Location: In your JS exploiting you and your system
revolution
Azu wrote:
I doubt any amount of RPM will get you ~2ns latency..
I doubt any amount of RAM will get you ~2ns latency either.
Post 27 Aug 2009, 16:25
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
revolution wrote:
I doubt any amount of RAM will get you ~2ns latency either.
It's called CPU Level 1 Cache Laughing

Why would I need paging file when I even use 1GB of my RAM as a ramdrive, that is virtual drive (on RAM)? I heard some people even put their paging file on such a ramdrive, which really makes me LOL as to why they have paging enabled in the first place Laughing

I don't like paging. I noticed Windows even swaps to disk stuff when I've got plenty of RAM available. I'm not sure about *nix, but why would I enable it?

_________________
Previously known as The_Grey_Beast
Post 27 Aug 2009, 17:35
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
revolution wrote:
Azu wrote:
I doubt any amount of RPM will get you ~2ns latency..
I doubt any amount of RAM will get you ~2ns latency either.
4*
Post 28 Aug 2009, 03:21
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
ManOfSteel



Joined: 02 Feb 2005
Posts: 1154
ManOfSteel
Borsuc wrote:
I noticed Windows even swaps to disk stuff when I've got plenty of RAM available


It's evicting pages that have been inactive for some time to make more room for future processes. This may be considered as a relatively good thing.

*modern* memory managers keep as much of the data you've been using nearby in case you'll ever need it again. As this process goes on, physical memory fills up, and inactive pages are swapped out. By doing that, it in fact delayed many unnecessary, slow HDD seek+read operations as much as possible and probably also increased your HDD life-span a little bit.

And swapping of inactive memory can be done preemptively when the system is idle so that when physical memory pages are needed, they will be ready, instead of swapping at the last minute and slowing the system down suddenly.


Borsuc wrote:
why would I enable it


What about hundreds of MBs that have been allocated by your programs at some time? If you have swap on, they are (partly or totally) paged out and their pages are reclaimed for something useful. If you have swap off, you're stuck and you have no choice but to close one or more of your running programs... all because of some temporary data... even if only a portion of this data was blocking the entire process!

There might also be hundreds of MBs of leftover from programs' initialisation phase that are not needed anymore for the successful continued operation of the process. However the system cannot remove them right away (or ever) because it can't guess it's all useless now. So by design it's the memory manager that "gets rid of the trash" in due time through swapping.
Same goes for all the code that was used to boot the OS and is now completely useless. The memory manager has no way to guess that either. A hierarchy of pages is all it sees.

Also note that evicting the file-backed portions of inactive memory (i.e. modified data that hasn't been written back to disk yet) right away when free pages are needed would be much worse than paging them out (and evicting them progressively) since writing to swap is much easier on the h/w than writing to the file system because swap is similar to the very simple page hierarchy while the FS is much more complex and involves additional levels of abstraction and therefore many more seek+write operations... (plus even more seek+read from the FS if the data is ever read again, which is very likely).

The memory management process is a fine-tuned whole that cannot be modified without potentially negative consequences.
Post 28 Aug 2009, 21:27
View user's profile Send private message Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
You don't get it? Why would it swap files if over 1 month of using my new comp not once have I got the message "Virtual Memory too Low" with disabled paging? I mean if I was concerned with memory I wouldn't have used 1GB of my RAM as a ramdrive, would I?

Swapping to disk is pointless in this case. No I don't like the OS to use my harddrive for nothing, I want to decide how to use my RAM not my OS. Apart from the noise it also decreases its lifespan (which is considerable, since we're talking about daily routines on a regular interval here -- i.e swapping).

But here's more: I actually WANT the system to inform me if I run out of my 2 free GB of RAM for apps, so that I can close the unneeded apps. Most likely it would be a GOOD thing to inform me to close down apps I don't need, if I forget.

It seems stupid to enable paging for temporary storage when I've even redirected the %TEMP% environment variable (for programs that use temporary files, instead of RAM) to the 1GB ramdrive and everything went fine even with "heavy" applications.

I mean, let's be serious! With my last computer almost all my apps were running OK, and that one had a meager 512MB of RAM + 2GB of paging. Total is 2.5GB... with this one I've got 3.5GB - 1GB (ramdrive) = 2.5GB so the same as the total amount of the previous one.

Since I never actually used my whole 2.5GB virtual memory with the previous one (would be painfully slow to do so as only 512 was physical...), why the hell would I enable paging as if I need more than 2.5GB???

If anything, when I increase my RAM next time, I would probably just increase my ramdrive Razz

_________________
Previously known as The_Grey_Beast
Post 28 Aug 2009, 23:29
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
ManOfSteel wrote:
Borsuc wrote:
I noticed Windows even swaps to disk stuff when I've got plenty of RAM available


It's evicting pages that have been inactive for some time to make more room for future processes. This may be considered as a relatively good thing.

*modern* memory managers keep as much of the data you've been using nearby in case you'll ever need it again. As this process goes on, physical memory fills up, and inactive pages are swapped out. By doing that, it in fact delayed many unnecessary, slow HDD seek+read operations as much as possible and probably also increased your HDD life-span a little bit.

And swapping of inactive memory can be done preemptively when the system is idle so that when physical memory pages are needed, they will be ready, instead of swapping at the last minute and slowing the system down suddenly.
The only time this would boost performance rather than degrade it would be if you start more programs than you have the RAM for. If you had enough RAM for them, then there is simply no benefit to "preemptive swapping", since it should never swap. There could still be the option to swap if you do run out of memory due to some unforeseen problem, but doing it preemptively would be bad.

Considering that 8GB of fast RAM cost well under $100 nowadays, I'm pretty sure the most common scenario is to have enough RAM.
Post 29 Aug 2009, 04:08
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
ManOfSteel



Joined: 02 Feb 2005
Posts: 1154
ManOfSteel
Borsuc wrote:
You don't get it [...] if I was concerned with memory

YOU are the one not getting ANYTHING from my posts.
Extending RAM was the primary function of swapping 20 years ago when you paid at least 100$ for a 16MB RAM. Today it's more about optimization. You don't get it that keeping as many pages as possible, even in swap space, is better (faster and easier on the hardware) than re-reading from the filesystem, do you?
You have enough physical memory as it is now and for the past month? Fine! It doesn't mean you'll always have, all the time, under every circumstance. What if you NEED to run a video processing software and decode/convert/process a 2-hours video, or run multiple virtual machines, or whatever you do with your computer that may easily need GBs?

Borsuc wrote:
No I don't like the OS to use my harddrive for nothing

It's not for *nothing* (at least not in every case) and swapping is not solely for RAM extension or else OS programmers (from MS to the OSS community) would've removed swapping COMPLETELY from all new OSes, like 5 years ago since RAM are now at least 1 or 2 GB each. And don't tell me they don't do it to preserve backward compatibility since new OSes (at least MS's, i.e. 90%+ of the market) don't even run on older computers because of critical lack of resources.

Borsuc wrote:
I want to decide how to use my RAM not my OS

Well, if you know better than OS designers who have been working in the field for years and have been studying the evolution of OSes for the past 40 years, you're free to design your own with a revolutionary memory manager scheme. Don't forget to send us the URL to the ISOs when you're ready for Alpha 1 testing.

Borsuc wrote:
it also decreases its lifespan (which is considerable, since we're talking about daily routines on a regular interval here -- i.e swapping)

WTF? As I already explained (but you have this tendency to not pay attention) 1) it actually increases it since it's keeping as many pages as possible in a portion of the HDD that is the easiest and fastest to read back; 2) It has a very low negative impact on the HDD lifespan because it doesn't always have to read/write to swap intensively, unless you are short on physical memory; 3) There are many thing much worse than non-intensive swapping, like spin-up/down every time you power your computer up/off... or forcing the re-reading of an MP3 from the filesystem when it could've just re-read it from swap because you listened to it 18 hours ago!

Borsuc wrote:
I actually WANT the system to inform me if I run out of my 2 free GB of RAM for apps

... and swapping symptoms are easily recognizable, (e.g. HDD activity LED flashing every 2 seconds or less). Plus to be sure, you could just ASK the system if it's actually using swap and close whatever you want if you REALLY don't need them anymore. No need for a "low on resources" messagebox and/or some applications getting stuck or even the kernel panicking because there's no more room for free expansion from now on and all allocation operations are failing badly... and it's too late.
Post 29 Aug 2009, 10:21
View user's profile Send private message Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
ManOfSteel wrote:
YOU are the one not getting ANYTHING from my posts.
Extending RAM was the primary function of swapping 20 years ago when you paid at least 100$ for a 16MB RAM. Today it's more about optimization. You don't get it that keeping as many pages as possible, even in swap space, is better (faster and easier on the hardware) than re-reading from the filesystem, do you?
NO because it also has to WRITE THEM to the swap.

ManOfSteel wrote:
You have enough physical memory as it is now and for the past month? Fine! It doesn't mean you'll always have, all the time, under every circumstance. What if you NEED to run a video processing software and decode/convert/process a 2-hours video, or run multiple virtual machines, or whatever you do with your computer that may easily need GBs?
They don't put that into the cache and it would be pointless to do so since videos are too large anyway. Not in the swap file, not in RAM.

VirtualDub doesn't, for example. Not sure if there's an option to enable caching, but it would obviously BE WORSE as it would put junk on my RAM that I don't have space for anyway (videos are big, paging or no paging).

ManOfSteel wrote:
It's not for *nothing* (at least not in every case) and swapping is not solely for RAM extension or else OS programmers (from MS to the OSS community) would've removed swapping COMPLETELY from all new OSes, like 5 years ago since RAM are now at least 1 or 2 GB each. And don't tell me they don't do it to preserve backward compatibility since new OSes (at least MS's, i.e. 90%+ of the market) don't even run on older computers because of critical lack of resources.
Because people (read: average joes) don't like to get "Out of Memory", they want the computer to automagically correct their mistakes. Those people never improve, that's what's bad.

And I don't want to manually check the Task manager to see how much I use, because I never got that message -- but in CASE I get it, it's much better to be automatically informed than to check it myself.

And if I do, it's an error on my part.
(also the kernel is fully loaded in RAM and cannot get out, so no worries about the system becoming unstable -- I set up the system that way through registry)

ManOfSteel wrote:
Well, if you know better than OS designers who have been working in the field for years and have been studying the evolution of OSes for the past 40 years, you're free to design your own with a revolutionary memory manager scheme. Don't forget to send us the URL to the ISOs when you're ready for Alpha 1 testing.
Strawman argument Rolling Eyes

NO OS can do that because no goddamn OS can predict what I'm going to use, unless it's freaking hooked to my brain.

I already have it in version v3, it's called, Borsuc's brain. I'm sure you "programmed" one yourself too.

ManOfSteel wrote:
1) it actually increases it since it's keeping as many pages as possible in a portion of the HDD that is the easiest and fastest to read back;
...yeah, because the swap file automagically contains that data, it's not like it has to write it there does it? nah...

ManOfSteel wrote:
or forcing the re-reading of an MP3 from the filesystem when it could've just re-read it from swap because you listened to it 18 hours ago!
Re-read it twice is not worse than Reading it once, Writing it to Swap, Reading it again from swap.

Especially because the Write operation would happen with Swap on regardless whether I'm gonna listen to it again or not.

But oh well, a magical OS can know what I'm going to listen to. Great stuff I guess. That's why I hate Vista's "prediction" crap approach.

ManOfSteel wrote:
... and swapping symptoms are easily recognizable, (e.g. HDD activity LED flashing every 2 seconds or less). Plus to be sure, you could just ASK the system if it's actually using swap and close whatever you want if you REALLY don't need them anymore. No need for a "low on resources" messagebox and/or some applications getting stuck or even the kernel panicking because there's no more room for free expansion from now on and all allocation operations are failing badly... and it's too late.
What do you mean "too late"?

Let me clear this again. If I get the "out of memory" message it means I am doing something wrong. Probably I forgot to close applications I don't need, I dunno, something like that.

It is MY fault, and like any fault, I want to be informed, just like in debugging bugs, not have "garbage collectors" or some other shit to do it for me while the "error" is still there.

But this is beside the point, everyone recommends disabling the paging file, especially for intensive I/O apps like disk streaming audio applications or video (due to latency issues of paging, which means -- you guessed it, slower).

Not that I use those myself, but you get the point that it's slower.
Once an application closes, it closes for good, that's how I want it. I don't want its contents to be put in paging because I'm not going to use it anytime soon (well not until I shut down the computer anyway).

Sometimes I close and re-open in frequent intervals, but guess what, all those are already in RAM caching so no worries for those.

Paging would just use my HDD for nothing. Well actually no, I mean the system can figure out that I'll need this app critically in the future multiple times right? Rolling Eyes (again: WRITING to paging file, READING it again is worse than just re-reading from the FS -- so it only works if it STAYS there in paging for TWO Reads, which is HIGHLY unlikely given that it would be put in RAM after each read anyway).


Tell me something. Why would I use a ramdrive, if I use paging? Wouldn't it be wiser to have 1GB more RAM for "all those fictional scenarios" instead and have less paging usage with it...

Paging file sucks like all other "predictive mechanisms". Until they get hooked to our brains, that is.

_________________
Previously known as The_Grey_Beast
Post 29 Aug 2009, 15:28
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
Borsuc wrote:
ManOfSteel wrote:
YOU are the one not getting ANYTHING from my posts.
Extending RAM was the primary function of swapping 20 years ago when you paid at least 100$ for a 16MB RAM. Today it's more about optimization. You don't get it that keeping as many pages as possible, even in swap space, is better (faster and easier on the hardware) than re-reading from the filesystem, do you?
NO because it also has to WRITE THEM to the swap.

ManOfSteel wrote:
You have enough physical memory as it is now and for the past month? Fine! It doesn't mean you'll always have, all the time, under every circumstance. What if you NEED to run a video processing software and decode/convert/process a 2-hours video, or run multiple virtual machines, or whatever you do with your computer that may easily need GBs?
They don't put that into the cache and it would be pointless to do so since videos are too large anyway. Not in the swap file, not in RAM.

VirtualDub doesn't, for example. Not sure if there's an option to enable caching, but it would obviously BE WORSE as it would put junk on my RAM that I don't have space for anyway (videos are big, paging or no paging).

ManOfSteel wrote:
It's not for *nothing* (at least not in every case) and swapping is not solely for RAM extension or else OS programmers (from MS to the OSS community) would've removed swapping COMPLETELY from all new OSes, like 5 years ago since RAM are now at least 1 or 2 GB each. And don't tell me they don't do it to preserve backward compatibility since new OSes (at least MS's, i.e. 90%+ of the market) don't even run on older computers because of critical lack of resources.
Because people (read: average joes) don't like to get "Out of Memory", they want the computer to automagically correct their mistakes. Those people never improve, that's what's bad.

And I don't want to manually check the Task manager to see how much I use, because I never got that message -- but in CASE I get it, it's much better to be automatically informed than to check it myself.

And if I do, it's an error on my part.
(also the kernel is fully loaded in RAM and cannot get out, so no worries about the system becoming unstable -- I set up the system that way through registry)

ManOfSteel wrote:
Well, if you know better than OS designers who have been working in the field for years and have been studying the evolution of OSes for the past 40 years, you're free to design your own with a revolutionary memory manager scheme. Don't forget to send us the URL to the ISOs when you're ready for Alpha 1 testing.
Strawman argument Rolling Eyes

NO OS can do that because no goddamn OS can predict what I'm going to use, unless it's freaking hooked to my brain.

I already have it in version v3, it's called, Borsuc's brain. I'm sure you "programmed" one yourself too.

ManOfSteel wrote:
1) it actually increases it since it's keeping as many pages as possible in a portion of the HDD that is the easiest and fastest to read back;
...yeah, because the swap file automagically contains that data, it's not like it has to write it there does it? nah...

ManOfSteel wrote:
or forcing the re-reading of an MP3 from the filesystem when it could've just re-read it from swap because you listened to it 18 hours ago!
Re-read it twice is not worse than Reading it once, Writing it to Swap, Reading it again from swap.

Especially because the Write operation would happen with Swap on regardless whether I'm gonna listen to it again or not.

But oh well, a magical OS can know what I'm going to listen to. Great stuff I guess. That's why I hate Vista's "prediction" crap approach.

ManOfSteel wrote:
... and swapping symptoms are easily recognizable, (e.g. HDD activity LED flashing every 2 seconds or less). Plus to be sure, you could just ASK the system if it's actually using swap and close whatever you want if you REALLY don't need them anymore. No need for a "low on resources" messagebox and/or some applications getting stuck or even the kernel panicking because there's no more room for free expansion from now on and all allocation operations are failing badly... and it's too late.
What do you mean "too late"?

Let me clear this again. If I get the "out of memory" message it means I am doing something wrong. Probably I forgot to close applications I don't need, I dunno, something like that.

It is MY fault, and like any fault, I want to be informed, just like in debugging bugs, not have "garbage collectors" or some other shit to do it for me while the "error" is still there.

But this is beside the point, everyone recommends disabling the paging file, especially for intensive I/O apps like disk streaming audio applications or video (due to latency issues of paging, which means -- you guessed it, slower).

Not that I use those myself, but you get the point that it's slower.
Once an application closes, it closes for good, that's how I want it. I don't want its contents to be put in paging because I'm not going to use it anytime soon (well not until I shut down the computer anyway).

Sometimes I close and re-open in frequent intervals, but guess what, all those are already in RAM caching so no worries for those.

Paging would just use my HDD for nothing. Well actually no, I mean the system can figure out that I'll need this app critically in the future multiple times right? Rolling Eyes (again: WRITING to paging file, READING it again is worse than just re-reading from the FS -- so it only works if it STAYS there in paging for TWO Reads, which is HIGHLY unlikely given that it would be put in RAM after each read anyway).


Tell me something. Why would I use a ramdrive, if I use paging? Wouldn't it be wiser to have 1GB more RAM for "all those fictional scenarios" instead and have less paging usage with it...

Paging file sucks like all other "predictive mechanisms". Until they get hooked to our brains, that is.
Careful, that day might not be too far in the future.. or even in the future at all! Shocked
They're already hooking up cameras/mics to the human brain successfully..
Post 29 Aug 2009, 15:48
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Yeah but that only uses motion sensors to detect what one wishes, not what one plans to do. They don't read minds, just impulses. "planning" is long term (e.g: "I want to reopen X after 5 hours" is planning).
Post 29 Aug 2009, 16:03
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
Wouldn't that be done with pattern-analysis in software, without any kind of computer-brain interface?
Post 29 Aug 2009, 16:05
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Humans don't have patterns, or if they have, the complexity makes them really long and probably repeating slightly. Like I said, "prediction" in OSes sucks (like Vista's). No it really sucks!

It might be fine for an average joe who only uses his computer to do small amount of tasks (which repeat probably since it's the only thing he does). It sucks for the real people who USE it rather than play with it.

_________________
Previously known as The_Grey_Beast
Post 29 Aug 2009, 16:19
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
Isn't it simple statistics? Like, if you have program A open 70% of the time, and program B open 30% of the time, it would preload program A in the background (if there is enough spare RAM), followed by program B (if there is enough spare RAM), and if that RAM is needed by a running application, give it to it? Confused
Post 29 Aug 2009, 16:22
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Huh? I don't do the same things everyday!

If I don't open program A twice today because I feel like it, it's still gonna cache it because it thinks it's going to be opened. What if I do it 50% of the time, but on some days don't open it at all? How will it know which day is it?

_________________
Previously known as The_Grey_Beast
Post 29 Aug 2009, 17:31
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
True it might not be perfect, but it would be better than nothing I think o.0
Post 30 Aug 2009, 04:08
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
No it wouldn't be better than my manual (minimal mind you) control over it Razz
Post 30 Aug 2009, 14:48
View user's profile Send private message Reply with quote
Azu



Joined: 16 Dec 2008
Posts: 1159
Azu
Why not? If there's free RAM, and the drive is idle, how can it not be a good thing to load up stuff ahead of time in it? Confused
Post 30 Aug 2009, 15:00
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger MSN Messenger ICQ Number Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Using my HDD when I don't want makes random noise when I least expect and reduces its lifespan Razz

EDIT: wait, I'm talking about writing to the pagefile, not reading in advance.
Post 30 Aug 2009, 15:07
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.