flat assembler
Message board for the users of flat assembler.
Index
> Projects and Ideas > RAMDisk for Windows Goto page Previous 1, 2, 3 Next |
Author |
|
kalambong 23 Jan 2013, 05:55
Wonder if anyone has any new thought on this RAMdisk thing?
|
|||
23 Jan 2013, 05:55 |
|
Spool 23 Jan 2013, 07:54
[ Post removed by author. ]
Last edited by Spool on 17 Mar 2013, 04:31; edited 1 time in total |
|||
23 Jan 2013, 07:54 |
|
Spool 23 Jan 2013, 10:36
[ Post removed by author. ]
Last edited by Spool on 17 Mar 2013, 04:31; edited 1 time in total |
|||
23 Jan 2013, 10:36 |
|
revolution 23 Jan 2013, 10:39
Spool wrote:
|
|||
23 Jan 2013, 10:39 |
|
f0dder 23 Jan 2013, 14:57
revolution wrote:
Whether this is better use depends a lot on what you're doing. Read caching works fairly well (there might still be some situations where you'd like to keep data readily available in memory even though the OS would have evicted it to cache other stuff). But for write caching it's a completely different matter. First of all, an OS must flush it's write buffers relatively often to avoid data loss. Second, journaling filesystems will make sure to keep at least the FS metadata in a good state (which means small, uncachable enforced writes with write barriers). There's a lot of (write-heavyt) things that can be sped up a fair amount by being on a ramdisk. On my 16gig workstation, I've got a permanent (and disk-persisted) 1gig ramdisk (%temp% and my firefox profile), and it's quite nice being able to pull in large on-demand disks when, well, the demand is there. _________________ - carpe noctem |
|||
23 Jan 2013, 14:57 |
|
zir_blazer 26 Jan 2013, 03:51
The only thing I can say is that I have been happily using Gavotte Ramdisk during the last 4 months ago with no issues, having been able to put to use the entire 4 GB of RAM under classic 32 Bit WXP SP3 with the Pagefile-in-RAMDisk trick. Actually, with PAE support, the theoric limit should be 64 GB. However, due to still having 4 GB of RAM I couldn't use the RAMDisk for my original purpose, that was installing games there to load them from.
|
|||
26 Jan 2013, 03:51 |
|
f0dder 26 Jan 2013, 21:24
zir_blazer wrote: with the Pagefile-in-RAMDisk trick. >_< _________________ - carpe noctem |
|||
26 Jan 2013, 21:24 |
|
zir_blazer 26 Jan 2013, 22:30
f0dder wrote:
I know, and I still think than the basis of Pagefile-in-RAMDisk is stupid due to adding overhead (Maintaining a file system to put the Pagefile on it and letting the OS manage the "move from RAM to Pagefile" business, that considering that is moving from and to the same place, are just wasted redundant read and writes). ...That is assuming that you have X amount of RAM that the OS can see and use, then decide to make a RAMDisk reducing your usable RAM to put a Pagefile on it, so you would still have the same X total memory capacity but with added overhead. But you are forgetting than this is allowing me to use all my 4 GB of RAM on 32 Bits WXP, via PAE. While my WXP SP3 only sees 2.87 GB, via PAE I can make use of the other 1.13 GB left to make a RAMDisk. Putting the Pagefile there, I can actually use all that RAM that before I couldn't use as WXP doesn't directly uses it. So technically I found a niche usefulness for it: Making 32 Bits WXP put to use the unseen RAM as actual RAM via Pagefile-in-RAMDisk. I could use it for browser cache or something else instead, but if you are doing MMO powergaming (Namely, having multiple clients of a game open in simultaneous), you need tons of RAM, so going 2.87 GB to 4 made a VERY BIG difference, and I actually need much more than that. Specially now that we live on a era of Internet browsers memory leaks... http://img694.imageshack.us/img694/7324/taskmanagermr.png |
|||
26 Jan 2013, 22:30 |
|
f0dder 26 Jan 2013, 22:33
zir_blazer wrote: While my WXP SP3 only sees 2.87 GB, via PAE I can make use of the other 1.13 GB left to make a RAMDisk. Putting the Pagefile there, I can actually use all that RAM to keep applications open that before I couldn't use as WXP doesn't directly uses it. So technically I found a niche usefulness for it. Wow, that's a lot of wasted RAM - I thought normally you'd only lose some 250-500 megabyte of the total? With that much wasted RAM, it actually does make sense to create a ramdisk for the pagefile. The sane thing would still be installing a 64bit OS, though _________________ - carpe noctem |
|||
26 Jan 2013, 22:33 |
|
zir_blazer 26 Jan 2013, 23:03
f0dder wrote: Wow, that's a lot of wasted RAM - I thought normally you'd only lose some 250-500 megabyte of the total? It depends on how many expansion cards you have installed. My Motherboard has a Chipset with IGP (AMD 785G/Radeon 4200), and it also brings a Sideport (Marketing name for a RAM chip installed on the Motherboard intended to be used as exclusive RAM for the IGP). If I had the Radeon 4200 set in BIOS to only use the Sideport and no shared memory, I was able to see 3.5 GB. Actually, that is more than what most other people that I know can see. After that, I installed a Radeon 5770 Video Card, and if I recall correctly, WXP only saw 3.25 GB with it, that seems to be the common standard. With two Radeons 5770, as I got currently, the second one seems to drain a whole 512 MB from the addressable space for some reason, as I only see 2.87 GB. MultiGPU users are the guys than the 32 Bits addressing limit was capable of hurting the most. |
|||
26 Jan 2013, 23:03 |
|
baldr 27 Jan 2013, 10:06
revolution wrote: Same. Indeed, for any RAM size it would be the same for me. Much better use of resources IMO. |
|||
27 Jan 2013, 10:06 |
|
f0dder 27 Jan 2013, 13:47
baldr wrote: I second that. It seems that your opponents are missing part of the whole picture: even RAMDisk accesses will go through cache manager (unless smartass program detects that those files are on RAMDisk and use FILE_FLAG_NO_BUFFERING or memory mapping for them, though I doubt it a lot . Well, windows does have a DRIVE_RAMDISK class, I wonder if it disabled the cache for those? Could be really interesting wrt. memory-mapped files... haven't researched it, though. And at any rate, even without optimizations like that, RAMdisks can be beneficial. It depends on what you're doing, of course. But a couple of scenarios that have been nice for me: 1) ISO building. Full read speed from my HDD, targeting a ramdrive. This was especially nice while getting just the right tweaked version of Win7 for my new workstation build, I did a lot of ISO builds. Was also good for speed when doing (yet a) vmware test install, with the ISO guaranteed in memory. 2) I occasionally deal with datasets that consist of a crapload of tiny files. Extracting such a dataset to my 10k-rpm velociraptor takes ~1:30 and ~0:30 to a ramdrive - on the ramdrive, I can be 100% certain that the dataset stays in ram, and that there won't be any annoying speed hiccups while I work with it. Furthermore, once done, I don't have to delete all the files (which takes a while), I simply unmount the ramdisk. 3) %TEMP%, %TMP%, Firefox profile. Putting the FF profile on a ramdisk makes it noticably snappier - none of the temporary stutter I'd get every once in a while. Having TEMP on ramdisk doesn't give much of a speed boost for normal situations, but it does mean my main harddrive can be kept spinned down a lot of the time. Whether you get a benefit really depends on what you're doing - speedwise, the biggest boost is when dealing with (writes to) a crapload of small files. For reads to be sped up (after the first cache-cold read, of course) you need some relatively special situations. Some games can reportedly get decent speed boosts when loading (I guess they're doing FILE_FLAG_NO_BUFFERING but not being as efficient as they thought they were?), whereas for other it makes absolutely no difference (I tested with Far Cry 2 some years ago, it was the only not-super-small game I was playing that was small enough that I could fit it in a ramdisk back then ). _________________ - carpe noctem |
|||
27 Jan 2013, 13:47 |
|
sleepsleep 27 Jan 2013, 16:31
ctl3d32 wrote: For those who are searching for a nice ramdisk drive software, try imdisk. seems like a great tool! thanks! never thought about using RAM disk as building iso placeholder, thanks for the idea, f0dder |
|||
27 Jan 2013, 16:31 |
|
f0dder 27 Jan 2013, 17:40
sleepsleep wrote:
You might also want to check out SoftPerfect, it's pretty well-polished (used to be a commercial product) and has great performance, and a nice featureset (differential saves, and the ability to save every X minutes means your logoff/shutdown time won't be too bad when you use persistant disks). sleepsleep wrote: never thought about using RAM disk as building iso placeholder, thanks for the idea, f0dder There's a few situations like that where a ramdisk can be a nice speed boost, if you've got enough RAM available. It can also be nice for building stuff like boost - you'd probably need insane amounts to be able to build something like KDE in-ram ;-P baldr and revolutions remarks are valid, though - there's a lot of other situations where you won't get much performance from using a ramdisk, and will actually pessimize system performance because you have less memory available for filesystem caching. You need to do a bit of thinking and a bit of performance monitoring to figure out where it's suitable _________________ - carpe noctem |
|||
27 Jan 2013, 17:40 |
|
sleepsleep 05 Feb 2013, 08:02
i got one question regarding this ramdisk,
suppose i got a text file around 500MB that need to be processed by text utility software this 500MB located in X:/ (ramdisk) would this software read the .txt file into memory (again) since the 500MB already inside RAM. is there any "method" or "programming method" to let the software know, that is loaded into RAM already, please process it directly. of the best method to use ramdisk is for "write purpose"? so in above case, set X:/ as the output of the software? |
|||
05 Feb 2013, 08:02 |
|
revolution 05 Feb 2013, 08:22
sleepsleep wrote: would this software read the .txt file into memory (again) since the 500MB already inside RAM. sleepsleep wrote: is there any "method" or "programming method" to let the software know, that is loaded into RAM already, please process it directly. sleepsleep wrote: of the best method to use ramdisk is for "write purpose"? |
|||
05 Feb 2013, 08:22 |
|
f0dder 05 Feb 2013, 15:52
revolution wrote:
[/quote] And in some cases, it can be a decent bit faster - but it depends a lot on your access patterns. Worst case scenario with mapped files, the actual file I/O is no faster than what you can achieve manually, and you get a pagefault for each 4k you process. Best case scenario, the file is already in filesystem cache, and you can access the data directly without buffer copying (this is fast). revolution wrote:
Indeed - I sometimes see people recommend mapped files as if they are the holy grail, which they definitely aren't... but even when not the fastest solution, they can be extremely convenient and easy to work with _________________ - carpe noctem |
|||
05 Feb 2013, 15:52 |
|
baldr 05 Feb 2013, 21:06
f0dder,
Devise a conscious access pattern for a text file (except for sequential reading at least once; not as a whole at a time, probably). |
|||
05 Feb 2013, 21:06 |
|
f0dder 05 Feb 2013, 21:32
baldr wrote: Devise a conscious access pattern for a text file (except for sequential reading at least once; not as a whole at a time, probably). For standard text file reading, you'd do a one-pass sequential scan - unless the file was already cached, doing it via mmf might be slower: one #PF per 4kb, compared to large blockreads with fewer r3<>r0 switches... OTOH, you could be endline scanning while more disk I/O is being done (not sure how much readahead you get when doing MMF, but regardless of readhead I do believe you still get a #PF per page). Didn't we have a thread here a while ago about alternate ways of processing a text file, though, or am I remembering wrong? Something about multiple parallel line-scanning threads... there's definitely some CPU overhead in managing those, though, and you'd need to be careful (or on a fast SSD) to not kill any eventual performance boost by seek overhead. Not sure if it's worth it. Another approach that might not buy you performance gains while loading, but could turn out interesting for an editor, would be linescanning the MMF, and keeping each line as a {index, length} pair, keeping the MMF open in SHARE_READ for the lifetime of the editor... you'd obviously need a bit of logic (span buffers thingies?) to handle edits, and in-place saving could turn out really tricky. The upside would be that in low-memory situations, the OS being could discard memory pages (for later source file pagein) rather than pageout+pagein. Not sure if the complexity is worth it in the end, but I think it's an interesting idea at least. And I get a sense of déjà vu writing this, so I've probably babbled about the idea before _________________ - carpe noctem |
|||
05 Feb 2013, 21:32 |
|
Goto page Previous 1, 2, 3 Next < Last Thread | Next Thread > |
Forum Rules:
|
Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.
Website powered by rwasa.