For those that don't know the term, a Software-based RAMDisk is physical RAM memory used to behave to the OS as a Disk Drive would.

http://en.wikipedia.org/wiki/Ramdisk


Software RAMDisk currently seems to be long forgotten, specially as Solid State Disks has entered performance and mainstream market segments. SSDs are much more practical than a RAMDisk, because they're not volatile, and because you can use it as if it was a Hard Disk, including booting from it. This makes a enourmous difference, because you can enjoy SSD performance in any scenario, while on a RAMDisk you can't, as you need to boot an OS to create it before being able to use it.
However, RAMDisk stills hold a niche: They're FASTER than a SSD, and, you don't need to buy a SSD to have one. The first point is questionable: Albeit they can be faster than a SSD, real world performance isn't neccesarily reflected by this other than benchmarking. SSD low latency to access data makes it MUCH faster than a HD, and although a RAMDisk should have even lower latency and higher bandwidth, most of the notorious difference that matters in real world scenarios is already covered from going from a HD to a SSD, and specifically due to the advantage of being able to boot an OS from it. The second point, though, remains. These days RAM is dirt cheap, and considering that there are people that for playing games may spend 300 U$D on Processor and another 400 on Video Card, even spending 200 U$D on RAM brings 32 GB (8 GB * 4) to the realm of possibility. So basically, a RAMDisk would be still useful on two scenarios: When you want more performance than a SSD, or when you have tons of RAM and no SSD.


Why a RAMDisk? Because even if you had all the RAM in the world, Software isn't usually intelligent when it comes to actually USE it. This is because data preloading sucks, as usually displayed by games. For example, on several MMORPGs games that I have played, if while wandering an area, a rare monster spawns, you get some shuttering because the first time in that session that you encounter it, the game must retrieve the data for it from the HD because it is loaded on demand only. Actually, on those games I usually noticed that a rare monster was nearby because of that. The point is, that, as data preloading sucks even if you got tons of RAM, the best way to reduce loading times and shuttering is to force the game to always be onto something faster than an HD. You can archieve these results by using an SSD, but maybe you don't have an SSD, yet still have tons of RAM (Or either the game is small enough). In that case, you could create a RAMDisk, install/copy the game to it, then execute it from there, and never suffer a shuttering due to data loading again.

What is the problem? Currently, there seems to be a lack of RAMDisk applications:

http://en.wikipedia.org/wiki/List_of_RAM_drive_software


They either support very small capacities (Come on, 4 GB is simply NOT ENOUGH when World of Warcraft and Eve Online can have anywhere from 8-14 GB of data that you must copy to the RAMDisk), or are propietary/paid. I would like to know if anyone has ever tinkered with these sorts of applications.

I think ramdisks fell out of favour because they were an inefficient use of resources. Once OSes became smart enough to do caching everyone forgot about ramdisks and moved on.

For your case where you only want to make your game more fluid then a different solution might be enough. You can leverage the OS caching as a form of ramdisk by doing a dummy copy of the critical files before the game starts. In a batch file:

RAMDisk started to fell out of favour due to SSDs starting to appear in mainstream around 4-5 years ago, with its main advantage being its ease to use, higher capacities and lower cost than a RAMDisk, though they were already very hard to find even then. I still consider that a RAMDisk could still be useful on some niches, because assuming you have enough free RAM, you can always set up one for an specific use and still get better performance than a SSD without needing additional Hardware (Assuming you have that RAM).

Actually, what I'm stating about Windows is that it is far from smart enough. Even if it may be good enough for caching basic file systems operations (Like, opening a folder in Windows Explorer, and having its content displayed with no Hard Disk activity because it is cached), what I want is a bit more pretentious and it seriously lags behind for that. Windows is so dumb that it usually likes to send things down to the Pagefile even if you have enough unused RAM to hold them there, reason why I disabled Pagefile (I don't like that if I leave my machine idle for some hours, when I sit again and maximize the applications I left open, the Hard Disk spins up again until it retrieved all the data that Windows sends to it for no good reason). But even then, in this case it is not exactly Windows fault because it should be the game or application the one that attempts to preload data when is close to be used (After all, the game should know when it is going to need a piece of data better than Windows), but they don't. So a SSD/RAMDisk is a forced fix for that.


Just check what a RAMDisk does for loading times in games:

http://www.youtube.com/watch?v=7l-DNg49ZSI

0:45, 0:53. I recall it took it like 15-20 seconds, and these are sub 2 second. Seriously, RAMDisks deserve a second look. At current RAM prices they make lots of sense, they couldn't ever make innefficient use of available resources more than a current hyperbloated OS do.




Mind you explain me what exactly the "for" command does? I googled this

http://ss64.com/nt/for_r.html

Does it forces loading files in RAM or something around that? Would like more info on that.

At a command prompt, type:

So can you use an external HD like a 500GB xternal HD as RAM?

Can you estimate how fast/slow?

In the company that I'm with, we still employ RAMdisks for workstations that needs really really fast access to static data

Those workstations typically have 32 or more GB of RAM each, and they in normal circumstance, the OS / Program run smoothly under 12 GB of RAM

The remaining RAM we shove them into RAMdisk

You've got this backwards. What you're talking about is "swap space", i.e. memory pages being stored in files or entire partitions on your disk.
"RAM disks" on the other hand are parts of physical memory being reserved and configured by the OS to act as normal parts of the file system.


--8<--


RAM disks have not fallen out of favor at all. Like many other people, I still use them everyday. Personally I use one to store files that I'm sure I will not need when I reboot/power off the machine: all kinds of temporary files such as my browser profile (containing history and cache), files that I've downloaded for temporary use, logs that I redirect from some applications, etc.

Also, having source codes, log files and any intermediary files (e.g. objects) directly on the RAM can significantly speed up build-time and is physically easier on the disk.

Why not just be like normal people and invest in a good computer.

8===D---

LOLWUT?!

So basically, command "for" works pretty much to repeat another command on a multitude of files and works nice in .BAT files, that is what I get from the explanation.
The other part is "copy" to nul. It reads the file, but doesn't actually copy it. For what I googled, it IS used for caching. However, I recall that DOS commands like "copy" worked only on the directory that you used them at and it ignored subdirectories, so basically, should I have to go to each directory of a game/application and repeat using copy to null, or just on the root of the application directory?

Still, considering that at this moment I'm sitting on just 4 GB of RAM (And I only see 2.87 GB of those, because WXP 32 Bits), and what I want is to shove into them 10 GB of data, I think it ain't going to work. Not as I want it to, at least... Need more RAM for that.
Assuming that I had 16 GB of RAM, do you think than I could archieve RAMDisk-like performance just by using copy to null, or it just improves caching? Improving caching wouldn't do what I want.


@ kalambong, ManOfSteel

What OS are you guys using, what size are the RAMDisks, and what application you use to create them? I find current applications lacking. The free ones supports just small capacities of up to 4 GB, you have to pay for professional licenses if you want to go beyond that. Any recommendations?




As a Hardware enthusiast, I can say that your comment isn't even funny. Normal people doesn't even know what a SSD is, and most think that they have "good computers" just because they have the latest Intel Processor on a mediocre Motherboard with a low end Video Card.
If you read my explanation and the other two guys Post, you should know already that you need a "good computer" to begin with, unless you saw on the wild many computers with 16-32 GB of RAM to attempt to create a good sized RAMDisk on it.

By the way, want RAMDisk Benchmarks?

http://fiehnlab.ucdavis.edu/staff/kind/Collector/Benchmark/RamDisk/ramdisk-benchmarks.pdf

A SSD is usually just 2 or 3 times faster than a performance-grade HD in sequential data reading/writing, but on random access it is faster by an order of magnitude. That is where its real life performance comes from, while you have to wait "seek" times on a HD, on a SSD that is nearly instantaneous. However, a RAMDisk is 10 times faster than a SSD on both sequencial AND random. Albeit it isn't as practical as a SSD, there is nothing that can put a fight on a RAMDisk performance.

The tmpfs driver on FreeBSD. The size can be set to anything, or if left out or set to zero it will use the entire virtual memory address space.
Other drivers are also available (e.g. md). Nothing for Windows though, sorry.

That is what for does. It recurses through the directories for you. Did you try it and watch what it does?

I didn't tested the "for" command. What I did test, was to copy nul a few videos that usually produces some disk usage and delays a bit when jumping through different places of the video for the first time if I want to watch a part in particular. It was a bit smoother, like if I already did it a few times and was already cached. Still, without a proper benchmarking tool I don't know if it is really smoother or just placebo effect.

Though so, I don't think I can appropiately use that for my actual goal, and at this momeny I can't really test it for what I want it for, because I'm expecting to need much more RAM for doing so. Currently sitting at 4 GB, though I only see 2.87 GB, and what I wanted to have in a RAMDisk weights 10 GB. This means that I need both a RAM upgrade and going to a 64 Bits OS .


...I think I could skip the OS change part though. Another thing that seems to be possible to do with a RAMDisk is giving an actual use to the RAM memory that is beyond what a 32 Bits OS can normally see, if the Motherboard supports Memory Remapping (There are no mentions of PAE, so I suppose it is not related to it). You can create a RAMDisk using that extra RAM, AND even use it to host Windows Pagefile, effectively taking advantage of RAM that would otherwise be unusable.

http://www.tomshardware.com/reviews/ram-memory-upgrade,2778-3.html

I recall having said to several guys (Maybe not here) that making a RAMDisk just to have the Pagefile in it is a stupid idea because it is redundant (Why you want a Pagefile to "simulate" RAM if it will be in it?) and should add some overhead, but on THIS case on particular, it has the potential to shine because it gives you the option to use more RAM that otherwise couldn't on a 32 Bits OS.


Anyways, I don't get why RAMDisks aren't popular. They are a niche, but they are godly in it.

I permanently run a 1gig ramdisk for %TEMP% and %TMP%, firefox profile + cache, Website Watcher bookmarks - and for misc small temporary downloads and scratch files. It's persisted to disk at system shutdown, and continously backed up while Windows is running.

While Windows' file caching has improved a lot over time, I still get a clear speed benefit from doing this, and having %TEMP% on ramdisk means a little less wear on my SSD (and avoiding spinning up my velociraptor, if I had moved it there).

For special needs (like manipulating ISO files, manipulating Win7 installs with rt7 lite, building huge projects), I sometimes create temporary ramdisks of various sizes. I upgraded my machine a couple of months ago, this time going for i7-3770 with 16 gigabytes of ram. It's definitely a luxury rather than need-to-have, but a little bit of luxury is nice sometimes

It's a shame there only seems to be ramdisks available for Windows, btw, and not something like tmpfs from the *u*x world. Back in the win9x days, there was the pretty darn nice vRamDir, but I haven't stumbled upon something similar for NT.

Hmm? 32bit XP?

You do need PAE to access beyond 4gig physical, but it's something that was introduced in the PPro. It's also likely to be enabled by a 32bit XP even though it won't grant you >4GB access, since PAE is required in order for per-page NX to work.

You really should just install a 64bit OS and turn off the pagefile, though. If you have a motherboard that supports enough RAM that this kind of ramdisk usage is viable, I'd be kinda surprised if you have a CPU that isn't 64bit capable.

Yes, I'm still using the decade old dinosaur.



Regarding OS support, Windows XP SUPPORTED PAE in its original release by using /pae in the boot.ini, but this feature was dummied out in SP2. It does nothing currently:

http://en.wikipedia.org/wiki/PCI_hole#Physical_address_extension

What I'm talking about is Memory Remapping, that is also related to how much RAM you can see, but is not the same as PAE:

http://en.wikipedia.org/wiki/3_GB_barrier#Address_remapping_.28the_.22memory_hole.22.29

If I recall correctly, when Athlons 64 were still new around 7 or 8 years ago, there were people claiming that on some Motherboards you could see more RAM (Not the entire 4 GB, but much closer to it) depending on how you configure some BIOS options regarding the Memory Hole Remapping. You could make RAM whose address was being used by other devices available by sending these devices to address above 4 GB and let the Hardware manage itself the address translation part, so you only needed BIOS support for that without the OS knowing about it. I don't recall the exact details on this (The link claims you STILL need PAE), just that I heared it back in its time.
Also, I was thinking that what they did on Tom's Hardware article rely only on the Memory Remapping feature so there was no PAE involved, however, now I just paid attention the test was done using W7 32 Bits, not WXP, and I don't know if W7 supports PAE or not. If it supports PAE, then it kills my dream of being able to use more than 4 GB on 32 Bits. And talking about how it works on W7 32 Bits is redundant because anyone could just go for the 64 Bits version in the first place.



Regarding Pagefile, all the machines I had during the last 8 years had it disabled, its a trend that for me began with a humble Athlon XP with 640 MB RAM when I was experimenting with Windows tweaks. After paying attention to disk activity I figured out that Windows seems to like sending stuff to the Pagefile everytime I left the computer Idle even when it had enough RAM to not having to do so (This is why I say it is dumb), so disabling the Pagefile forces Windows to keep everything in RAM. I only enable it on purpose if I'm going to load something really heavy that goes beyond the boundary of my physical RAM. However, while that was rare, currently I'm hitting the wall on pretty much a daily basis.
I used to see 3.5 GB when I had only the Radeon 4200 IGP, until one year ago I added a pair of Radeons 5770, after that moment I started to see only 2.87 GB (This is because devices uses RAM address for some of its own memory, and where the Memory Hole thing should supposedly help). With 3.5 GB I didn't had any issues, but with just 2.87 GB I do. This is because the game that I play grows quite a bit on size after prolonged sessions, going from 800 MB on a fresh session to 1.5 GB after traveling several regions, and Windows itself seems to have a memory leak somewhere, or either spends up to 400 MB in caching. When I boot WXP I have 15 Threads with 200 MB RAM usage, but after being on for a week or more, with the same 15 Threads it sits at 600 MB, and this forcing me to either restart the game client or the entire machine every now and then. And we're talking about 2.1 GB JUST for Windows itself and the game, you also have another memory hog these days in the form of Web Browsers, plus any other application I wanted to have opened at the same time.

I'm sitting on an Athlon II X4. Yes, I could go for a 64 Bits OS and benefict from it JUST from the fact that I will be able to use the 1/4 RAM capacity that I'm currently missing. The problem is that after one decade of sitting in WXP, I don't feel confortable if I were to migrate to another OS. Most people praises W7 64 Bits, but after 10 years of using WXP I simply don't want to go though the learning process when starting to using a brand new OS JUST for the missing RAM (Reason why Tom's Hardware article was interesing when I found it yesterday, as if I could reclaim the missing RAM by making a RAMDisk on remapped memory space to put the Pagefile on it, it would make for a great workaround). Assuming I wanted to, I would be using Linux at this moment instead of W7, because Linux is much more useful for many of the other things I want to do. So the only real choice for me that I would be confortable with, would be the long-forgotten WXP 64 Bits.

However, do not take my current RAM issues as related to my RAMDisk idea. The reason why I started to think about a RAMDisk is because I COULD use some saved money to do a RAM upgrade on this machine, so I could go from my current 4 GB to 16 GB, heck, at current RAM prices even 32 GB are in my realm of possibility. The point is that assuming I do so and I switch to WXP/W7 64 Bits to see them, I doubt I will feel the computer any faster because Windows and its applications are totally dumb by default. Reason why if I want them to actually take advantage of my amazing pool of RAM, I have to force them into it, and a RAMDisk is a way to do that. As my machine is on 24/7 and I restart it once in a week or so, creating a RAMDisk and copying/installing the entire game to it everytime I restart (Hey, its only 10 GB) doesn't bothers me if that obliterates loading times.
The main issue is that there is a lack Software to make a RAMDisk in Windows that is both free and allows for big capacities, reason why if I were going to do such a RAM upgrade, I will not be able to exploit its actual potential.

This Thread made me figure out that RAMDisks are still widely used among servers or power users and that my idea was far from crazy and actually very logical, though.

Memory remapping is done by the BIOS by programming the chipset/memory controller, so it's OS independant - but since parts of your physical RAM will be moved above the 4GB mark, to use the remapped memory, the OS does have to support >4GB physical addresses - and thus PAE.

Vanilla XP was limited to 4GB physical *memory*. SP1 and onwards is limited to *first 4GB of physical address space*, even though is runs in PAE mode (for NX support). Apparently, too many 3rd-party drivers were lamely programmed, thinking that "[/i]hey, I'm a 32bit driver, I only need to bother with the 32bit PHYSICAL_ADDRESS.LowPart[/i]". I repeat: you are running your CPU in PAE mode, you're just not getting memory at >4GB physical addresses handed out.

Since I switched to 64bit XP, I haven't been running a 32bit system since (why on earth would you, on 64-bit capable hardware?). Microsoft's memory limit documentation directly states that not only does non-server (aka client) 32-bit Windows version not support more than 4GB of memory, there's also this direct quote: "X86 client versions of Windows don’t support physical memory above the 4GB mark, so they can’t access these remapped regions. Any X64 Windows or X86 Server release can."

But, since you are running in PAE mode, it's possible to address the additional memory, and that's what a few pieces of special software (like ramdisks) do. I dunno if there's, after all, some kernel support to help with this on non-server OSes, or if the software resorts to really nasty low-level manipulation of pagetables, but it can be done.

Same here, it worked 90% of the time with 1 gig of ram, and after I got 2 gigs even games worked fine. Keep in mind that there were good reasons for the XP memory manager to be aggressive - it just didn't take into mind much larger amounts of installed memory than it had been designed for. Also, things have gotten better in Vista and Win7.

There's more to it than that. There's been various kernel improvements, the scheduler is better for multicore CPUs, x64 code runs faster, 32bit large-address-aware programs can use pretty close to 4GB memory (if you've got the physical RAM ), since Vista a graphics-driver crash doesn't result in a BSOD, there's been various security enhancements, et cetera.

I wouldn't recommend that - it had notoriously poor driver support. Really, go for Win7 - it'll take you less than a month to get accustomed to it, and there's a lot of nice little details in it that makes your daily life a bit easier, once you've gotten used to some things being in different places.

Although AFAIK your limited to contiguous memory of ~2GB due to a small amount of kernel allocation in the ~middle of the virtual address range. IOW you can not allocate ~4GB in one lump of contiguous memory.

Maybe some of the memory manager functions can be used for the >4GB addresses, for example MmMapIoSpace or maybe one of the MDL functions.

IIRC there were some restraints with 32-bit windows with virtual addresses in that you only got ~as much as RAM + PageFile so it might be useful to have a pagefile in this instance otherwise you could be running out of page allocation before using all that RAM.

zir_blazer, while the newer operating systems offer more advantage there is IMHO no shame in still using XP 32-bit if it does everything you want in a timely and safe manner. So are you going to be writing your own RAMDrive software? There are still a few RAMDrive softwares around if not.

For those who are searching for a nice ramdisk drive software, try imdisk.
http://www.ltr-data.se/opencode.html/

Albeit I usually read FASM Forum, I am not an ASM programmer myself. I couldn't go beyond a Hello World, let alone doing something that is ACTUALLY useful. But I was expecting to someone else having experimented with this.



According to its FAQ, you can use it to make a RAMDisk using PAE.

http://reboot.pro/15593/

However, if I recall correctly, altough I saw it (Its on the Wikipedia list I posted early), I didn't used it because when I was reading Threads in their Forum about this matter it was hard to make it work that way.

After some more research I ended up using Gavotte Ramdisk, that as far that I readed is based on W2000 ramdrive.sys. This application doesn't have an official web page as far that I know, and the last version I found googling is from 2008. Filename was Gavotte_RAMDisk_1.0.4096.5_200811130.7z for those interesed.
Gavotte Ramdisk does pretty much exactly what I wanted, the way I wanted it to: It uses a driver to create a RAMDisk that is active as soon as Windows boots and plays nice with a Pagefile there without needing to manually set anything else after the first time. With PAE enabled in the application it does uses the RAM that Windows didn't see, so I was capable of making a 1.1 GB Ramdisk there. Actually, as it uses PAE, I suppose that the actual limit for it is 64 GB, so sooner or later I may give it a test run with more RAM. It also have a functional GUI, maybe the only thing its missing is that I didn't found an option to format the drive in NTFS instead of FAT32.

The two things I still don't know are: How much performance you are losing for using PAE (It supposedly adds overhead, but I never saw benchmarks on it. At least on Linux, PAE with 4 GB of RAM barely impacts performance: http://www.phoronix.com/scan.php?page=article&item=ubuntu_32_pae&num=5 ), and what was actually dummied out in WXP SP2 regarding PAE support. As it seems that it still works I don't know what they actually did to it. I recall seeing some screenshots from the first WXP using 4 GB of RAM via PAE with no issues, while currently you can't see it, even though PAE seems to be enabled. I suppose that it works in a hybrid fashion, not showing its PAE capabilities to regular Drivers (As they would requiere to be PAE aware, then) and applications, yet still letting some applications to bypass that if they know how to access that RAM space. So Windows is actually aware of all the RAM.

Probably all the "regular" memory functions (allocation, free space reporting, ...) were changed to only report memory from the lower-4gig-physical-addresses.

Looking at the 'awealloc' driver from ImDisk, it seems that the NT low-level MmAllocatePagesForMdl() doesn't have the lower-4gig-physical-addresses requirement - I wonder if it enforced the client-windows "only supports 4gig total" limit? The implementation seems somewhat dirty! (tries to alloc >8gig, then >6gig, then >5gig, then >4gig addresses, rather than using a "proper" API).

Also, the documentation for MmAllocatePagesForMdl() says:
...and on top of that, you'll need to manage the currently mapped 'window' to the allocated memory, complicating the ramdisk driver compared to the simplicity of 64bit address spaces.