Virtual Memory ... worth to modify ?

[Cov]

As you may know, our HDDs (SATA/150) have transfer speeds of approx 118 MB/s.

My DDR2-800 Memory is significant faster ...

When I check under ...

Start › system › advanced system settings › performance › settings › advance › virtual memory › change

... how much virtual memory has been assigned in my system, I get the following info:

• By default, “Automatically manage paging file size for all drives” setting is selected so that Windows Vista system can manage the paging file without user’s interruption.
• If you want change the paging file size, move the pagefile.sys to another drive, or disable virtual memory paging, uncheck the check box of Automatically manage paging file size for all drives.
• Select and highlight the appropriate drive that users want to change the paging file settings under the box of “Drive [Volume Label]“.

• To fix a permanent size for the paging file, click on Custom size button, and set the values of Initial Size (MB) and Maximum Size (MB).
• Both values can be of same size to make the size of pagefile.sys static.
• Then click on Set button.

• System managed size is useful for users who do not want pagefile.sys to locate on system drive root and want to move its storage location to another drive, but still want Vista to dynamically manage and allocate the drive space for the paging usage.
• To move the paging file to another drive, click on No paging file radio button, and click on Set button. Then select a different drive to set a new pagefile.sys with custom size or system managed size.
• To disable paging file or virtual memory, simply set no paging files on all drives.

At the moment I have 4 GB physical RAM (Vista x86) and I'm aware that the more RAM you have, the more Window assigns as paging file.

If we consider about 4 to 5 GB memory to be reserved for the swap file and I had 8 GB RAM (Vista x64), couldn't I make Windows use my RAM instead of the HDD ?

According to Everest are only about 25% of my RAM used after Vista has booted up.

3.25 GB - 25% would mean up to 2.5 GB unused RAM

Has anyone tried this out yet ?

-----------------------------------------------------------------------------------------------------------------------

Edit: Was just thinking about Vista's Ready Boost but that is limited to max 4 GB additional virtual memory from a USB device.
Then I thought about using the whole of my 8 GB USB stick instead of the HDD, but looking at the transfer speed it seems as a no-brainer:

USB 2.0 has a raw data rate at 480Mbps = 60 MB/s.
Besides this, doesn't flash memory "wear out" ?

 

[Joe Average]

Put a 1GB static page file (1GB min/1GB max) on each individual physical hard drive you have, done. There really is nothing else you can do (short of putting it on a RAMdisk which is possible nowadays too) to improve performance as much as this one concept will - at least from altering just a system variable...

 

[alpha1471]

What's the value in putting the pagefile in a RAMdisk? It seems counter-productive to me. You're just using the RAM for the same purpose, but instead of being used directly you're adding pagefault/lookup overhead to memory accesses. In fact, I think you're wasting memory because the OS needs to free up a frame in your physical memory and copy the frame from the pagefile to that empty frame and do a bunch of page table management for what it just swapped out.

My advice is to let the system manage it if you have more than enough disk space available. Any tweaking done on your part will not provide any substantial performance increases in today's systems (4GB+ RAM). If you're short on disk space and you know your overhead for physical memory is higher than your peak usage, you can get away with disabling it.

The large pagefile size (1.5x) is an obsolete rule for legacy systems with lower RAM and for temporary storage in the event of a complete memory dump, which needs to be at least as large as the amount of physical memory available plus some overhead. This can be disabled in Startup and Recovery settings and you can reduce your pagefile size if you think it's too large.

Your 4 GB of RAM seems like enough overhead to me, but I don't know the details about your usage. For stability's sake if you need to leave your computer on for a long time, you're better off leaving the pagefile system managed or reduce the initial size and allow it to expand a decent amount. If you restart daily, then you can probably get away with reducing or disabling it.

Parallel swapping across multiple drives will increase swapping performance. Does Windows support this? It's the same concept as a RAID 0 configuration, but with less realized performance gains if you rarely need to swap in the first place.

 

[Joe Average]

It's not "parallel swapping" and it doesn't work like RAID 0, that's not the point of it. The point of having multiple static pagefiles is twofold:

1) Windows won't need to resize them
2) Windows can always access one of 'em (given the number of tasks)

Multitasking improves when it can access one of 'em at any given time because no matter how fast hard drives or even SSD hardware gets, they still can only do one thing at a time: read or write. Having multiple page files (almost) always gives Windows the opportunity to do two or more things at the same time with respect to reading/writing data to/from page files.

With one hard drive and one page file, especially the default one on the system partition/drive, if you're loading something and something else needs to be pushed out of RAM for paging, the OS can't read the data in and write the old data out at the same moment, so you just cut efficiency 100% because it now takes twice as long to do something - it reads, then it writes... or it writes, then it reads.

Add a second hard drive (note this doesn't work with multiple partitions on the same hard drive - you have to have separate physical drives) and you can read the data in while writing data out at the same time, and multitasking performance skyrockets to twice what it was previously.

As for the "why put a page file in RAM," I could offer a lot of technical reasons but I've done that a few hundred times around here. Search is your friend, if you can get that bitch to co-operate... aside from that, you'd just have to get some RAMdisk software that supports placing the page file in RAM and test it out yourself. You'd most likely be very pleasantly surprised at just what a difference it can make in system responsiveness.

 

[pallesen]

If I have one fast and one slow harddrive, do I want to put a small static 1GB pagefile on both of them, and force Windows to use the slow drive when I've filled up the other?

If you create the ram disk out of ram that the OS can use, you won't gain from it, you'll lose from it.

 

[alpha1471]

Point taken on preventing Windows from resizing the pagefile. Although the cost of resizing is amortized, I can see why some people wouldn't want that. It's similar to how some programmers don't like hash maps being resized and allocate more memory to it at initialization. It all depends on your foresight on usage, whether it will be resized at all. With sufficient memory the resizing will not occur anyway.

I'm confused by your response about parallel swapping. What you just described in point #2 is parallelism, but you say it's not parallelism and call it multitasking instead.

Linux can assign different priorities to swap partitions and swap in parallel (multitask in your terminology) if the priorities are the same. I was wondering if the Windows kernel supports this. If not, then splitting the pagefile across multiple drives will see no benefit. Again, if you have sufficient memory, this scenario will not occur very often and there will be little to no benefit regardless.

If you create the ram disk out of ram that the OS can use, you won't gain from it, you'll lose from it.

QFT.

 

[Joe Average]

If you had said just "parallel" I wouldn't have said a word, but in the same sentence you mention "RAID 0" so... while I got the gist of it, and others probably did as well, the fact that you even compared it with RAID 0 is where it just went way over the line...

I know what I know, and others know what they know. My experience shows tangible provable and consistently better performance using the tactics and techniques I've come to learn over the years with respect to RAMdisks, page files, and system performance. That's all I can say about it, and my advice or opinions are based on that experience.

If I say a specific thing will help a system, it's because I can do it myself and note a benefit from it, or I have done so in the past and had very happy clients because of it.

YMMV, as always.

 

[alpha1471]

Ah, glad to hear the misunderstanding was cleared up.

Also, I'm not completely dismissing RAMdisks. One performance improving application I can see for them is reducing compulsory misses during load operations with unpopulated TLBs, which the OS cannot predict but you can.

Placing pagefiles in RAMdisks however... by just following the theory it should not have any performance benefit. I'd be very skeptical about any benchmarks that say it does. If there's a sufficient sample size of tests done with a reasonable confidence level that show a performance increase from it, then I'll accept it.

I apologize if I'm being rude, but I'm more interested in the theory and understanding the architecture when it comes to computers, and this very idea conflicts with my understanding. Please forgive me.

 

[Joe Average]

No apologies needed, I've been having to explain things from my experience for decades. If I was there at your PC and installed a RAMdisk, in 20 mins your machine would scream like never before... but alas, I'm not.

 

[pallesen]

Accessing the ram directly will always be faster than accessing it through a pagefile located on a ram disk. That cannot be questioned.

If you create a ram disk out of usable ram, you leave less ram to be directly accessed, and you will force the OS to page out large amount of data sooner than would otherwise be necessary.

Now, if you install more than 4GB in a 32bit xp/vista machine, and create the ram disk out of the ram above 4G, the story changes, because then you won't take away usable ram from the OS.

 

[Joe Average]

Accessing the ram directly will always be faster than accessing it through a pagefile located on a ram disk. That cannot be questioned.

If you create a ram disk out of usable ram, you leave less ram to be directly accessed, and you will force the OS to page out large amount of data sooner than would otherwise be necessary.

Now, if you install more than 4GB in a 32bit xp/vista machine, and create the ram disk out of the ram above 4G, the story changes, because then you won't take away usable ram from the OS.

In principle what you're saying holds water; in reality, it doesn't work that way, hence me never backing down on it.

As for "accessing the ram directly" we're talking about nanoseconds here, not milliseconds. A few nanoseconds difference between direct access to the RAM or going through a driver to work with the RAMdisk is absolutely irrelevant in the big picture.

While I'm all for eeking out every last possible ounce of performance where technically one thing is faster than another, even I wouldn't give a shit about 1 nanosecond compared to 5...

There's a point where you have to say "Fuck it, it's fast enough" hence my eternal question...

"How fast is fast?"

Because what you think is fast probably looks pretty slow to me.

 

[alpha1471]

Now, if you install more than 4GB in a 32bit xp/vista machine, and create the ram disk out of the ram above 4G, the story changes, because then you won't take away usable ram from the OS.

Can 32 bit XP/Vista even address that extra memory? I was under the impression that it couldn't without PAE support, unless you're talking about using a separate hardware device like an iRAM.

 

[Joe Average]

SuperSpeed.com's RamDisk Plus 9 will allow you to actually have access to the >3.25GB of RAM in a machine with 4GB or more, but only for use with a RAMdisk. While that does mean you can install some apps, especially ones designed for true portability (PortableApps.com style programs, etc), it's not "officially" useful space meaning it's not there for applications to use as system RAM (meaning an application can't "see" it as system memory).

It's temp storage, at best, but very useful for some items like temp variables, browser caches, etc - all the things that make using a RAMdisk so valuable in the first place.

 

[pallesen]

hence me never backing down on it.

And you won't back off your theory that the cpu requires a pagefile because it operates with virutal memory and paging. Do you see my problem?

As for "accessing the ram directly" we're talking about nanoseconds here, not milliseconds.

No matter how long it takes, you are still taking away usable ram from the OS (in case 1).

 

[Joe Average]

Using RAM to increase system performance = more than acceptable to me and most anyone else, especially if you've got a lot of it.

Only reason I don't have a RAMdisk currently is SuperSpeed.com hasn't released an updated version that's Windows 7 compatible just yet, probably won't till it's RTM (even though I keep hounding them to let me be a beta tester...) so...

You do realize this is the [H]ardForum, right? Where people tend to do things that pull out every last drop of performance from a system? Even in spite of some folks thinking it's a waste of time even when experience is contrary to that? Hrmmm...

This is another beating the dead horse thing happening.

 

[pallesen]

Dead horse? I think the core issue is understanding just exactly what is going on, how and when.

 

[alpha1471]

deleted

 

[Joe Average]

Windows operating systems page to the page file - when one exists; if not, a temp one will be created on the fly - regardless of the amount of RAM in a system or how much is in actual use and reserved at any given moment in time. That's just how it works, how it always has worked, how the architecture the OS runs on works and always has, and until we get that architectural change, we can't base a new type of OS on it.

We're still talking about two different things here yet they keep trying to be considered one and the same thing. Processors page, period, I'm not going to argue that because it's an architectural fact. Since the processors work in conjunction with the OS, and information is paged out of kernel memory and mapped to physical RAM, virtual memory paging happens, and there's gotta be someplace for that to be stored, hence the hard drive and the page file is where it traditionally is stored.

While we can now put page files on RAM (which doesn't matter since the OS is going to page regardless of the amount of RAM installed or how much is actively in use) which helps performance because - tada - the OS will page data as required by the processor when necessary.

And... while I rarely if ever recommend wiki pages, there is a nicely structured and accurate one at:

http://en.wikipedia.org/wiki/Intel_80386

which leads to:

http://en.wikipedia.org/wiki/Paging

which finishes up with:

http://en.wikipedia.org/wiki/Virtual_memory

I'm done.

 

[pallesen]

I'm not going to continue this again, it is getting tired, but I will say it again: you have a totally wrong picture in your head of what virtual memory and paging is when you are talking about cpu architecture. This has nothing to do with the pagefile.

 

[alpha1471]

Are you confusing the page table with the page file?

 

[jimmyb]

We're still talking about two different things here yet they keep trying to be considered one and the same thing. Processors page, period, I'm not going to argue that because it's an architectural fact. Since the processors work in conjunction with the OS, and information is paged out of kernel memory and mapped to physical RAM, virtual memory paging happens, and there's gotta be someplace for that to be stored, hence the hard drive and the page file is where it traditionally is stored.

I thought we were going to have to wait another 4 months before having this conversation again...

Processors don't swap in and out pages themselves. When a page fault is encountered an interrupt handler is run. What that interrupt handler does is up to the programmer.

The page file, memory pages, paging, and memory virtualization are all related, yet orthogonal concepts.

As the second sentence of the article you linked to points out, paging is specific to an OS implementation, not the CPU. Keep in mind that it isn't even necessary to have an OS to use a processor, so to suggest that a page file is somehow necessitated by the processor architecture is absurd. Also consider that you can setup completely functional Unix systems with no swap partition.

I've even seen evidence to suggest that even in Windows you can turn off the page file completely, you disputed this, but presented no evidence.

It strikes me as completely counter-intuitive that creating a RAMdisk for the page file would do anything but hurt performance, but I have no experience with this.

 

[pallesen]

You can indeed turn off the paging file 100%. And no, Windows will not just enable it again, which is why Windows/applications crash in funny ways, if all memory gets eaten up.

Mark Russinovich, if you trust him as a source, writes this in his Windows internals books. There are also many others trustworthy sources that say the same. And if you investigate your own system, you'll see that it is the truth.

 

[Jon55]

Why would you turn it off? For performance reasons? Because that's the ONLY reason I can see understandable (space is not a good reason). If so, let's see some benchmarks/proof that says turning off paging file is worth it.

 

[pallesen]

#23 Right now, the debate is how the stuff works. Not if one would actually run their system with no paging file.

 

[Cov]

My window Vista Ultimate 64 bit with 4 gigs of RAM runs perfect without paging files.
I run 3ds Max 9, and Adobe Master Collection as my primary software to perform my work.
Games Microsoft Flight X, and spiderman 3.
I just concluded that with 4 gig why have a need of paging files, and my programs at least seem to open faster.

Read this for specifics: http://support.microsoft.com/kb/889654

Important line:
"When no page file exists, or the page file is insufficient, the system reserves some virtual memory in the physical memory for its paging needs."

In translation, if you don't have page file paging memory will be still used in physical memory which is surely faster then on HDD.
And due to fact that paging is done even if there is a LOT of free memory there are some gains when working without PF.

Which applications need PF to work I really don't know and as a programmer I would really like to see the problem which can't be solved without using PF.

Anyway, I work with 4GB XPx64 and without PF.
I have one Linux VM (with 1GB reserved) working almost all time except when playing games.
And I tend to play high load games like Crysis or COD4 in 1680×1050 on ultra high settings…

Disabling PF gives no visible performance increase generally, for example in some rare occasions you will spare your self waiting to copy large file when Windows suddenly decides to write large chunk of data to PF.
Or after long inactivity without you will not need to wait for windows to restore PF data from disk as it will do instantly from memory.

The most important reason I don't use PF is because my HDD is a LOT LOT quieter without it.

Anyway gains and risks for disabling PF are minor, so I recommend disabling it and if after 1month (which is generally the time to populate your system with mostly used applications) you don't experience any problems leave it disabled. In this period save often

Regards..

Another consideration is security.
With paging file, Windows can potentially save your files to disk.
These files on HDD can be recoverable. Without paging files, anything in memory will basically not be "logged".
That is why for some security programs, they also recommend turning paging off to prevent having a copy of your sensitive document/data to be saved on a HDD.

Virtual memory is FAR slower than physical memory.
If something gets move into the virtual memory, and you have to use it, you'll notice a delay in accessing whatever application got moved.
Windows will move things to virtual memory (WHETHER OR NOT YOU HAVE PLENTY OF PHYSICAL MEMORY FREE) simply because it hasn't been active for X amount of time.

Virtual Memory is a great safe guard against memory leaks, however, Windows' behavior on moving application to virtual memory even before the physical memory is reaching capacity is really annoying and a performance hit.

If you are one of the few that have frequent FBI visits or do REALLY REALLY private stuff on your computer, you can set the paging file to empty out during shutdown, however, this significantly increases shutdown time and isn't recommended.
It's also easily possible that the FBI would be able to access it even after it has been deleted (even after a secure "shred-style" delete), so it is likely not worth the trouble.

ect ect ...

 

[pallesen]

Read this for specifics: http://support.microsoft.com/kb/889654

Important line:
"When no page file exists, or the page file is insufficient, the system reserves some virtual memory in the physical memory for its paging needs."

The writer of this kb might have gotten something wrong when he wrote it. That section was removed in a later revision.