Monitoring I/O on a single file in XP.

P

Phoenix86

Supreme [H]ardness
Joined
Mar 28, 2002
Messages
6,653
Title says it all.

Specifically I want to monitor pagefile.sys for activity in my nefarious attempt to defile it's use. :)

Please recommend an app., and let me know of your experiences with it. Of course something cheap (read legally free) would be nice. Thanks in advance.
 
OK, the images come up as red Xs or "Image" in IE and FF, but I think I'm seeing where you going. I didn't know perfmon had a PF counter. So I just need to create a counter for the PF % usage?

I think this will work just fine. I want to monitor PF usage on a sytem with enough RAM that it shouldn't be paging. I think you know where I'm going with this...

Thanks Ice, you rule. Oh and where was this painfully simple solution months ago? ;)
 
Ice Czar said:
right in my little reptilian hind-brain :p
Click to expand...
OK, so learn me. I have it set to monitor % usage and %usage peak on the PF. The %usage hovers at about 34 (%usage peak is hovers about 40). However, I'm getting no drive activity, so what am I looking at?

I want the options for physical disk on just the PF. I want to measure reads and write times/sec. on just that file.

Am I missing something in those images that aren't showing? What object was it monitoring in those images? I'm assuming it was Paging File.
 
Phoenix,

I played around with this some a while back trying to figure pretty close to the same as you... how much is my PF being accessed, how much is used etc. What I think is being monitored is the percentage of PF usage (i.e. how many megs are being used in the PF, in percent) and then for PF Peak, the max percentage used. Since PF used has a max used, I don't fully know the difference in them.

edit: I think what you are looking at there is that the PF is 34% full, but not being accessed because you have enough RAM that it doesn't need to be (or nothing in it needs to be used).

I don't think that perfmon is going to show you exactly what you need, but you might be able to (as I did) couple it with Memory/ Availble MBytes, and then PhysicalDisk/ Disk Reads and Writes (there's a few more there that could help as well). In this instance, you should be able to monitor when RAM hits a certain point, the PF should pick up.

Examples: I just started up all the counters I just mentioned and then flogged my PC by opening up 48 2MB images in Photoshop all at once. Not an easy task for PS to do. On the monitor, I can see RAM fall until the point where I am out of excess RAM to put pics in. Then my PF usage picks up at the same point (maybe a few ticks later, but close enough). I close PS, my RAM goes back to where it started and my PF is... not returning to it's orignal state.

However, by also monitoring disk read/writes, I can compare that to PF usage and see that when RAM ran out, disk usage jumped (I was also opening files at the same time).

In short, I don't think it'll show you disk i/o's with the one counter, you're going to have to put a few in and take the graph as a whole.

edit2: I also think that to get this saved out and have a lot of meaningful data to look at, you have to use the Logs (on the left side), set it up, and let it run a while. Though I can't imagine how big that log would be.
 
arkamw said:
Phoenix,

I played around with this some a while back trying to figure pretty close to the same as you... how much is my PF being accessed, how much is used etc. What I think is being monitored is the percentage of PF usage (i.e. how many megs are being used in the PF, in percent) and then for PF Peak, the max percentage used. Since PF used has a max used, I don't fully know the difference in them.

edit: I think what you are looking at there is that the PF is 34% full, but not being accessed because you have enough RAM that it doesn't need to be (or nothing in it needs to be used).

I don't think that perfmon is going to show you exactly what you need, but you might be able to (as I did) couple it with Memory/ Availble MBytes, and then PhysicalDisk/ Disk Reads and Writes (there's a few more there that could help as well). In this instance, you should be able to monitor when RAM hits a certain point, the PF should pick up.

Examples: I just started up all the counters I just mentioned and then flogged my PC by opening up 48 2MB images in Photoshop all at once. Not an easy task for PS to do. On the monitor, I can see RAM fall until the point where I am out of excess RAM to put pics in. Then my PF usage picks up at the same point (maybe a few ticks later, but close enough). I close PS, my RAM goes back to where it started and my PF is... not returning to it's orignal state.

However, by also monitoring disk read/writes, I can compare that to PF usage and see that when RAM ran out, disk usage jumped (I was also opening files at the same time).

In short, I don't think it'll show you disk i/o's with the one counter, you're going to have to put a few in and take the graph as a whole.

edit2: I also think that to get this saved out and have a lot of meaningful data to look at, you have to use the Logs (on the left side), set it up, and let it run a while. Though I can't imagine how big that log would be.
Click to expand...
OK, % usage of that file's max makes sense. The machine I'm on has a GB of RAM, and shouldn't be adding stuff to the PF at all. However, if it's at 35% used, why is anything there. Anyways, this is kinda the point to what I'm testing. Why/when does data get spun to the PF, and is that benefitial. Hmm, there is that little "explain" button that says "The amount of the Page File instance in use in percent." So that makes sense, it's the amt. of the PF that's filled vs. it's total size.

That combination of counters seems to get me close enough. I would like to show raw read/writes to the file, but really measuring the flow of data from RAM to PF should do. Thanks.
 
well the total I\O is a little deceptive
as Windows backs up data that is actually in memory to the pagefile as well
so that in the event you need alot of RAM its already in the pagefile

try monitoring hard page faults instead which gives you a better idea of how often its actually being hit for a given process\ap\usage pattern
instead of just the "in case" senerios
 
Ice Czar said:
well the total I\O is a little deceptive
as Windows backs up data that is actually in memory to the pagefile as well
so that in the event you need alot of RAM its already in the pagefile

try monitoring hard page faults instead which gives you a better idea of how often its actually being hit for a given process\ap\usage pattern
instead of just the "in case" senerios
Click to expand...
I don't think HDD faults will help me monitor when I'm loading data from the HDD, which is exactlly what I'm going to end up testing. I want to see how much the PF gets touched when it's loading lots of data from the HDD. So my HDD faults will naturally increase during testing, how can I tell how many of those faults are PF vs. other access? That's why I wanted to monitor that one file vs. the whole HDD.

That first statement is exactly what I want to prevent. I'm operating under that assumption that there is sufficient RAM for all tasks, which windows can't know.
 
From Wikkipedia:

Page fault:

An interrupt that occurs when a program requests data that is not currently in real memory. The interrupt triggers the operating system to fetch the data from a virtual memory and load it into RAM.

An invalid page fault or page fault error occurs when the operating system cannot find the data in virtual memory. This usually happens when the virtual memory area, or the table that maps virtual addresses to real addresses, becomes corrupt.
---
It's also under memory and not a bad counter to throw in. It's interesting to note that when page faults jump, so does disk write??
 
page hard faults not HDD faults
http://www.wilsonmar.com/1perfmon.htm
http://www.microsoft.com/resources/...2003/all/deployguide/en-us/counters1_paaz.asp
Page Faults/sec Shows the average number of pages faulted per second, which is equal to the number of page fault operations because only one page is faulted in each fault operation. This counter includes both hard faults (those that require disk access) and soft faults (where the faulted page is found elsewhere in physical memory). Most processors can handle large numbers of soft faults without significant consequence. However, hard faults can cause delaysbecause they require disk access..
Click to expand...




also review
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/optimize/wperfch7.mspx
http://www.sysinternals.com/ntw2k/source/cacheset.shtml
 
again from the second link above

Page Reads/sec Shows the rate, in incidents per second, at which the disk was read to resolve hard page faults. This counter shows numbers of read operations, without regard to the number of pages retrieved in each operation. Hard page faults occur when a process references a page in virtual memory that must be retrieved from disk because it is not in its working set or elsewhere in physical memory. This counter is a primary indicator for the kinds of faults that cause system-wide delays. It includes read operations to satisfy faults in the file system cache (usually requested by applications) and in noncached mapped memory files. Compare the value of Page Reads/sec to the value of Pages Input/sec to find an average of how many pages were read during each read operation.

Page Writes/sec Shows the rate, in incidents per second, at which pages were written to disk to free up space in physical memory. Pages are written to disk only if they are changed while in physical memory, so they are likely to hold data, not code. This counter shows write operations, without regard to the number of pages written in each operation.
Click to expand...
 
OK, hard page faults will work, but there's no counter for it under memory, just page faults/sec which is both hard and soft faults.

I'm going through the links now... Thanks.

It's interesting to note that when page faults jump, so does disk write??
Click to expand...
That makes sense. When your paging faults increase the system will page some data from RAM to disk and "shuffle" what's paged where to make room in RAM. Again, this is only needed if you need more RAM, an assumption windows is making, and something I want to limit. How much RAM do you have an how much memory are you using, ala commit charge total/limit in task manager?

In fact that's the #1 claim of running sans-PF, decreased I/O. When, where and how much is what I'm trying to monitor so it can be evaluated with real terms.

Heck, monitoring total disk I/O would be a great way to show this. If I'm running XYZ tasks and can show reduced total I/O w/o a PF that would lend credit to "it just seems faster" claims which are never prooven.

Good stuff guys, thanks again. :)
 
you want just pages/sec (incidents of Hard Faults)
subsets are from disk\sec and to disk\sec

whereas Transition Faults/sec are non-disk soft faults

again from the 2nd link

Pages/sec Shows the rate, in incidents per second, at which pages were read from or written to disk to resolve hard page faults. This counter is a primary indicator for the kinds of faults that cause system-wide delays. It is the sum of Pages Input/sec and Pages Output/sec. It is counted in numbers of pages, so it can be directly compared to other counts of pages such as Page Faults/sec. It includes pages retrieved to satisfy faults in the file system cache (usually requested by applications) and noncached mapped memory files.
Click to expand...
 
Phoenix86 said:
Oh and where was this painfully simple solution months ago? ;)
Click to expand...

No shit...that could have saved us a hell of a lot of trouble. Anyhoo, if we can agree on a method, I'll be a tester.
 
:cool:

Awesome, I think I have all the counters I need to measure what I want.
 
Interesting... I have a system setup with 1GB RAM, and a static 400-400MB PF. My memory usage has peaked ~500MB RAM, and my PF is fairly active at times. Below we see pages/sec (red) going through the roof (scale is 0-100), with memory usage WELL below total RAM. This is exactly what I wanted to demonstrate.

PF useage 1GB RAM-static 400-400 PF.JPG


I wonder why my total I/O isn't correlating with the increased faults...
 
You must log in or register to reply here.
Back
Top