What are the most popular benchmarking tools?

C

crazlunatic

n00b
Joined
Oct 13, 2007
Messages
48
Hi. I want to do my own benchmark comparison between Windows XP and Vista. What are the most popular tools to use?

Here are some I know?
3dmark
pcmark

What about calculating highest fps, average fps, minimum fps?
Thanks.
 
crazlunatic said:
What about calculating highest fps, average fps, minimum fps?
Click to expand...

That sort of varies per game. Some games have there own benchmark utilities, e.g. GTAIV, Far Cry 2. Those two you named though are pretty much the mainstream for benchmarking. 3DMark just gaming. PCMark is the overall performance of the computer in every aspect.
 
not just games, im particularly looking to benchmark the OS as a whole
btw just thought of super pi
 
sisoft sandra was a biggy last time i was really big into computers about 3-4 years ago they're still around also
 
what games do people generally use to benchmark nowadays other than crysis? would hl2 be outdated
 
SuperPi is useless in today's multi-core world; it's single threaded and while it can prove useful for simple stuff, why bother testing one core if you've got 2, 3, or 4 (heaven forbid you've got more because I'd just be too damned envious...)

wPrime is the best "Pi" calculation tool these days, multi-core aware from the gitgo.

There really is no good full overall system benchmark, there never has been in my opinion (and I've spent thousands of hours doing benches and testing over the past few years). Long ago we had Winstone which was pretty good for what it was: tons of application suites like Office (Word and Excel primarily), 3DStudio, Lotus 1-2-3, and even some high end mathematical suites like MathCAD after it first came out. Winstone would create a test environment with compartmentalized apps (sorta like "portable apps" nowadays) and run scripts of actions at max speed and then report on the results.

Then things shifted towards benching PCs with primarily 3D related stuff: Futuremark first came into being with their old stuff, back in the MadOnion days where they had fun to watch demos and benches. Futuremark tried to get an "overall system performance" benchmark with PCMark but it failed miserably; still useful to some degree but basically passed over.

Now they've got Vantage which is supposed to be a "nice" Vista "overall system performance" benching application, but in my experience of using it, the results are highly irregular and not very consistent. That could be primarily because of how Vista itself works: Superfetch will be watching stuff as it happens and tuning itself accordingly, so with each successive run of a benchmark the results can be - and realistically should be - quite inconsistent, so...

I personally figured out several things I can do on my own hardware or testing machines to show me what my hardware is really capable of, and not just playing a damned game:

- I have a directory with about 2,500 fully tagged (with cover art too) mp3 files that I "feed" to Windows Media Player 11 and let it add them to its Library. It's a CPU/RAM/hard drive task that is very intensive and has a shitload of small random reads because of pulling the metadata off the mp3 tags. XP Pro x64 consistently outperforms all other versions of Windows in my testing for this task (except Server 2003 x64 which is essentially the same OS anyway), sometimes by being twice as fast for the entire operation start to finish.

- I use MP3Gain to do an analysis pass on those same 2,500 mp3 files - this is massively CPU/RAM/hard disk intensive and has a tendency to choke the shit outta most machines. XP Pro x64 gets it done faster than the other versions of Windows I test on my test box (a Celeron 430/2GB DDR2 667 3-3-3-9/250GB 7200 rpm 16MB SATA I drive). This one takes a while to complete, but it's really damned intensive.

- I use a tiny but very cool command line version of Geekbench from years ago, 32 bit and 64 bit versions, to get a general idea of comparison. It is, again, a command line app only (console app) and does nothing but run CPU and RAM intensive testing for latency, bandwidth, and raw CPU computing performance. The newer version(s) of Geekbench suck, in my opinion, so I'll be using this command line edition for years to come probably.

- DVDDecrypter rip performance - this is just ripping the content from the actual DVD media to the hard drive, and not a measure of encoding. I use it for testing that my optical drive transfers the data adequately with the same DVD for each OS I'm testing (I use "Rounders," one of my all-time fave movies). XP Pro x64 gets the nod in this one also, but not by a wide margin - it's effectively the same across Windows in general.

- DVDShrink encoding performance - this is the actual encoding test to check out CPU/RAM/hard disk performance as a whole. The content is first analyzed (not part of the testing) and then when it's ready to roll, I do a regular convert operation from DVD9 format to DVD5 with a full analysis pass, the entire contents of the disc, and also maximum sharpness for the end result. I save the output as media files (VIDEO_TS folder, with IFO and VOB files) and that's that. The time of processing is reported when it's done so that comes in handy, no need for a stopwatch. :) XP Pro x64 once again gets the nod by roughly 10% over XP 32 bit and about 15% over Vista in general. I wish someone could get the source code for DVDShrink and make a 64 bit version... also, DVDShrink is properly multithreaded and has been for many years now. If you have a dual core or quad core CPU (or a triple core too), it'll hit every single core you have... amazing app.

And some other stuff I do. I don't think there will ever be a seriously useful all around benchmark for Windows, and with Vista and Windows 7 on the way, I prefer to do my testing in this manner: test specific applications and time them in tasks that are considered every day common things and see just how fast the box can get it done with the OS I'm testing.

Simple. :)
 
You must log in or register to reply here.
Back
Top