Is Windows Experience Index actualey accurate?

[oymd]

What does it exactly mean? Does it actually benchmark all the subsystems....and can it be used as an accurate reference?

I wonder what would a 10 be like??

Does this look ok?

 

[Stoly]

I think it tops at 5.9

 

[DeaconFrost]

I think it tops at 5.9

Only if you don't know how to edit an .xml file. To the OP, it is a largely worthless number.

 

[Mithent]

Correct, 5.9 is the maximum you can currently get; they might extend the scale in the future though.

No benchmarks are necessarily an accurate tool for assessing performance, and the Windows Experience Index is hardy the best of them anyway. It's just a ballpark figure which, in theory, can be printed on boxes to provide an estimate of the system requirements, because people might be able to remember their computer is a 5.3 but not understand the system specs. I've not really seen it used though.

 

[bigdogchris]

The Experience Index is used if you buy software through Microsofts online store. I think it's a good idea and they will continue to use it and tweak it in future OS. You have to admit that having a single number to base "can I run it on my computer" is a neat idea that will help a lot of people who don't know much about PC's.

I think eventually everything will move to this type of system and that are published by Games for Windows will have this number right on the front of the box. Also a good chance 3rd party software will too.

 

[Joe Average]

If you pick up a box of software, especially gaming software, made during the past 10 years or so you'll find a section on the box, usually the bottom or perhaps some other side of it, that lists all the "minimum hardware requirements" and then the "recommended hardware requirements" to be able to play said game or use said software in a way the software publisher deems "useful." The issue with that is that most people consider such "fine print" just that and rarely if ever read it. I'd go so far to say that most computer owners don't even understand the fine print in most respects, all they want to know is "can I play this game on my system?" and that's that.

The concept of WEI is to provide a numeric scale that makes it easy for the consumer to go to a store, pick up a box, look at the WEI number (let's say some child's game or education software has a WEI of 3.0) and the consumer remembers that his/her machine at home has a WEI score of 5.3, then it's simple math: 5.3 is greater than the 3 on the box, hence the game/software should run pretty damned well on his/her hardware.

In terms of using it for a "rating" of power across the variety of PCs out there, it's effectively useless as the results are kept in a simple .xml file as has already been mentioned and can be easily modified. Hell, I had a P3 750 MHz laptop last year with 512MB of RAM and an 8MB video chip on it with a 20GB hard drive and it had a WEI of 5.9, go figure.

So in terms of comparing your machine to another's performance, the WEI is effectively useless. Real benchmarks are still the order of the day for truly [H]ard computer owners. The WEI is just a way to help your average Joe (not me, I assure you), consumer that happens to own a computer, find software that will work well on his hardware without going blind trying to read the fine print, let alone comprehending it.

 

[Mithent]

The specifications on the box, even if you understand them and what you have, aren't necessarily helpful anyway. Never mind the 'minimum' specification usually meaning 'yes, it will actually run, provided you enjoy playing your games at 640x480 with the lowest settings at 5fps' (which, while technically accurate, isn't very useful); rather that they often give unhelpful descriptions of things.

It's improved a bit lately, but a lot of software says it needs a '1.6GHz CPU' and a '128MB video card'. 1.6GHz is pretty meaningless without knowing what CPU they're calibrating against (and even then, how does it map onto your CPU?), and specifying the video card memory is next to meaningless without saying what cards they support. Even then, saying that you support whole graphics card families is pretty useless, considering the difference in performance within each.

I don't know how anyone is supposed to understand these specifications, really. It would be nice if the WEI was used more to clear this up.

 

[DeaconFrost]

Has anyone actually seen a game or application with a WEI rating on the box? I can't say I have.

 

[milkweg]

The index is useful for 'Games For Windows' certified games as it gives you an idea of what index rating you need to play the game at a decent frame rate. Index is based on lowest performing tested hardware component. My dual core 3.4ghz cpu is faster than a quad core 3.0GHZ cpu in most games even though the quad will get a higher index rating.

 

[Forceman]

I question its value when games like World in Conflict have a required WEI of 1.0, with a recommended of 3.0 - what kind of resolution/frame rate is that going to support.

 

[Serpico]

It helps with marketing on the box of some programs but it really isn't a good metric compared to other benchmarking tools out there. Way too vague and non-specific, pretty useless.

 

[mikeblas]

No benchmarks are necessarily an accurate tool for assessing performance,

This is a bit of an over-generalization, as it is possible to benchmark the actual workload the machine will see in order to compare two machines. The problems happen when people use metrics from a given workload to try to forecast the performance of a different workload.

 

[Mithent]

This is a bit of an over-generalization

That's true, if you're actually benchmarking the task then that's a different matter. I should have qualified it by saying synthetic benchmarks, which might give you a vague idea of relative performance but can't say anything for sure about the differences you'll see in real applications.