Brent_Justice
Moderator
- Joined
- Apr 17, 2000
- Messages
- 17,755
Hey everyone, sorry for the lack of response recently, both Kyle and I were in Toronto these past few days.
We have seen that there is a general consensus about our current reviewing method, and that is that people do not like the fact we are benching each card at different resolutions and quality settings and showing it in a graph. We do appreciate everyone's feedback and are striving to improve the method. We also encourage everyone to sit back for a second and try to understand what we are trying to show.
We put this warning on Page 4 of the Preview: http://www.hardocp.com/article.html?art=NjA2LDQ=
Now, if that isn't clear enough in explaining what we are doing then I will try to clear that up here.
Basically what it comes down to is trying to figure out which video card offers a better gaming experiencing. A better gaming experience is achieved with delivering the game the way the content developer meant the game to be seen and with the highest image quality possible. The higher the image quality, the more immersive the experience. Performance is also part of that equation. What we strive to do is find the balance on each card that provides the best image quality with playable performance. Then we look at the consistency in performance is achieved on each card at those settings. Then we compare.
What you are looking for in the graph is to see which card can play the games at the highest resolution with the highest quality settings with playable performance.
When looking at the graphs FIRST look at the top of the graph where it shows the Resolution and AA and AF settings achieved with each card. THEN look at the performance in the graph below. Then you can say for example: "Oh, ok, Card A can play at a higher resolution, a higher aa and af quality than Card B and sustain the same amount of performance with Card B at a lower resolution and a lower aa and af setting".
You also must understand different levels of performance are acceptable with different games. Some games 30-40fps is acceptable, some games 60fps or more are the acceptable framerates.
Now, of course we strive to improve the method, it is a constant process. We understand that some people really see value in "apples to apples" resolution and quality settings. One thing you must understand is that you cannot say NVIDIA's AA and AF and ATI's AA and AF are "apples to apples" to begin with. Comparing NVIDIA's 2XAA, or 4XAA or 8XAA, or 8XAF or 16XAF are really not directly comparable because different methods are being used, they are close, but not exactly the same. About the only thing that is directly comparable is resolution.
But we understand that though that fact exists there are still people who insist on seeing the cards compared at the same resolution, AA and AF settings. Ultimately the optimal solution is to show both ways in each review. The problem with that is of course time. Benching 10 games, both ways, on 3 different cards, which perhaps 2 more graphs per game, is a time consuming effort. Nevertheless we do see value in catering to both groups. Therefore in future retail cards reviews we will try to appeal to both groups so that we are sure you are getting the full gamut of gaming experience evaluation. FRAPS will also remain as our main performance measuring tool capturing framerate every second.
So, any questions on the review or tech article themselves? I know there is a 23 page thread here, but that would take a day to read through and respond to
We have seen that there is a general consensus about our current reviewing method, and that is that people do not like the fact we are benching each card at different resolutions and quality settings and showing it in a graph. We do appreciate everyone's feedback and are striving to improve the method. We also encourage everyone to sit back for a second and try to understand what we are trying to show.
We put this warning on Page 4 of the Preview: http://www.hardocp.com/article.html?art=NjA2LDQ=
Warning!
Be aware that not all results represent "apples to apples" comparisons. We try to find the highest playable resolution, anti-aliasing and anisotropic filtering settings for each card in each game. Keep that in mind as you look at the graphs and find that each card may be set at a different resolution, AA and AF level. What we strive to illustrate to our readers is consistency in performance and which cards provide the highest level of gameplay performance. Should you need apples to apples benchmarks that focus on average framerates, there are plenty of other hardware sites to deliver that content to you.
HardOCP.com "video card reviews" have stopped being video card reviews. We are truly trying to measure the level of gaming experience the hardware can provide. Average frame rates only tell the real gaming experience story if every frame is "average," and we all know that is far from being the case.
That being said, it is HardOCP's goal to provide our readers with information that should help you understand what level of gameplay experience would be provided by the hardware in a similar computer of your own.
Now, if that isn't clear enough in explaining what we are doing then I will try to clear that up here.
Basically what it comes down to is trying to figure out which video card offers a better gaming experiencing. A better gaming experience is achieved with delivering the game the way the content developer meant the game to be seen and with the highest image quality possible. The higher the image quality, the more immersive the experience. Performance is also part of that equation. What we strive to do is find the balance on each card that provides the best image quality with playable performance. Then we look at the consistency in performance is achieved on each card at those settings. Then we compare.
What you are looking for in the graph is to see which card can play the games at the highest resolution with the highest quality settings with playable performance.
When looking at the graphs FIRST look at the top of the graph where it shows the Resolution and AA and AF settings achieved with each card. THEN look at the performance in the graph below. Then you can say for example: "Oh, ok, Card A can play at a higher resolution, a higher aa and af quality than Card B and sustain the same amount of performance with Card B at a lower resolution and a lower aa and af setting".
You also must understand different levels of performance are acceptable with different games. Some games 30-40fps is acceptable, some games 60fps or more are the acceptable framerates.
Now, of course we strive to improve the method, it is a constant process. We understand that some people really see value in "apples to apples" resolution and quality settings. One thing you must understand is that you cannot say NVIDIA's AA and AF and ATI's AA and AF are "apples to apples" to begin with. Comparing NVIDIA's 2XAA, or 4XAA or 8XAA, or 8XAF or 16XAF are really not directly comparable because different methods are being used, they are close, but not exactly the same. About the only thing that is directly comparable is resolution.
But we understand that though that fact exists there are still people who insist on seeing the cards compared at the same resolution, AA and AF settings. Ultimately the optimal solution is to show both ways in each review. The problem with that is of course time. Benching 10 games, both ways, on 3 different cards, which perhaps 2 more graphs per game, is a time consuming effort. Nevertheless we do see value in catering to both groups. Therefore in future retail cards reviews we will try to appeal to both groups so that we are sure you are getting the full gamut of gaming experience evaluation. FRAPS will also remain as our main performance measuring tool capturing framerate every second.
So, any questions on the review or tech article themselves? I know there is a 23 page thread here, but that would take a day to read through and respond to