Apples to Apples or death!

Status
Not open for further replies.

BurntToast

2[H]4U
Joined
Jun 14, 2003
Messages
3,678
You REALLY need to have an apples to apples comparison for EVERY benchmark. I like coming to [H] for the news and I read or at least browse every one of your reviews. But when I come face to face with a review for example...

ATI 1900XTX 1280x1024 4XAA MED QUALITY
NVIDIA 7800GTX 1280X1024 2XAA HIGH QUALITY

I'm like WTF and go to google to look for an Apples to Apples comparison. Same game, EXACT same setup in multiple resolutions/settings and then decide for myself which card is the better one for the settings that I run.

Whoever’s idea it was to do benchmarks based on playable settings needs to be shot. Because at least in my case those reviews don't even matter to me.

BTW I have had this opinion for a L-O-N-G time. But it was the "ATi Radeon X1900XTX Performance & Overclocking" headline that I read on the front page that got me fired up.

My Style Review.. review I fully read -
http://www.nordichardware.com/Reviews/?skrivelse=469

No My Style.. review I spent 1min looking and then went to the review linked above -
http://enthusiast.hardocp.com/article.html?art=OTUzLCwsaGVudGh1c2lhc3Q=

Now I don’t care if you consider your review of a higher quality than that of www.nordichardware.com. The fact is that since you didn’t have an apples to apples comparison, I went to a site that did and there for relied on his information solely. Of everything in a review, benchmarks are the #1 thing that I come for. Followed by stability/compatibility issues and then the reader’s opinion.
 
There are good reasons for both apples v. apples and stable v. stable.
 
Looks like the OP should go back and reading [H]'s REASONS for their methodology.

You can't please everyone.
 
No need to flame guys. I think the best way to review would be a combination of apples to apples, as well as equivalent playable. This way no one would feel the need to complain, and the reviews could not be accused of being anything but thorough.
 
The used to contain some apples to apples numbers and the end of their product reviews, probably still do in some reviews
 
I've been wondering about this for a while myself... I very rarely take reviews seriously now, even [H]ard|ocp. I've read far too many reviews lately wondering how much the site got paid for it, nobody seems completely honest anymore.
 
If [H]ardocp did "apples to apples" reviews like all the other review sites out there, then that's exactly what they would be: just another review site. Personally, I would not pay as much attention to [H]'s reviews if they were generic time-demo reviews like all the other sites do.

I'm not saying that I don't like apples to apples reviews as they are "easy on the eyes" and I can quickly see the performance difference between each of the cards being compared. However, there are tons of sites out there that do this, and having [H]ard try something different is pretty cool.
 
BurntToast said:
needs to be shot
That really isn't a good phrase to use if you are trying to prove a point.
BurntToast said:
Now I don’t care
That is also one of those phrases.

From [H] Review:
Please be aware we test our video cards a bit differently from what is the norm. We concentrate on examining the real-world gameplay that each video card provides.
 
Who cares about brownie points.
I don't walk through life with brown shit on the end of my nose.

I have to agree with the OP, I find the reviews here to pretty much suck.
I have never liked this style of reviewing ever since they started doing it.
In fact I don't even read the reviews or go to the front page anymore.
I just come here to cruise the forum as another source of info.
Of course you have to wade through the endless !!!!!!ism bullshit posts to get some good info, but it can be found.
 
kcthebrewer said:
From [H] Review:
Please be aware we test our video cards a bit differently from what is the norm. We concentrate on examining the real-world gameplay that each video card provides.

But you see this is exactly what makes the reviews bad.
They are giving us there "opinion" on what they think is playable.
My definition of playable is different then others.
Not to mention what I think looks good, looks different to to others as well.
The only true way to review is in cold hard numbers.
Give me the numbers and I will make up my own mind, I don't need someone else to do it for me.
But this is all my "opinion".
 
JRT said:
The only true way to review is in cold hard numbers.
Give me the numbers and I will make up my own mind, I don't need someone else to do it for me.
See this is where you run into an issue. Those straight numbers sites just do the regular 0x0x 4x8x and possibly at different resolutions (I know that is generalizing a bit). That may have been alright back in the day, but you have so many more things to take into account now such as game settings, HDR, supersampling, transparent anti-aliasing, adaptive anti-aliasing, non-angle dependant anisotropic filtering, etc. [H] shows what is playable at the highest quality they can get per game per card and at playable rates. If that doesn't suit you, there are tons of other sites to go to. Still, that shouldn't call for heading to the forums and telling the website owners/employees/readers to **** off and they don't know ****.
 
Just because you can't understand the depth of information in these types of reviews as compared to those other reviews doesn't mean they are wrong.

"You should read all the reviews you can get ahold of before you make any decisions and buy anything" -Micah (paraphrased)

~Adam
 
The highest playable settings are the [H]'s choosen methodology for reviewing cards. If you have issue with this method you can always email kyle@hardocp.com and explain to him what you don't like.
 
I just wanted to address this topic, if you go back and read our editorials you'll understand why we do things the way we do. I'll go ahead and summarize some points though.

First of all you have to understand we aren't reviewing video cards.

We are evaluating the gameplay experience provided by video cards in games.

What is "playable" is subjective, to a point. Some games run differently than other games, for example Call of Duty 2 I have no problem playing the game at 25FPS and higher, the gameplay is just different at lower framerates than some other games. In the Sims 2 I have no problem playing the game at 20FPS and higher, again the gameplay is simply different. However in most games our baseline is 30FPS and higher with vsync off. I think most will agree if you can sustain 30FPS or higher CONSISTENTLY (key word there) then you will have an enjoyable experience. Consistency is very important since as we as humans notice the CHANGE in framerate, and that is what annoys us most.

Since we use this same formula and criteria for every game we test with every video card the results are very much comparable. Even if you disagree with the settings we have found playable you can still tell which card is faster, an example.

Card A = 1024x768 noaa/noaf

Card B = 1600x1200 4xaa/16xaf

An extreme example, but you can easily tell that Card B is a LOT faster. This being able to tell if the card is faster is the same result you get when doing an apples to apples test, since isn't the goal to see which card is faster? Well, you can see the same thing with our gameplay evaluation.

A lighter example

Card A = 1280x1024 2xaa/16xaf

Card B = 1280x1024 4xaa/16xaf

Ok, obviously you can still tell Card B is faster, but not a great deal. This is important because then you have to look at the price the card is selling for and determine if the price difference is worth the performance improvement/visual improvement and experience compared to the other card. Our eval method can tell you all of this, it covers ALL bases.

Visual quality is important to gamers and our eval method takes into account ALL forms of features out there, such as in-game settings like HDR and other settings like Transparency Multisampling/Supersampling, Adaptive AA, High Quality AF, Gamma Correct AA enabled on NV cards, etc.... Our site is unique in that we explore and use all of these features just like gamers will.

Our eval method is setup just like a gamer does when he plays a game. What do you do when you get a brand new game? I bet you put it in, install it, and go inside the game and mess with and figure out what in-game settings you can enable, how high you can set them, what resolution, AA and AF level you can run at, right? Well, that's exactly what we do as well! It matches perfectly with how gamers play their games.

The facts are every other site does cookie cutter apples to apples benchmarks, if you want to see those kind of "reviews" then by all means do so. We don't do that anymore, we have chosen to implement a system that more closely matches how gamers play games and what a gamer looks for, visual quality/experience.

We always urge everyone to read several reviews, that's the great thing about the Internet, you can read ours, read other websites, and make up your own mind on a product.
 
Status
Not open for further replies.
Back
Top