I want your opinions and feedback

If it is possible it would be cool to have a overclocking section. I see you did it with some cards, but the newer ones I would like to know how well they OC and what benefit it may bring. I would preferably like to see some water cooling.
 
This critique may come off as harsh, but I am using this opportunity to express my honest assesment of you video card reviews. In my opinion, these "best playable setting" comparisons are pretty useless without having a range of combinations in contrast. You guys are tinkering with a formula that works extremely well. For example, in Hexus's latest review, they include the standard 1024 -> 1920 resolution range, with high settings and high + AA/AF settings, and in the comments section for each game they express what they feel is the best playable settings. Brent, it is not for HardOcp to decide what I want to play my games at. Your reviews used to be some of my favorites, before your "apples to oranges" decison, and image quality debacle. Image quality comparisons are of course important. However, when reading your reviews, I feel like I'm seeing a conclusion before I see the experimental results. I rarely feel like I come away with a well-rounded opinion of the product.

What I would like to see:
1. Introduction
2. Test Methods, Setup Procedure
a) Overclocking results
3. Results, catagorized by Game/Resolution/IQ Settings
a) Comments for each graph/best playable settings
4. Discussion Section
a) IQ Assesment of each card
b) Value assesment/return of fps/$
c) Heat noise, etc
5. Conclusion Section
a) Your personal opinions
b) Discussion of availability

As it is your reviews come off as overly subjective, based on your opinion, your experience, and feel like they take into account your biases. What I want is objective reporting, with a specific section where you express your thoughts. Give me your input on how you like to run your cards, but give me the privilidge of deciding how I want to run them. I would like to be able to see how the games run in a VARIETY of settings/ resolutions, so that I can best make my own assesment. Your "apples to apples" section is nowhere near in depth enough for this. It is a case where a "bit of everything" is not enough.

I understand that you and Kyle want to distinguish yourselves from the pack, but I don't like the deviation, and your reviews receive my attention last, if at all. Thanks for making this thread, I'm interested to see what changes will come about.
 
There's something that's always really wound me up about HardOCP's reviews and that's the failure to produce proper graphs.

Take a look at this;

1131492082JqTATvC5ZP_5_9.gif


You should be able to see the tops of all the peaks of all of that line. Yes it's relevent info. Why leave it out? Why not extend the Y axis to let all of the line be in view?

It's just looks REALLY like amatuers work. It looks like you never went to school and were taught how to present data properly in a graph. Yes I realise the intended subjectivity behind the reviews and test scenerios and that you consider anything above 75fps to be irrelevent, but what's the point of having a graph that doesn't show the entire data?

Either have a full graph or don't. I'll decide what is a relevent fps cut-off point thank you very much ;)


Which leads me to a more general point....
 
Ok, I realise HardOCP has deliberately moved away from sharp apples to apples data comparisons, but sometimes it's good to have. By this I mean the inclusion of things like (oh blasphemy) plain 3dmark scores.

Don't get me wrong, I don't care much for the scores or the whole optimisation thing yada yada, BUT... If I see a Radeon something get twice the score of a Radeon something-else, it instantly gives a useful hit of data. It's just one 4 digit number, why not include it?

By having this whole mentally of a 'subjective' review what you are basically doing is hiding the raw data and interpreting it for us. I don't want someone else telling me what is a 'good' framerate or what is the best image quality/performance compromise...etc.. I'd like to judge that for myself based on presented data.

I can't do that if I'm presented with deliberately limited graphs and deliberately compromised 3D settings.
 
I like them overall, but I really want to have apples to apples comparisons. Turn of AA, turn of AF, slam it out at 1600x1200 and let me know how they do processing the game at the same res, with nothing else on. When they're close, that can often tell me which has a little more headroom. Just with one or two games, but I'd really like to see that.

EDIT: In addition to what you have now :)
 
Higher resolutions (1920x1200). You don't need to test 1680x1050, since that is actually slightly less resolution than 1600x1200(1.76MP vs 1.92MP) and the performance of the two should be very close.
 
I would also agree with the apples to apples comparisons. It would paint a better picture as to how the hardware works at various resolutions with specific settings in mind. I would personally like to see the apples to apples on a minimal quality, average/about average, and then FBD - Framerate Be Damned aka highest available. If applied to both OpenGL and DX based games, this I feel would help paint a better picture as to what is truly the best card for the games we play.
 
how about an evaluation between intel and amd platforms? does amd really have the gaming advantage when the games nowadays are more gpu limited? and maybe cpu scaling with intel sli/crossfire setup and amd sli/crossfire setup?
 
I want to see more midrange systems used. I cant afford a big and bad system I would like to see how to make the biggest bang for my buck. Basically a full midrange system.
 
I am not happy with the reviews, I love the graphs that show min/max fps per second but even so there are problems.

If you can take a minute to read and consider this that would be great.

Supposedly the current system "cheats the cheaters", and "gives real world evaluation"

So lets think about hypothetical scenario.

You have CardA and CardB by two different companies.

CardA has some questionable optimizations that makes it run very well with 4x AA on.

So on an "apples to apples" reviews lets try these hypothetical numbers, since I can't create a graph for the hypothetical:


New Game demo1
1024x768 4x AA
CardA = 90 fps , min fps 50
CardB = 79 fps, min fps 30

1600x1200
CardA = 79 fps, min fps 30
CardB = 45 fps, min fps 25

Ok so for this "old style" review the evil cheaters have won the day with their questionable optimization.

On a "new style" review, apples to apples optimization what would you do?

You would just graph CardA @ 1600x1200 playable settings, and CardB at 1024x768 playable settings, and call it a day.

How are the cheaters cheated? How is it any better than apples to apples?

I simply don't understand, this site used to be my favorite site for video reviews but ever since you started this system I either skip right to apples vs apples or get my reviews elsewhere.
 
I like your reviews and find them very informative, however, I don't own an AMD based system. It would be nice to see the numbers with a high end Intel rig. I realize this would double the work load required for a review if you were to show how the cards performed on two different systems. I think there are a lot of [H]ard core gamers out there still using Intel rigs.
 
With increasing number of cards supporting VIVO, there seems to be very few site that goes into any great depth in regards to how well each vendor implements their respective VIVO feature. Most reviews consists of a passing note that the feature is included. A coverage of greater depth (color accuracy, ability to handle really really BAD analog signal, etc.) might set [H]OCP reviews apart from others.
 
Brent_Justice said:
thanks, we've actually been discussing these things, i do want to include heat and noise tests, i think they are important for enthusiasts, power is as well, but at this time we lack the equipment and expertise for it
Brent, as far as power usage, all you really need is a one or more Seasonic Power Angel. I can't see anything else you would really need. As long as you keep the rest of the platform the same, and are just changing video cards, the Power Angel should suffice.

Oh yes, I also feel that you should start concentrating on the 1920 x 1200 resolution . When you read just how many people here, (including myself) have bought Dell's superb 2405FPW monitor, 1600 x 1200 is just not the ne' plus ultra of resolutions any more.
 
More quality comparing shots. AA/AF comparisons between cards, etc.

And Widescreen resolutions. 1680x1050, and 1920x1200 would be my two favs.
 
I realize that most people run games at 16x12 or less, but it would be nice just for comparison sake to see higher res benchmarks. Especially for those with high end CRT's or running 2405fpw at 1920x1200.

It would be nice to see what card does best at the native resolution of monitors like the 2405fpw.

will also stress the cards that much further.

Dave
 
One previous poster mentioned a desire for comparisons with cards from earlier generations. I agree that there is some value in seeing exactly which card is top dog in what games, but many of us are evaluating our options for upgrades. I would personally find it very useful to know how current cards compare with cards two generations back, e.g. benchmarks on a Radeon 9800 Pro/GeForceFX 5900 Ultra, a sample from the 6x00/Xx00 series, and whatever the current card on the test bed is.

I realize that not all of the games you use for benchmarks today were used previously (and vice versa), but I would be perfectly happy using the numbers from the previous reviews for the games that are still in your test suite. Who cares if they were run on a machine with only 512MB RAM instead of 1 GB, and with a slower processor? Personally, I'm on an older machine with less RAM and a weaker video card, and I'm considering building a new machine in the near future. I would like to get an idea for how much better my gaming experience will be relative to what I have now. If I can get $FPS in $game on a Radeon 9600 with 128MB RAM, how much better will my experience be using a GeForce 6800?

I guess the point I'm trying to make is that not everybody upgrades their video card every generation, and some not even every other generation. It's nice to know how the latest generation of cards perform on their own, but it would be nice to see how much better they are than cards that are a year or two old.

 
#1 i would like to see the peaks on the tops of the graph lines as well. the graphs just look wrong without showing all of the data.
#2 i would like to see more apples to apples comparisons as well. i understand why [H] moved away from doing those types of benches, but at the same time i like to be able to do a direct comparison.
#3 I know the generally accepted "playable" frame rate is 30 fps, but I cannot handle that even for a millisecond. even more so if i just paid $500 for the damn card. this one is purely subjective but i refuse to play below 45 FPS. it just feels smoother.
#4 3D Mark 05... it sux, we all know it, but we all use it. it would be nice to see the default bench come back if only so we can tell when another less scrupulous review site pads their scores.
EDIT: #5 i also am a widescreen guy (Sony GDM-FW900) and would like to see benches at higher res. i doubt you will do them at my max(2048x1536) but 1920x1080 would be nice for us HD folks.
 
Widescreen performance numbers, mention of the shimmering with the NV cards.
 
I like every thing for the most part. The graphs are not so hot, but you put the numbers at the bottom so I cant complain.

But please! Keep the Apples to Apples section. And please always include the Apples to Apples section.

Your idea of playable and mine are not the same :) And it just throws me off to cypher each video card's resolution that your comparing and claiming playable. I cheat :D and go straight for the Apples to Apples section every time.

Thanks For Listening
 
dnottis said:
...mention of the shimmering with the NV cards.

It has been addressed I'm pretty sure.

You guys definitely should post overclocking results, versus just how high you can clock your card. My radeon can have a 60mhz raise on core and mem, but does it make a difference? Nope. My point: what's the case with other cards?
You don't have to do every single game in my opinion, probably just the one where performance lacks the most. ("however you can help your situation out, blah blah blah)

I also agree that capped graphs are pretty weird, especially with the high fps numbers you can't really tell what's going on.

2048x1536. I know you can do it, you did it with doom 3 when it was first being released. I also know most gamers don't run at that res, but it's still cool to see where technology is these days.
THEN widescreen.
 
Here is what I would like to see added.

1. Widescreen resolution as they are becoming more popular
2. Noise reading from the cards
3. Will it fit in a shuttle?
4. A simple plain old apples to apples on the top two games out
5. Some X2 benchies would be great too!
 
I dont know if its been mentioned at all...but how about widescreen resolutions??? ;)
Also I'de like to see a comparison with other cpu's...I think more [H]'ers have better gpu's than cpu's...so even though I have a GTX...I'm running it with a 3200venice.
 
The reviews have been very soild lately; keep up the good work! :)

I'd appreciate some testing in widescreen resolutions.

Also, because HardOCP seems to strive to find the maximum playable settings for the average gamer, perhaps you should run some more tests at 1280x1024. This resolution is very popular due to the number of people with 17-19" LCDs.
 
ThirtySixBelow said:
The graphs are the best part of these reviews. An average FPS ratting gives you no idea of how the GPU performs overall. One could blast through the whole game but have one major slow down that would bring it's avg fps down to an unrealistic score. HardOCP reviews are one of the very few reviews I really base opinions on. Everything is solid and gives you real world marks. .
BS.


You are telling me that they play through the entire single player game of Doom 3 every time they test a new video card?

Not a chance.

Edit: I don't even read HardOCP video card reviews anymore...I hate having to decipher their "real world gameplay" into useful information.
 
Far Cry benchies and more apples to apples. I'm jumping in this thread now I'm jumping out. You wanted to know what I look for so there ya go.
 
Here's my $0.02:

-As mentioned earlier, adjust the scale of your graphs to fit the data measured.

-Use graphs for apples to apples measurement. If one card is running at 1600x1200 2x tr ssaa/16xaf and another card is running at 1600x1200 4xaa/16xaf they don't really belong on the same graph. The information on maximum playable settings belongs in the evaluation, just not on the same graph, IMHO.

-Emphasise the difference between the graphics settings and how it looks on the screen. If a person spends $100.00 more so they can go from 4xaa to 8xaa, both cards are bouncing on 27FPS, and you can't really see a difference on the screen then that's not really a selling point for me. If that $100.00 gets me a noticeable increase in eye candy and lets my FPS never get near the realm of unplayable then I'm probably thinking about becoming a customer.

-Try to get the makers of F.E.A.R. to get you a fix for the autosave problem that drops your FPS to zero. ;)



The next bit is more of an article, or articles, I'd like to see rather than standard testing measures for all reviews.

- NVidia's latest drivers are supposed to begin taking more advantage of multi-core CPUs. Throw a dual core CPU against a single core with some popular video cards.

- Another poster recomended an Intel vs. AMD test, try it with some popular cards.

- You use an FX-55 for most tests to avoid bottlenecking. At what point does a particular card start to get bottlenecked by the system? A 4000 San Diego, a 3500 Clawhammer, a 3200 Venice?
 
First off all the reviews are top notch. I trust them as you guys are real class acts. I dont care about the 3D mark scores. I dont think that you should include them as you guys know you set the trend for others to follow I dont want people and noobs to read the scores and think that they have any bearing on performance.
I would like more coverage on wide screen / high res stats. I only use wide screen formats so that would be great for me. I would also like some kind of chart or graph that would say when cpu bottleneck would occur at different resolutions. It would be nice to have some slidding scale so that you could build your system around what resolution you use. Use of dual cores would be nice when they would show a difference. AMD vs intel only when there is some competition :D
 
R1ckCa1n said:
Here is what I would like to see added.

1. Widescreen resolution as they are becoming more popular
2. Noise reading from the cards
3. Will it fit in a shuttle?
4. A simple plain old apples to apples on the top two games out
5. Some X2 benchies would be great too!
I agree
 
Thank you all very much for the feedback, I'm going to read through this thread tomorrow and make notes :)

We are always looking to improve our evaluation process and want to make sure we are showing the kind of information gamers want to see.
 
Brent_Justice said:
Thank you all very much for the feedback, I'm going to read through this thread tomorrow and make notes :)

We are always looking to improve our evaluation process and want to make sure we are showing the kind of information gamers want to see.

By asking for feedback, you are creating an appreciate support base. Amazing how a nod in the public's direction creates community. Hope the posts are useful to you.
 
Nihilanth99 said:
Also, because HardOCP seems to strive to find the maximum playable settings for the average gamer, perhaps you should run some more tests at 1280x1024. This resolution is very popular due to the number of people with 17-19" LCDs.

This is how I feel, lately I have been rather ignoring most of the review, since 1600*1200 does not apply to me since my LCD does not support it. Therefore seeing the framerate avg with 2xaa and 8AF or at whichever setting doesnt mean much to me at all since my LCD has the limit of 1280*1024.

That is really the only gripe I have about the reviews.

EDIT: I would also like to see more applies to apples comparisions when it comes to resolution, AA and AF settings.
 
I'd like more attention given to the Apples to Apples section. Such as giving the comparisons for settings other than totally maxed out.
 
Personally I don't see any point in apples to apples tests. When I buy a video card I would rather get an idea of what settings I can run the game on rather than looking at numbers. Numbers are numbers, nothing more, nothing less, and dont give me an idea of what settings I can play the game on.

- I think 1280x1024 comparisons would be a good thing.
- and Widescreen.
- Athlon 64 X2 ;)
 
1. Fix the graphs. They are not big enough and when there is more than two cards (two lines) I can't tell what the hell is going on. I pretty much skip by those now. However, if you list the FPS for every second during the test that would be cool, cuz I can use Excel.

2. Keep the "best playable" section of the review but add a "benchmark" section. By benchmarks I mean testing results of game play at different settings, not 3DMark. I'm sure you guys actually run a game at different settings right? So why not list the results of those settings in table and/or graphs? All the resolutions supported, different game settings, different driver settings, and a mix of all 3. This would be the apples to apples section but change the name to test results. You don't even need to write about anything is this section. Just list the numbers you saw. Leave the performance results in the best playable settings section. Doing this will let people check your testing data for a setup that is maybe similar to there's but at the same time let them see which card has the most horsepower

3. Comparisons with old/cheap cards would be cool. Comparing two $500 cards is nice and all but I'd like to see how much they beat the $200 card. You can't do a review of a video card without comparing it to another video card but you don't have to get crazy and list 50 video cards. If you did please to list them all on the same line graph.

4. Percentiles are hella cool. I like the benchmark built into FEAR.
 
I like the FPS graphs because they give me an idea on how the card will run. If you do decide to get rid of the FPS graphs, simple information in the article about the distribution of FPS would be nice (similar to what the FEAR test gives you).

Similar to what others have said, it would be nice to have max playable settings at lower resolutions, especially when not all of us have monitors that are able to do 1600x1200 well.

OC results would be nice, especially as the coolers on the graphics cards get more advanced.
 
1. Similarly Equip 2 Rigs, 1 Intel Dual core & 1 AMD Dual Core 2GB Ram a piece.
2. Always test with identical IQ settings side by side. If a card is not playable put N/A in the graph.
3. If 8 games is the limit, bench 4 Open GL and 4 Direct 3d Games.

4. Show scores at higher resolutions.
5. Speak of the image quality you personally experienced as sometimes screenshots can degrade quality and not reflect the actual image quality in real time.
6. In the graph choose 1 card over the other since comparisons will have same IQ. You can call it best gaming experience. This should be based on performance and image quality. Declare a winner for each game.
7. Overclocking results based on 2 games 1 open GL and 1 Direct 3d. Both cards overclocked to max stable settings and compared side by side.
 
The good:

The way you do the reviews in general rocks. You are the only site that compares how each video card plays games at whatever settings they handle. This by far gives the best picture of what you can expect when actually playing. Love it. I could care less if you guys do straight up setting-for-setting comparisons. A million other review sites already do that. Don't waste your time. And thanks for the SLI info. It really help me decide when to switch out hardware. For instance, I've been fretting over the release of 512 GTX cards, well they do rock, but not enough to justify either the price or my getting rid of brand new 7800 GTs. So I can wait.

The bad:

As mentioned above, I understand what you are trying to do with the graphs, but you need to totally redo them so they are readable. Make them about 4x as wide, and use some other program or type of graph. It looks unpro. Clean it up so we can easily see which cards are performing poorly at the bottom end. I want to be able to clearly see which cards are spending the most time below playable frame rates. It doesn't matter if a video card wins the overall average, if it has very high peaks, but also very long slow spots. What if someone created a video card that can pump out about 60fps tops, but is designed in such a way that it never drops below 40. Another powerful video card may have an average of 70fps, but it frequently dips into the teens. I'd rather have the more consistent card.


Thanks for the great work [H] !!!
 
I agree on measuring the environmental aspects of the cards being tested. Power usage is relatively easy to measure as pointed out earlier, and as far as noise goes, a subjective opinion would be sufficient for most, although a simple dB measurement with a dB meter shouldn't be that difficult either.

One other thing I'd like to see added to reviews is the quality of TV-out if present.
 
I noticed in 2 of the most recent reviews (that I have read) that you are benchmarking with Everquest 2, one thing that had me scratching my head was the quality settings, you guys seemed hell bent on running the game at 1600X1200 with AA etc, with balanced quality settings, yet FEAR you guys ran at 1280x960 with max detail... how about a comparison of Everquest 2 at a higher quality setting with lower res and perhaps no AA (like 1024x768)

Another point that I do not agree with is your belief that you can not tell the difference for any framerate above 60, I'll tell you right now that there is a noticeable difference between 75 -> 85 and even 85 -> 100, here is my reasoning:
I am an avid Quake 3 player, and have completed the single player on Nightmare difficulty (as many avid Q3 players have) and have found that on some maps, Nightmare bots cannot be beaten with a framerate/refresh rate of 75hz and in some cases even 85hz, 100 hz is no problem, one particular example, I would loose a particular map at 75hz, however at 100hz I can triple the bots scores (4 bots, frag limit 20, I win by 15 frags) that is a huge difference.

The highest my monitor goes at 100hz is 1024x768, as far as I'm concerned, the "holy grail" for gaming is 100hz not 1600x1200 res (both would be great, but I value the frames over the res), and with 6 temporal adaptive AA and 16AF 1024 x 768 is a fine res to play at.
 
I want milk and cookies sent to every reader with each review.
I want to know how the cards run on *my* platform of choice: A PII300 with 16MB RAM.
And..
And...

I want readers who can extrapolate the data you give them, allowing them to get a good idea of how a card compares to whatever card they have in mind..

I do agree on the FRAPS graphs. Not really useful enough for the purpose of the review to keep 'em. The chart's enough.

You folks want more apples-to-apples and 3dmark ? As has been said many, many, many times: Why do you think that links to other sites' reviews are posted ?
 
Back
Top