I want your opinions and feedback

B

Brent_Justice

Moderator
Joined
Apr 17, 2000
Messages
17,755
With the latest evaluations as of late we've been pumping out solid gameplay experience evaluations with our video card reviews.

I want your opinions and feedback. This is your chance to post what additions/subtractions/changes you feel we should make to the evaluation process to further improve them.

We want feedback from gamers out there to how we can provide better information that is relevant to you. What do you want to see as a gamer?

Please read our latest evaluations, see what we are doing, see how we are presenting the information and then give us some feedback.

http://www.hardocp.com/article.html?art=ODg1
http://www.hardocp.com/article.html?art=ODgy
http://www.hardocp.com/article.html?art=ODc1
http://www.hardocp.com/article.html?art=ODU1

All offtopic posts will be removed, please no flaming. If you wish to provide feedback in private instead please do so at [email protected].

Thanks
 
Honestly I don't know what you guys can do to improve your hardware evaluations. You have everything I'm looking for: highest playable setting cross-referenced with minimum and maximum framerates. The methodolgy is clearly written up and consistently presented.

The only addition I would like to see is DoD: Source included as a benchmark game. The mixture of drawdistance and HDR can really choke the fastest machines, esp if you're after high minimum framerates. The inclusion of a Lost Coast evaluation addresses a lot of the same issues, but in online multiplayer (much the same way Q4 MP behaves differently than Q4 SP) the hardware is really stretched to the limit. Maybe this points to a new set of MP evaluation criteria for new hardware?
 
  • temperature differences inside the case between the cards tested.
  • difference in power usage between the cards.
 
I would really like to see 3Dmark. I know that it is no good for determining "real world" gameplay performance, but it does give the reader a fair idea of how the card stacks up to the rest. Now, I'm not talking about a 3Dmark graph that has every CPU and GPU in existance, that is a useless comparison. I want to see 3Dmark tests that compare the GPU on the line to it's brothers from the same manufacturer running on the same CPU. For example, here is a scenario:

The testbed is an ASUS SLI board, FX-57, and 2GB of RAM. The cards to be compared would be the 6800GT, 6800U, 7800GT, 7800GTX, and 7800GTX 512MB (or X1800 series if you are red). Notice how there would be no cards from the compettitor, this is to prevent 3Dmark optimizations coming into play. If all the cards are from the same manufacturer, then the optimizations will cancel out. This helps the reader get a gauge at how the card being reviewed stacks up against its predecessors (little brothers). For example, someone with a 6800nu will be able to see exactly how much he will gain by jumping to the 7800 series. The purpose of the 3Dmark would be, not as another benchmark, but to compare this card with its siblings.
 
Opie said:
Honestly I don't know what you guys can do to improve your hardware evaluations. You have everything I'm looking for: highest playable setting cross-referenced with minimum and maximum framerates. The methodolgy is clearly written up and consistently presented.

The only addition I would like to see is DoD: Source included as a benchmark game. The mixture of drawdistance and HDR can really choke the fastest machines, esp if you're after high minimum framerates. The inclusion of a Lost Coast evaluation addresses a lot of the same issues, but in online multiplayer (much the same way Q4 MP behaves differently than Q4 SP) the hardware is really stretched to the limit. Maybe this points to a new set of MP evaluation criteria for new hardware?
Click to expand...

Thanks for the feedback.

We used DoD Source a bit before HL2 Lost Coast was released. We also plan to upgrade to Aftermath once it is released, there are only so many games we can include in each evaluation though.

I'm also looking at COD 2
 
Brent_Justice said:
I'm also looking at COD 2
Click to expand...


That has been something I have wanted to know because it's a damn good game lost in the hype of Quake4 and FEAR. Is there any chance of seeing an RTS such as AoE3 as well in future benchmarks?
 
Spank said:
  • temperature differences inside the case between the cards tested.
  • difference in power usage between the cards.
Click to expand...

thanks, we've actually been discussing these things, i do want to include heat and noise tests, i think they are important for enthusiasts, power is as well, but at this time we lack the equipment and expertise for it
 
tornadotsunamilife said:
That has been something I have wanted to know because it's a damn good game lost in the hype of Quake4 and FEAR. Is there any chance of seeing an RTS such as AoE3 as well in future benchmarks?
Click to expand...

Those type of games are a lot harder to test, but it is something I'm researching. Again there are only so many games we can include in each evaluation, 8 games is pretty much the limit to do a timely evaluation.
 
You guys have been doing an excellent job, lately.

The only thing I'd like to see is a short round up on drivers. Doesn't even need to be a paragraph, or anything written. A simple graph of, say, the last three/four driver releases would be cool. 81.85 vs 81.86 vs. 81.87 vs. 81.89. Could get hairy, I'm sure, but I found myself shuffling through drivers this past weekend trying to find which ones would fix my B&W2 problem. :D

Thanks.
 
I think the Sims 2 benchmarks could be replaced with something more useful. I usually just skip over the Sims 2 benchmarks because in my opinion, they dont really tell me anything. The game runs smooth, but the fps are kind of low/medium. But it doesnt matter because after a certain point the game is smooth anyway. I dunno, its kind of confusing and I dont think most people buy these high end cards to run the Sims 2 all maxed out. But I could be wrong, It did sell a billion trillion copies.
 
Cousin Patty said:
I think the Sims 2 benchmarks could be replaced with something more useful. I usually just skip over the Sims 2 benchmarks because in my opinion, they dont really tell me anything. The game runs smooth, but the fps are kind of low/medium. But it doesnt matter because after a certain point the game is smooth anyway. I dunno, its kind of confusing and I dont think most people buy these high end cards to run the Sims 2 all maxed out. But I could be wrong, It did sell a billion trillion copies.
Click to expand...
True, I'd have to say that 99% of the people here play FPS, RTS, and MMO games. Very few play sims or racing.
 
I did not see any 3Dmark scores?








:p j/k.

I like the reviews and I know that time limits you from including more games, but I would like to see more games. And maybe start using a dual core CPU? Maybe some video tests?
 
Just my 2 cents

I think the Evals are great. We are talking about one revieiw right

Then hence we dont need apples to apples comparisons

I would like to See Dod: Source
A major MMO (Matrix online whatever.. you know World of warcraft)
 
TeamStrykerCore said:
Just my 2 cents

I think the Evals are great. We are talking about one revieiw right

Then hence we dont need apples to apples comparisons

I would like to See Dod: Source
A major MMO (Matrix online whatever.. you know World of warcraft)
Click to expand...


WoW isn't a very *graphically* stressful game, neither is the game I play, Dark Age of Camelot. Despite having good graphics, they stress the CPU a lot too. I don't think they are particularly well-suited towards being used for benchmarks :(.

Age of Empires III is pretty graphics-intense, and I would love to see it included somehow as it is a different genre from the standard stuff that is usually tested.

As far as Call of Duty 2, I think that would be a good addition, as well. I don't own it, but from what I've seen it looks very stressful, and on Firingsquad, they have benchmarks where the 512mb DOES make a difference.
 
It would be nice if we could see the max playable settings @ 1280 too, not only @1600x1200.
I think the majority of gamers have a 17"or 19" LCD, 1280x1024 is their max. For them it would be interesting to see if the new GTX/512 can handle Battlefield 2 @1280/8xssaa or that the XT can handle 1280/6xADAA. :)
 
SPARTAN VI said:
Good idea. Lots of 2005fpw/2405fpw gamers, including myself. :D
Click to expand...

Apple740 said:
It would be nice if we could see the max playable settings @ 1280 too, not only @1600x1200.
I think the majority of gamers have a 17"or 19" LCD, 1280x1024 is their max.
Click to expand...


Both good suggestions, in my opinion. I have a Dell 2005FPW, but I think a LOT of gamers have a 1280x1024 resolution LCD on their primary gaming rigs.
 
Couple of Ideas to consider (and that have been kinda listed above)...

1. Widescreen Benchmarking added (you know everyone is doing it :D ).

2. The addition of last Generation cards added to each Videocard review (so I could figure out just how good of an upgrade I would get going from say a 6800GT to 7800GTX 512MB).

3. Cost of the Card (on the first page please, with perhaps links to where you can buy it).

4. 64Bit performance Reviews. So we can see if there is a difference running on WinXP64 vs. the Standard version.

5. Perhaps a Future section with Windows Vista performance (since the whole desktop will be 3D and use acceleration. Also could tie into #4 above, as Vista will have a 32bit and 64bit version.

6. Updates to the review (or perhaps a whole new Section/Review area) that include new Drivers with their performance (gain/loss). An example would be like this...

CPU: FX-60 with 2Gigs of Memory, WindowsXP 32bit version
Video Cards...................................Min. FPS...Max. FPS...Avg.FPS...Playable Settings
Drivers used: Forceware 81.94 for Nvidia Based Cards, Catalyst 5.11 for ATI Cards.

Nvidia GeForce 7800GTX 512MB..........33.............155.............59.2........1600x1200etc
BFG GeForce 7800GTX 256MB............33..............99..............50.1........1600x1200etc
ATI Radeon X1800 XT 512MB...............23..............68..............40.1........1600x1200etc
Nvidia GeForce 6800GT 256MB............11..............40..............26.2........1280x1024etc

CPU: FX-55 with 2Gigs of Memory, WindowsXP 32bit version
Video Cards...................................Min. FPS...Max. FPS...Avg.FPS...Playable Settings
Drivers used: Forceware 81.94 for Nvidia Based Cards, Catalyst 5.11 for ATI Cards.

Nvidia GeForce 7800GTX 512MB..........31.............143.............56.9........1600x1200etc
BFG GeForce 7800GTX 256MB............31..............91..............48.9........1600x1200etc
ATI Radeon X1800 XT 512MB...............21..............65..............38.2........1600x1200etc
Nvidia GeForce 6800GT 256MB............10..............40..............25.8........1280x1024etc

CPU: FX-55 with 2Gigs of Memory, WindowsXP 64bit version
Video Cards...................................Min. FPS...Max. FPS...Avg.FPS...Playable Settings
Drivers used: Forceware 81.94 for Nvidia Based Cards, Catalyst 5.11 for ATI Cards.

Nvidia GeForce 7800GTX 512MB..........32.............145.............57.3........1600x1200etc
BFG GeForce 7800GTX 256MB............32..............92..............49.2........1600x1200etc
ATI Radeon X1800 XT 512MB...............22..............66..............39.1........1600x1200etc
Nvidia GeForce 6800GT 256MB............11..............41..............26.0........1280x1024etc

CPU: FX-55 with 2Gigs of Memory, WindowsXP 32bit version
Video Cards...................................Min. FPS...Max. FPS...Avg.FPS...Playable Settings
Drivers used: Forceware 81.85 for Nvidia Based Cards, Catalyst 5.9 for ATI Cards.

Nvidia GeForce 7800GTX 512MB..........29.............137.............55.2........1600x1200etc
BFG GeForce 7800GTX 256MB............29..............87..............47.5........1600x1200etc
ATI Radeon X1800 XT 512MB...............20..............64..............37.2........1600x1200etc
Nvidia GeForce 6800GT 256MB............9...............32...............20.6........1280x1024etc

It would be kinda cool to maybe have this in a separate section with updates all the time (maybe not so many, but at least a few) that could contain new CPU's (and there affect on the Benchmarks) as well as additional memory (maybe some new boards allow 4Gigs of Ram), etc.
 
Brent,

The thing that stands out most to me is Tables below each chart. Right now they are Setup to be 5 columns stating: Card, MinFPS, MaxFPS, AvgFPS, Playable Settings. The Charts are supported by the tables below, the tables are in turn supported by the paragraph to follow. So, it is easier to read them all from bottom to top.

I believe the Tables could stand on their own much better if you added a 6th column that simply stated the difference in settings. Examples: EverQuest2 the 6th column would say “High Quality” or “Balanced” for game setting. FEAR would have “+soft shadows” if they fixed the AA bug in the future and only 1 of the cards in the test could run it fine that way.

Settings that are the same for all cards in the comparison wouldn’t need notes in the table as they will be stated well enough in the paragraph that follows “..all cards played very well with textures at extreme”

I believe this would make the tables much more meaningful to your readers and hardly change the way things are being done by you.
 
I guess what I would like is a review to see how CPU limited a card is.....more specifically, how would a high-end video card work with a more mainstream CPU. A lot of us would like to upgrade to a better videocard but can't afford a whole new rig.

I would have like to have seen the 7800 GTX 512 in a 3200+ or similar system. I think no-one in their right mind would have this rig, but it definately would have been useful for the 6800GS review.

One of the reviews I saw today showed some solid gameplay on rigs with lower-end CPU as long as the eye-candy was turned up. Your style of review would go well with evaluating systems as I described.
 
Pesonally I like the noise and, temp in the articles now. I too would like to see the cost early in the reviews and, as was said using the previous cards it is supposed to replace. I like up-grading (might need to form a support group for it) but, I like to see large improvements before I do. Thats about all I can think of.
 
Brent and Kyle:

I think you should include 3dMark2005 the free edition benchmarks... I know you're all about gameplay performance but a majority of the [H] community uses 3DMark2005 as a basis in comparing each other's oc'ing scores and so on.... Plus it would bring more ppl to the site lookin for 3DMark scores instead of having to go somewhere else for them :) (Which would mean more $ for your advertisers and in the end you right?)
 
First off, the reviews you have are really excellent. A lot of people certainly appreciate it.

But if you are looking for new ideas, here's something that's radically new:

Post your conclusions page FIRST. I mean u know most people jump there anyway and then start backtracking to see exactly how the cards fared in the benchmark. Also allows the n00bs/people without time to just skip the gobblededook and get into the heart of the matter (yes I'm aware u have a jumpto combo box).

But it would certainly set you aside from other review sites. It would be "Here's WHAT we think of the card", and then "Here's WHY we think so", and you can argue your points right through.

Oh, and do add sound and temp readings as the other posters have suggested. I'm sure you'll be able to find the required equipment and expertise quite easily (perpahs even within the [H] community)!
 
Perhaps power consumption

And Call of Duty 2 is what I am playing currently, i would love to see #'s, preferably with a recorded demo from multiplayer playedback
 
Get rid of the big damned fraps graphs...I don't know about everyone else, but I find them just about useless, and they take up way too much of the page. We trust you guys enough to just take the raw numbers at face value without a big crazy graph. I would make it smaller such that it doesn't dominate the page and link it to a larger version for those who like that kind of thing.

Also, people want to see what cards do at different resolutions. This concept of "highest playable" settings is nice, but raw information is what most people want. And not in a separate section at the end of the review.
 
^eMpTy^ said:
Get rid of the big damned fraps graphs...I don't know about everyone else, but I find them just about useless, and they take up way too much of the page. We trust you guys enough to just take the raw numbers at face value without a big crazy graph. I would make it smaller such that it doesn't dominate the page and link it to a larger version for those who like that kind of thing.

Also, people want to see what cards do at different resolutions. This concept of "highest playable" settings is nice, but raw information is what most people want. And not in a separate section at the end of the review.
Click to expand...

Good thing I read to the end, because this is exactly what I wanted to say. I just prefer the apples to apples comparison. The graphs are not really neccesary, link to them if you want, but they just clutter up the sceen when you have so many layed over each other into one graphic.
 
/me Cracks Knuckles

Let's get down to business!

It seems that we should have a logical approach and structure to Video Card Evaluations.

The first thing would be to create an umbrella technology review/preview.
i.e. Nvidia 7 Series Technology or ATi X1000 Series Technology
In this re/preview you would discuss the feature set, the silicon process, the architecture and so forth.

The second thing would be to do reviews on the NVIDIA or ATi supplied (reference) HW for each offering. This will be the baseline performance on which all consumer cards are compared to. So in this case, you would do a review on each card in the 7 series.

As a subcategory, to the former, you will do individual card reviews from manufacturers. Only in these reviews, (though you will benchmark them all thouroughly) you would only report on the deltas from the reference cards. Stock Clocks, ram type, OCs, temperatures, bundles, warrantees, noise, instabilities etc. This way, I can say "hrmm, should I go with XFX or PNY?" and flip to meat of the subject fully knowing that the performance will be very similar.

With regards to the actual reviews of the reference HW, I would say this. Spend some time creating and documenting a review suite of software....Seriously. Pick some games/software like the following.

1) DX8 (DX7 may be a little old) Titles
1a) FPS/Action Adventure (do 2 titles)
1b) RTS
1c) Driving/Racing
1d) Simulation

2) DX9 Titles
2a) FPS/Action Adventure (3 titles)
2b) RTS
2c) Driving/Racing
2d) Simulation

3) Professional Rendering Program
4) Other canned benchmark. (for apples to apples.....not because it has any validity, but because you could roughly see the perf increase when OCing.

Pick the 'flagship' game from each, try to create a timedemo or standard benchmark.


Now in an ideal world you could set these up as a batch process on a ghosted OS install. However, I know it's not that simple. General Thoughts

Clean Install of XP- (there are ways to ghost a general setup, so that one image can be used on different patforms. Be sure to slipstream all major SPs into the image.

Use latest Drivers

Test Game 1, restart Test Game 2 etc.

Fianally, with regards to CPU testing. Do one test on each consumer chip architecture (A64, X2, FX) and if you get enough bitching from the intel folks, throw them a bone.

I would also omit the graphs...they confuse and overwhelm me. All I want to know is how does this card compare to it's peers? Is the technology a leap forward, or a minor step.

Throw in some IQ and SLI/Crossfire benchies and you are golden.
 
Brent,

As I have a nice Samsung 17" LCD, I'd like to see "Max Playable Settings" at different resolutions, not just 1600 x 1200, ie...

16AA x 16AF @ 800 x 600
8AA x 16AF @ 1024 x 768
4AA x 8AF @ 1280 x 1024
2AA x 8AF @ 1600 x 1200
 
How about adding some flightsims to the games tested? They're usually very demanding on the PC and it would be nice to see how the new hardware handles them. You could use LO-MAC (A hog IMHO, but very graphics intensive) and IL-2 which is played by a lot of people online, and can show the differences between how ATi and NVidia handle OpenGL games.
 
Put FS2004 back in there. :)

3DMark would be a general number that people can compare with, even though it is synthetic and system wide, but with everything else the same, other than the video card, it should be an OK comparison.

I really like the highest playable settings in your reviews. I love IQ, and I will sacrifice some frames for better IQ. Keep that in your reviews!

Compare similar settings to different cards, rather than 4x on one and 6x on the other, even if it isn't as playable on one card. At least we can see how "unplayable" it is, rather than just be told it is unplayable.

Other than that, great work!
 
I'd have to agree with the people who don't like the graphs. To me they are pretty useless
and unreadable most of the time. A simple min/max/average bar graph would be much
nicer. Also, I have no problem with "max playable settings" but I believe it should be
secondary to the "apples to apples" comparison and go from there. It would also be nice
if you'd have the timedemos for the users to download (where applicable) to compare their
comps to the reviewed unit.
 
WideScreen resolutions such as 1920x1200, or I forget the 20" Dell Widescreen, either way yea!

3dmarks but ONLY in same family (Green VS Green, and ATi VS ATi) so that eliminates most if not any driver optimizations between each other,

and IMO the graphs gotta go, too confusing to read, while I understand what they stand for (fps durring durration of playtime/timedemo) otherwise its perfect

Chilly
 
You guys and girls(?) do a great job.

Possible suggestions...

See the conclusions first.
Have an enthusiast *Red team* writer for new product releases (this is not a criticism, I'm *Green* myself). No spin doctors please...
D3v01D idea of wide screen resolutions is almost crucial for today's gamers.


Thanks for everything.
 
I think you could improve the summary 'max playable settings' by making it more graphical.

For example, you often mention x resolution is 'almost playable'.

I imagine a graph with the games and/or card at the bottom, the resolution on the left with green (playable) and yellow (arlmost playable) and red (not playable). This would be a vast improvement over the text-based chart that you have now.

I agree 100% with your methodology, however.

I would also love to see a bit more CPU scaling data included, especially X2 vs. FX55 and FX 57. We see a lot of discussion on the forums about 'bottlenecking'.

Hell, if you wanted to go all out, a comparison of 1G vs. 2G of RAM would be icing on the cake. We are seeing games like BF2 and FEAR that memory seems to make a difference in the playability of the game. Your methodology would show this better than most who use simple timedemos where memory is not a big factor.
 
I'd like to see some laptop gaming added into the mix. It's a growing market, and the GPUs in the laptop segment really don't get much notice. I have a Mobility X300 128MB PCI-e: I'd like to see how it stacks up in new games with reasonable settings. I can get an average of 45-60FPS in DoD:Source at 1680x1050 with everything on low. Let's see the effects of new drivers and updates on performance for this particular segment.
 
I'm probably going to get lynched for this, but how about performance in things other then FPS games? Not everyone playes them afterall, and while its a good benchmark, i feel some types of games possibly stress some video cards more, say like the partical effects from 100+ explosions going off at once in an RTS for example...
 
I think the evaluations are sweet but what I don't like is looking at a card being benchmarked without anything to compare it to. I think it'd also be nice if you benched with both dual and single cpu systems at the same clock/cache rates. :D
 
Hito Bahadur said:
I guess what I would like is a review to see how CPU limited a card is.....more specifically, how would a high-end video card work with a more mainstream CPU. A lot of us would like to upgrade to a better videocard but can't afford a whole new rig.

I would have like to have seen the 7800 GTX 512 in a 3200+ or similar system. I think no-one in their right mind would have this rig, but it definately would have been useful for the 6800GS review.

One of the reviews I saw today showed some solid gameplay on rigs with lower-end CPU as long as the eye-candy was turned up. Your style of review would go well with evaluating systems as I described.
Click to expand...
The main reason the high end CPU's are used is because it minimizes and cpu bottleneck to get the maximum performance out of the gpu can be seen without the cpu holding it back any. There will still be a difference with lower end cpu's but you won't get as good of a scale. If you cpu bottlenecks your whole rig, then no GPU will make any difference. I just remember reading that as the reason for the high end cpu's in one of the reviews.

I'd have to agree with the people who don't like the graphs. To me they are pretty useless
and unreadable most of the time. A simple min/max/average bar graph would be much
nicer.
Click to expand...
The graphs are the best part of these reviews. An average FPS ratting gives you no idea of how the GPU performs overall. One could blast through the whole game but have one major slow down that would bring it's avg fps down to an unrealistic score. HardOCP reviews are one of the very few reviews I really base opinions on. Everything is solid and gives you real world marks. The graphs are what make these reviews different and more credible. Getting rid of the graphs would make their reviews just as general and imprecise as all the other crappy reviews around the internet. Apples to apples is nice, but seeing how the playable settings compare and the screen shots to prove the difference in how your games will look with a given card is awesome. I wouldn't change anything except maybe a power section to show the power cusomptions.
 
You must log in or register to reply here.
Back
Top