How Far We Have Come

Svperstar said:
I like HardOCP and I appreciate what you guys try and do with the reviews but I usually skip them and go right to apples to apples or I just don't read them anymore.

Why? The way you guys "cheat the cheaters" is to just find what settings and resolution makes the cards you are reviewing score about the same, and call it good. That is very subjective. I like how you guys don't just give a bar chart and instead use a graph that shows the Min/Max framerate but I don't like how you arbitrarily just put different settings on a graph and call it a day.

Also the message on the apples to apples section sounds very condescending:


As if those that want to see what two cards score at with the same resolution and AA are ignorant.

I just prefer to make up my own mind, please give me the graphs you guys use, I love the Min/Max numbers, but putting 1024x768 no AA with Card A and 1600x1200 4x AA with Card B then declaring a winner makes little sense.
Click to expand...

First off, there is NOTHING arbitrary about our testing at all. So if you think that is the way it is done, then I can fully understand that our methods have ZERO value to you.

It might sound condescending, but it is fact in my book.

I appreciate your thoughts here, but I find them to not really be grounded. And I don't really understand you last paragraph at all. Sorry. :(
 
I think he takes issue with definining what settings each video card is able to use and still run the game playably, because it results in an "apples to oranges" comparison, in the sense that the numbers weren't obtained by using the same settings on each card.

But, you've already clarified why you do that. It all depends on your sensibilities, I suppose.
 
What it all really comes down to is that trying to set cards by the "same settings" is pretty much impossible these days because the settings are no longer "the same". So many people are still hung up on wanting apples to apples, but some of the apples have mutated into oranges, things just aren't the same anymore. And that wouldn't really tell you high you could set the quality settings in each game anyway's.

Our methodology is that of trying to re-produce what a gamer does when they play a game.

Example, I just purchased Serious Sam 2 this past weekend. I installed the game onto my computer, the first thing I did was go into the graphics settings and set the game levels at a level I thought would play well. Turned out for the graphics card in my computer I had to lower some settings. I ended up having to customize the graphic settings, the resolution, the AA and AF levels in-game to find the best balance of performance and image quality with my video card.

This is what we do in our reviews. By using the same test system we can then compare the graphics cards and see which one let us turn on more game features and raise the resolution, AA and AF higher, obviously the card that allows this and with higher frame rates, is the stronger card for that game. We can then step back and look at what kind of gaming experience is being offered from both cards in terms of framerate and image quality.
 
It seems to me that the [H] policy is based on two levels of trust --one where there is none at all, and the second where there is trust as an article of faith.

The first, is in anything going on that isn't directly tactile or in your face --don't care, don't wanna know, they're making sausages down there and we all know you do NOT want to see sausages made if you ever want to be able to eat them again.

The second is a faith in being perfectly attuned to the gaming community. I think the [H] staff, including Kyle, thinks that not only are they primarily gamers with really cool jobs, but that they spend all their professional time and a goodly bit of their personal time staying in close contact with a broad range of gamers, and understand/connect to the community at a bone-deep level --that something nearly mystic --[H] channeling the community-- happens when they sit down to play with a new card.

In this model, writing the review is "simply" (was that Brent choking I just heard?) communicating to the gamer what he would have got for himself if he'd been sitting there.

It's not that I don't think there is value in that approach --there is, particularly as one viewpoint in the smorgasboard that is available to anyone who bothers to look on Review Day. Personally, I love the FPS graphs and always stop by to look at them.

But aside from concerns over the value of synthetics (feh, let's not beat that horse again here and now; no one is going to change their mind), there is actually a third "trust" involved in which [H] also has perfect faith --the trust that the community shares their faith in point two above.

No doubt [H] would say (and Kyle seemed to be saying in the article) "we have our record; we're proud of it --and the record suggests that our faith in trust #3 is fully justified."

Perhaps, but it does make for a goodly amount of churning and speechifying from all involved. And, personally, more trust than I can give anyone.
 
Dillusion said:
Good article, but not much more to say than what you stated.

One thing to note, though: Hasn't nvidia ALWAYS been on top less the FX series? ATi's gem against nvidia was the 9700 series and prior to it, ATi was struggling- and now in the post-9700 era, ATi is back to struggling...
Click to expand...

Even if that is true; aren't you joyous that ATI put out the 9700 series? Just think how long we would of had to dabble in the crappy fx line before Green Team got off their butts and made the next leap [fix]. The 6 series was a savior for them and now they have taken it to the next level with their non-paper launch 7 series!! WOOHOO!!

We need competition in this market or the consumer is gonna get bogged down. Technology in this market has accelorated because of the competition. So be glad.

P.s. I like the way [H] does reviews. I feel like my money/ interests are always represented first. Good job Kyle, I like the direction you are steering the [H]
 
I think the change in reviewing standards mirrors the evolution of the video card technology. FPS used to matter more because you didn't know if your card was going to be able to render enough FPS to avoid a "slideshow". Now, most cards can easily put out 30 FPS minimum for your favorite games, the question is- 30 FPS with what IQ and Eye Candy?

It used to be that when people asked "why does 100 FPS matter?" they were ignored or chuckled at. It's nice to see the trend move to image quality and playable settings. I like the reviews here- yeah, it makes me think more because often it's not "apples to apples" but it does asnwer the question "how will it look?"




PS. If this sounds stupid, then I'm blaming my 101 F fever. Can anyone spare some heatsinks for my head?
 
great article. it's always nice when someone explains the methods used to obtain the results. I have to say the argument on syn/real benchies is something that should really be left with the !!!!!!s cuz what is a !!!!!! other then someone looking for 1-upmanship on the others :D

However, I think driver tweaking and optimisation for particular games by a vendor is not such a bad thing. If there is a patch to make game D run better on hardware A then hardware N you can bet I will use it. I like to think the state of the industry is now at a point where I can buy something and get acceptable performance out of it. [H] reviews more then shows that. Should I need extra oomph then I will go get the optimisation as per vendor.

This whole issue reminds me of when Mechwarrior 2 came out. The game was brilliant on it's own (for that time). Then 3dFX and Matrox released patches that totally upped the ante in terms of fps and IQ specifically for their cards and if I remember correctly the others in the market began doing the same right after them.

Kyle and gang have done a great job in showing us what the hardware can do, I think if people take issue with optimisation being added as a later release that smacks of !!!!!! thinking. On the filp side it's always fun to see the !!!!!!s arguing over this :p

edit - how come "fan boys" has been changed to '!!!!!!' ?
 
About utility's from the game makers themselves...I think it'd be wonderful if they gave your a graph like in the [H] reviews. Select a point you want to start from, select one you want to end. It'd be of your actual gameplay exeperience, easily saved to game folder, and you can analyze it afterward. Hell, maybe even thirty seconds it could take a mini screenshot and place it below the graph, showing where you were up to at soandso time, where your framerate might be spiking in either direction. Then say it was shader heavy where you had the problem, you can easily change your options to allow for a smoother play area next time. Well, at least I'd appreciate that. Possibly make it easier on reviewers too.
 
The following statements are just off the cuff remarks, having just read the article and being as totally unbiased as possible:

Kyle is right.

I've been regularly and browsing websites for tech info for a few years now. My bookmarks folder is full of different websites which can easily tell you how well a video card performs a loop of AquaMark whatever and run loops of a time demo of the brand new game that we are all currently gawking at but bored to tears when we play it. Finding out that most of these reviews are based off of not a singe minute of actually playing the game is probably about as disheartening as finding out how those movies I want to see seem to get the same quoteable quotes from the same practically-no-name-syndicate-or-news-wire author of one review.

This is the reason why most gamers on the planet are more than happy to use an X-Box and willing to pay for an X-Box 360. One console lasted five years and brought a lot of entertainment, and another one is about to show up and probably bring even better entertainment for another five years.

Granted, we, the 1%, will never settle for being "last-gen" and will always seek bigger and better, and that's how nVidia and ATi get away with "cheating the benchmarks" or "game optimization"; they are reliant on our egotistical one-up-manship to constantly upgrade to "the next best thing" and that we don't care how we get there. nVidia's SLI website is an ironic showing of this very fact. At the top left hand corner, you can see what the fastest benchmarked systems there are from the idiots who waste their time using phase-change cooling to make their scores go into the quintuple digits. If you read the "Ask Mad Mod Mike" Q & A's, you'll read that whoever writes the responses in the column tells us just that; those are guys who are just trying to improve their scores and not actually playing the games.

These people and their benchmark scores should have nothing to do with how graphics cards are sold or produced.

This is about gaming, and loving every minute of it.

If one 7800 GTX or X1800 XT will fill my monitor with the pretty colors I've come to expect, then that's good. I don't need two. I don't need to know how well it performs in a looping rendered environment. I don't care. So long as the game looks good, and I'm having fun, nothing else matters.
 
geo said:
It seems to me that the [H] policy is based on two levels of trust --one where there is none at all, and the second where there is trust as an article of faith.
Click to expand...

You're right, but you could have dumbed it down for the masses better. Not everybody is a third year poly-sci student who just finished polishing thier bong. I’m kidding, please don’t go ‘che on me.

I bought a new computer after playing the FEAR demo because I thought my old machine was hobbling, just to find that my new one is also. That's immensely frustrating. The comment about the 20" LCD rings true. FEAR runs like a dream at 768 but sucks at 1200. It's the monitor that made me this way.

Although I buy my hardware after lots of research, I think that I can gauge the upgrade fever based on the charts - and INCREDIBLE FORUM FEEDBACK - that the [H] provides. I still run synthetic benchmarks to check out my overclocks: just yesterday I used nbitor to freak my card and I earned 1000+ more 3dMark05's. That's worth it when I'm testing my own stuff...and irrelevant when I look at other people's gear. I think that distinction should be made: not all synthetics are evil. They serve their own purpose, at least for me. I get a boner sometimes.

It is those who would dichotomize the two who are to blame for this debate in the first place. Real world performance belongs in external comparison (which *you* do), while the synths do a pretty good job in telling me where I stand now, as opposed to five minutes and a BIOS flash ago.

I lean on the [H] because the government filter doesn't pick it up and I respect it for the comparisons. Kyle, I really think it's cool that you open this up for discussion and maintain an active role, I'm butt kissing, but thanks.
 
As a ex-full time games tester with the task of running the games on all available cards, and from that writing the min spec. I can honestly say that playing the games on one machine at 1600x1200 4aa 8af DX9 was absolutely no more enjoyable than at 800x600 DX7 with nothing. I got the min spec as a machine that was just as much fun as any other.

I got a definition of the word "Game" from the dictionary. Take a read...

< An activity providing entertainment or amusement; a pastime: party games; word games.

1. A competitive activity or sport in which players contend with each other according to a set of rules: the game of basketball; the game of gin rummy.
2. A single instance of such an activity: We lost the first game.
3. games An organized athletic program or contest: track-and-field games; took part in the winter games.
4. A period of competition or challenge: It was too late in the game to change the schedule of the project.>

Did they miss the "the game of basketball, with a ball that looks marginally better, if you take a still photo, will be more entertaining game" ? I think not

If theres frame rate drop to a slide show then theres an issue but beyond that, I frankly have never really cared... the game is still for enjoyment at the end of the day and if I get that, then the card is a "good" card. IMO reviews of cards should include the "how much did I actually enjoy playing the game with this card" score
 
Well acra2 I think you've gotten something mistaken as they're not reviewing the game itself, but the card that runs the game. If you're looking as to whether a game is good or not then that's a different site.

That said I really appreciate the manner that they review video cards here as it's really the only site that does this. I can get the graphs anywhere, and as Kyle mentioned on a launch day they usually have tons of links to other sites that do this, so the way Hardocp reviews hier stuff now makes them a must check for any launch as far as I'm concerned.
 
well mAcOdIn, please read my comment again....

If theres frame rate drop to a slide show then theres an issue but beyond that, I frankly have never really cared... the game is still for enjoyment at the end of the day and if I get that, then the card is a "good" card. IMO reviews of cards should include the "how much did I actually enjoy playing the game with this card" score
Click to expand...

it WAS about the card !!!
 
Since I generally play at 1024x768 (since I don't have a 21 inch monitor) I like to see how much the various driver settings affect the behavior at a fixed resolution.
For example. I had to find out by trial and error where a card/driver would choke on a given setting. Plus where a card would just tank at the next resoltuion. Now maybe a higher resolution would negate the need for the setting, but since I cant play at that resolution...

The simple game based FPS scores at least could be used to extrapolate (guess in reality) what I could expect. Meaning if you tested at a higher res with the settings on, I could know a lower res would be ok. I can't infer anything when the resolution is so high and things like AA/AF are off to get there.

I agree with what you are trying to accomplish, and continue to read, daily, maybe some of us are just asking to have both the subjective and the game based FPS runs together in 1 place we can trust. Yeah I know, the time it takes makes it a PITA.

I DO look at the image comparisons to get an idea of the visual quality of a given card. SOmone above suggested maybe something like a slide show capture every so many seconds, that sounds like a great idea, showing the scene with the FPS and it gives us the quality, plus the details of "The card really took a hit due to the reflective lightinh here" . Now that souds like a real next step in how you review cards. :)
 
Eventually Kyle/Brent/whomever is going to get tired of responding to the mindless rambling about how their benchmarking methods suck. At that point, they will stop reading these threads, but continue to do their benchmarks in their own way, so all of the "apples to apples" people can just stfu and read another site. Nobody is holding your head to the monitor saying "read hardocp or die."

The bottom line is, when [H] benchmarks a game, they do what we would do. Set the games to the best settings to run it, and see what FPS they get. Then they make adjustments. Then they examine the IQ of the different cards being compared. Then they draw the conclusion: "Card A gives better IQ/higher settings @ 45fps in Game Y than Card B, so Card A is the best card for playing Game Y." Simple as that. That's all anybody needs to know about video cards.

At the same time, it is also useful to look at some of the "synthetic" tests and analyze what the synthetic results are telling you. If a card is completely choking on 3dmark07, and 3dmark07 happens to be heavily dependent upon a particular shader or something that is prevalent in soon-to-be-released games, that is a clue into what the card's performance will be in those upcoming games. Perhaps the final benchmark score can be thrown out the window, but I would say 3dmark still has a valid use as a tool, when properly used and analyzed.
 
jebo_4jc said:
At the same time, it is also useful to look at some of the "synthetic" tests and analyze what the synthetic results are telling you. If a card is completely choking on 3dmark07, and 3dmark07 happens to be heavily dependent upon a particular shader or something that is prevalent in soon-to-be-released games, that is a clue into what the card's performance will be in those upcoming games. Perhaps the final benchmark score can be thrown out the window, but I would say 3dmark still has a valid use as a tool, when properly used and analyzed.
Click to expand...


What I like about synthetic benchmarks is that you can see the 'growth' of your chosen card with them, - As you are tweaking the core/mem of a video card, for example, your 3dmark05 score can go up.

Many many many games will "just not get any faster" - or are hitting some other sort of wall (as you can see in tests where 10 cards all stop at 170fps..)


If you compare the 3dmark05 benchies at THG

http://graphics.tomshardware.com/graphic/20050524/vga_charts-02.html

Then look at the unreal tournament benchies there..

http://graphics.tomshardware.com/graphic/20050524/vga_charts-03.html

If you went by "video card performance in game X" rule, you would be led to believe that a single 6600GT (160.2 fps) or a radeon x700pro (153 fps) are JUST AS GOOD, as the 6800Ultra SLI, (160.7 fps) or the radeon x850xt-pe (162.6 fps)

Since they are all so close to each other, wouldn't you then be the smart man for getting an x700pro for $150 instead of an x850xt-pe for $450?

However, with the "synthetic" benchmark, 3dmark... - You see that those cards DO NOT perform ANYWHERE similiair x700pro = 1451 vs x850xtpe = 3402.

Without having 25 cards in hand, it's hard to know that the game of unreal tournament (or quake4, or fear) is not cpu limited, frame locked, or whichever.

3dmark (and other synthetic benchies) while they may be able to be tricked, and are also *sometimes* inaccurate, tend to give a good part of the picture that is missed with the reviews here (and for that matter, at THG...) What I do like is that THG, unlike [h] gives "all" of the details.

It also doesn't matter to me what they, [H], got as the best "playable" resolution. I think it would be nicer to see where card X lags versus card Y..

lastly.. When I see benchies that say .. "Card X averaged 39.4 FPS, and Card Y averaged 38.7 FPS, so that means that Card X is the OMFGTIIYMBIOYSD" (Oh my Farking God, this is incredible, you must buy it or you should die) card to have.. I just laugh.

Or, when it's 150fps versus 145fps.. and people claim the first card is "Vastly superior"

It just makes me want to publicly laugh at them and make them wear a pointy hat with bells on it.

[look at a deck of playing cards if you don't get the reference]

I dont' care if ati or nvidia have the better offer, I'll use either. But I don't like a benchmark that I don't find informative. Just because that ati card could only do 1280x1024 on *your* system.. it doesn't mean it can't do 1600x1200 on mine [not that I ever game at 1600x1200....]
 
Svperstar said:
Well basically what I mean is when you guys run the game and decide for us what playable settings are, and whatever you pick is then added to a graph, if it is 1280x960 4xAA for Card A, then 1024x768 no AA for Card B, then so be it.

I don't understand how I am supposed to get anything out of that, I appreciate that you try something different but it doesn't work for me.
Click to expand...

I appreciate your feedback. I am sorry our format does not work for you. I am unsure as to why you don't understand how HardOCP showing you which card will provide better IQ and playable framerate is of no value to you, but I can accept that. Make sure you check the news page for other resources. :)
 
Malogato said:
lastly.. When I see benchies that say .. "Card X averaged 39.4 FPS, and Card Y averaged 38.7 FPS, so that means that Card X is the OMFGTIIYMBIOYSD" (Oh my Farking God, this is incredible, you must buy it or you should die) card to have.. I just laugh.

Or, when it's 150fps versus 145fps.. and people claim the first card is "Vastly superior"
Click to expand...
I don't know if this is what you meant to imply, but I sure don't think [H] has ever drawn conclusions like this. If card X & Y are very similar in avg fps, min fps, and IQ, they will just come out and say, "both of these cards are good for this game." Also, I think the 145vs150fps days are gone with the Q3 engine. Fact is, Source, D3, and other modern engines will keep most cards below 60fps at applicable settings. That is also part of why [H] uses "best settings only" instead of benching at 1024x768 no AA no AF.
Malogato said:
I dont' care if ati or nvidia have the better offer, I'll use either. But I don't like a benchmark that I don't find informative. Just because that ati card could only do 1280x1024 on *your* system.. it doesn't mean it can't do 1600x1200 on mine [not that I ever game at 1600x1200....]
Click to expand...
I don't think your system will be blowing a reviewer's system away, since most VC reviews I've ever seen use a high end A64 or FX CPU. You're getting into a whole other argument here.
 
Damn you Kyle! Your waxing nostalgic in the editorial also got the gears in my head spinning on gaming days way back when... And I mean way back too -- back to the days when I was "cutting my teeth" on a TI-99/4A, playing games on it and actually doing some very very light programming (ie. copying lines of code from a magazine hehe :p ). Also had me thinking how much I enjoyed my Commodore 64 and my Amiga 1000.

You are very much indeed right, we have really come a long way from the software-rendered games to the new whiz-bang hardware accelerated eye-candy fests of today. Some of my favorite games are old school games -- the kind that any teeny-bopper would look at and puke over.

Thanks for the jog down memory lane! Great editorial, don't pay too much heed to those who don't like the way you and the crew do things. For them, there are less accurate websites out there who aren't so willing to throw their stuff into the wind, so to speak. ;)
 
UncaMilty said:
Keep in mind also, that the fixation on numbers and driver or benchmark cheats is far older than 3D. I can remember when PC Magazine revealed that some video card companies had cheated on their 2D benchmarks by hard-coding a text string in the video chip in order to massively inflate their score in one of the tests (which repeated the text string several thousand times). By influencing one portion of the test by a huge amount, the video cards in question could bump up their overall scores quite a bit.
Click to expand...
They have 2D now?

Who would ever have thought to make a 2d benchmark? Lol. Just seems funny looking back on it, doesn't it?
 
Svperstar said:
... you just refuse to run cards at the same resolution and AA settings, and they we are left to try and peice it together.
...
Basically what I am saying is bench the cards with no aa, 4x aa and af, all that jazz, at all different resolutions. Use the awesome min/max graphs you guys have, and then let me make up my mind on what are the best "playable" settings by comparing the results, instead of just putting different settings on the same graph and telling me what is playable... let me make up my mind on what are the best "playable" settings by comparing the results, instead of just putting different settings on the same graph and telling me what is playable.
Click to expand...

The basis for [H] reviews is that we (the gaming community) agree on what is a playable FPS (somewhere between 30 and 60 FPS). So, they give us side-by-side charts of the cards playing different games at the max IQ while keeping the FPS roughly between 30-60 (example). Do you think the FPS standards should be different? Or do you think the max IQ standards the [H] gives are inaccurate?


Svperstar said:
...Basically what I am saying is bench the cards with no aa, 4x aa and af, all that jazz, at all different resolutions.
Click to expand...

While I agree in theory with having this info available (under the "more info = better" principle), the new IQ options (TSAA, HQAA, ADAA, FAS-baby AA, etc) features that are brand-specific really aren't comparable in an apples-to-apples way. How can I compare ADAA between the 7800GTX and the X1800XT when only the X1800XT has it? Or that spiffy FAS-baby AA when it's only on the 7800GTX?

Also, when is anyone with a 7800GTX or an X1800XT going to be playing a current game with no AA? And when is a person with a 6200 going to be playing a current game at max detail? It just doesn't justify the time expense to do those benchmarks, because nobody is going to play a game with those settings.

-SEAL
 
well no offense, but I really wasn't sure what you were trying to get at in this short article Kyle. More than likely it has something to do with the fact that I just don't see the need for highend cards and people upgrading every six months. I used to do that, but personally I believe things have tappered off some. You don't HAVE to upgrade to have fun. The number of games a two year old card can run is much more than it was five years ago. Sure, you might not be able to play BF2 yet, but NOLF, RTCW, and just about any RTS are perfectly playable on a 9800. Not even the pro version ! So, not to be a stick in the mud I just see people upgrading like crazy is , well, crazy. Thus the benchmarking, etc are a non-issue. Just play the game and have fun by gah.
 
compulsivephreek said:
well no offense, but I really wasn't sure what you were trying to get at in this short article Kyle. More than likely it has something to do with the fact that I just don't see the need for highend cards and people upgrading every six months. I used to do that, but personally I believe things have tappered off some. You don't HAVE to upgrade to have fun. The number of games a two year old card can run is much more than it was five years ago. Sure, you might not be able to play BF2 yet, but NOLF, RTCW, and just about any RTS are perfectly playable on a 9800. Not even the pro version ! So, not to be a stick in the mud I just see people upgrading like crazy is , well, crazy. Thus the benchmarking, etc are a non-issue. Just play the game and have fun by gah.
Click to expand...
This is a hardware enthusiast site and nobody is telling you that you must upgrade to the latest and greatest. Some people will upgrade constantly, some will not. Regardless of when you do decide to, you will want to have the information available to make an informed descision on what to purchase. That is where [H] comes in.
 
jebo_4jc said:
I don't know if this is what you meant to imply, but I sure don't think [H] has ever drawn conclusions like this. If card X & Y are very similar in avg fps, min fps, and IQ, they will just come out and say, "both of these cards are good for this game." Also, I think the 145vs150fps days are gone with the Q3 engine. Fact is, Source, D3, and other modern engines will keep most cards below 60fps at applicable settings. That is also part of why [H] uses "best settings only" instead of benching at 1024x768 no AA no AF.
Click to expand...
I wasn't saying that they specifically did any of these things (Although, some are quite accurate - I see some of the performance graphs here and they tell me absolutely nothing. Or, without full comprehension, they would actually lead me to believe the wrong thing. -

"Why doesn't that brand X card get run at 1600x1200? Oh,brand X cards can't be run that high and play 3d.. Who KNOWS what people are going to think."

I personally say that it would be .. more effective .. when putting 2 cards "head to head" that what you should do is, if say, one card is obviously losing.. Drop the "winner" to the same settings (and don't give me this bullshit that apples have turned into oranges...) 1280X1024, 4xAA, 8xAF is a commonality between them.

If card A can play Quake4 at 40FPS (@ 1600x1200, 4xAA, 8xAF) and Card B can only get 32FPS (@ 1280x1024, 4xAA, 8xAF) that tells me something.

Wouldn't it be nice to know what card A can do at 1280x1024, 4xAA, 8xAF?

Wouldn't it be helpful to see Card A completely trounce Card B?

How about.. I dunno, finding a standard res, and running everything at that.. Oh, wait.. that's an awful lot like what was done by some synthetic benchmark.

Sure, throw in your other "What is the max they can hit" graph too, it would give better understanding of how things really are taking into effect.

THG's method *IS* flawed also... At least it was until they stopped comparing PCI-E cards to agp cards (cause that usually meant an architectural change - different mobo, etc..)



I don't think your system will be blowing a reviewer's system away, since most VC reviews I've ever seen use a high end A64 or FX CPU. You're getting into a whole other argument here.
Click to expand...


First off on this second part of your reply..

1.) I was giving a hypothetical. Do they use FX-57's clocked at 4ghz with 650mhz ram like some people?
There are, I am sure, many users here, I'd say.. 100s of them, even, who have better machines than what brent uses to benchmark.

2, you have no idea the resources some people have.
 
Malogato said:
Wouldn't it be nice to know what card A can do at 1280x1024, 4xAA, 8xAF?
Click to expand...
What if Card A looks better at those settings than card B? Shouldn't this be taken into consideration?
Malogato said:
First off on this second part of your reply..

1.) I was giving a hypothetical. Do they use FX-57's clocked at 4ghz with 650mhz ram like some people?
There are, I am sure, many users here, I'd say.. 100s of them, even, who have better machines than what brent uses to benchmark.

2, you have no idea the resources some people have.
Click to expand...
1.) No [H] doesn't use a phase change cooled FX57@ 4ghz, but to my knowledge, niether does any other review site. But they do use an FX-55, which is plenty to cover all but the most extreme benchmarkers anyway. Anybody with a CPU stronger than an FX-55 probably does not wait for a review to base their purchasing decisions anyway.
2.) What in the world do you know about what I "have no idea" about? I have a pretty good feeling for what even the most hardcore enthusiasts are doing with their systems, and what resources they have.
3.) What's your point with bringing up this subject to begin with? They need to vapor-cool their systems when benchmarking?
 
The thing is most people that are complaining about [H]'s review methods are not getting is- *They are doing something different!!*

You want X resolution with 4x AA + 8x AF?? There are TONS of sites with that. You want X resolution with no AA + no AF? There are TONS of sites with that. Most sites do Apples-to-Apples.

The [H] gives you yet another resource to draw upon. They focus on what you MIGHT select as your best playable resolution for each video card. If you wanted a different resolution, you could extrapolate off their data or visit the other 1233897438743 sites that have the specific fps for a specific resolution.

I applaud the [H] for adding another dimension to the video card reviews that I didn't think was possible. It is invaluable to me since I get the fps at X resolution shoved down my throat everywhere else.
 
Skystalker said:
No offense, Kyle...

LOL (that put your defenses up, didn't it!)

The best thing about [H] is that you HAVE an opinion. You back it up, and you do things differently. I can go to the other sites and get different slants on the hardware to get a balanced overview of the product.

Although I consider [H] one of THE BEST sources of information and reviews (Brent rocks, BTW), I want to read other reviews to get a balanced "feel" for the product. Anyone that trusts only one source of information will (most likely) not get the "full" picture.

You are not afraid to link to other sites. If I am not mistaken, that is unique.

Kudos.

-Sky
Click to expand...

Thanks bro, that is appreciated. :)
 
Synthetic benchmarks for almost every warp of life have never been reliable, its the nature of benchmarking - especially in an environement where the results fill or empty pockets.

It's really supprising that it took THIS long for everyone to catch on to be honest, oh well live and learn!
 
The one problem I see with showing the best playable resolution with some image quality settings turned on is it assumes you want the quality settings more than the resolution. For a CRT this is fine but on my LCD I will turn off all the quality settings I can to get it to run at the native resolution to avoid the issues with scaling. Maybe showing what resolution has an acceptable frame with all (or some) IQ settings on and another with them off would be doable. It's more work if you are not taking data when trying to find the sweet spots but just a bit more reporting if you do take data while finding that spot. I think the IQ settings can be fairly flexable between cards as the options that they support (and support well) vary a lot so I don't see an issue with this not being apples to apples all the time.

The one other thing the 3DMark/timedemo graphs could help with was figuring out CPU vs. GPU dependancy but only if several grades of CPU and GPU are used. I remember a great article on Doom3 performance here that did a great job with this. Because of the raw test time needed to accumulate the data needed to see the trends this it is really only feasable to do this with a roundup or compilation type review however. I expect this to be harder in the future as we get more IQ features and less raw power. You eyes don't really need anything faster than a steady 60 FPS update and I don't think we will see displays go above 1600x1200 for a reasonable price anytime soon. That points to higher resolution textures and other IQ effects being the places we will see the most improvements I think.
 
sxr71 said:
Yeah this is how I see it too. Everyone keeps talking about how current games can't take advantage of modern hardware and about how our hardware is so good, but I fail to see what they are talking about. In your experience you are having trouble with 1680x1050; what about people running 1920x1200 or higher? The problem is even more pronounced.

Adding to that, your defintion of acceptable framerates may be different from mine, as I'll accept even 30fps as long as it doesn't keep dropping below that. There are games now that with 4xAA and 16xAF at 1920x1200 that will scarcely allow a 7800GTX to run at a consistent 30fps.
Click to expand...

I run at 1920x1200 4 x /8x without issue. I have been watching the threads for awhile and it seems those that are dissappointed are overclocking and subsequently using relaxed ram timings. Perhaps a different approach to your tweaking is in order.

Anyway great read Kyle, I look forward to you and your team tackling the up and coming PPU technologies.
 
jacuzz1 said:
Anyway great read Kyle, I look forward to you and your team tackling the up and coming PPU technologies.
Click to expand...

Now that is going to be a challenge. That is certainly going to be a brand new area of benchmarking-- pretty exciting really.

Opps... off topic.

-Sky
 
isnt this what your editorials are about in cpu this and next month? i completley agree with you too, benchmarks say nothing of how a game feels to someone. i come from the tnt/voodoo days, and probably because of that prefer speed over alot of quality. but anyway, thats just my opinion. i remember the voodoo speed fiasco too, also the ati maxx, etc. those were days when running at 1024 (no such thing as fsaa until voodoo 4/5) were high tech..
 
acra2 said:
As a ex-full time games tester with the task of running the games on all available cards, and from that writing the min spec. I can honestly say that playing the games on one machine at 1600x1200 4aa 8af DX9 was absolutely no more enjoyable than at 800x600 DX7 with nothing. I got the min spec as a machine that was just as much fun as any other.

I got a definition of the word "Game" from the dictionary. Take a read...

< An activity providing entertainment or amusement; a pastime: party games; word games.

1. A competitive activity or sport in which players contend with each other according to a set of rules: the game of basketball; the game of gin rummy.
2. A single instance of such an activity: We lost the first game.
3. games An organized athletic program or contest: track-and-field games; took part in the winter games.
4. A period of competition or challenge: It was too late in the game to change the schedule of the project.>

Did they miss the "the game of basketball, with a ball that looks marginally better, if you take a still photo, will be more entertaining game" ? I think not

If theres frame rate drop to a slide show then theres an issue but beyond that, I frankly have never really cared... the game is still for enjoyment at the end of the day and if I get that, then the card is a "good" card. IMO reviews of cards should include the "how much did I actually enjoy playing the game with this card" score
Click to expand...

I see where you are going with this but would like to offer an alternative view. For one, the reference you make to Basketball does not really make much sense when applying it to video games. Real life images viewed through my eyes do not have aliasing or resolution. I do not have a FPS performance hit depending on what I am looking at like you do in a video game.

The real question to you would be "Would you still watch basketball if it were shown in stick figure animation versus true to life images in which there are radiant colors, sweat, emotion, and tone and definitition in the players and their surrounding environment?"


The purpose of a video game is to immerse a player in a world of situation to duplicate real life, ala simulations, or something that would not normally be possible in real life, ala Quake, Doom, WoW, etc. There is a discernable difference especially if you are talking about the simulators between low resolution and very high resolution with high FPS. Are you really going to tell me that a Flight Simmer would rather play games from the 90's with 2d Graphics versus the powerhouses available today with very crisp graphics and beautiful scenery?

One thing I definitely agree with you on is the fact that a game will be good independently of the eye candy in it. A pretty game with shitty gameplay is nothing more than a benchmark in my book. Have you even played the likes of say HL2 with full eye candy on recently? The gameplay is phenomenal to begin with but the graphics are some of the best available today when run at full tilt with a high end system capable of sustaining high frame rates.

That is the experience so many of us here are going for and the main reason why the reviews here make the most sense out of any I have read on the web in the past 6 months.

I too would love to have a rating of "how enjoyable was it to play on this card". The problem would be trying to make that objective and unbiased. If anyone can do it in the future, I am sure the [H] will.

...
 
I remember exactly where I was standing in the map the first time I saw a Quake 2 game environment at 1024x768 on my monitor, and I thought that was incredible
Click to expand...

That's so funny! Because I remember playing Quake 2 SP way back in the day, only to realize after playing a few maps that I was in "software emulation" mode and that I changed it to Hardware and after I got back into the game it was like BAM! It looked 100% better, I was amazed how good it looked. And I also remember the exact spot I was standing when I first saw that :)

Great article Kyle.
 
Frosteh said:
Synthetic benchmarks for almost every warp of life have never been reliable, its the nature of benchmarking - especially in an environement where the results fill or empty pockets.
Click to expand...
Synthetic benchmarks are useful when what they are measuring is simple and straightforward. Early on, both CPUs and video cards (and pretty much any computer component) were pretty simple and straightforward. They dealt with problems via brute-force number-crunching, so even a fairly narrow test could give some indications as to how a part would perform compared to other parts.

But over time this has changed, more from the hardware side than from the testing side. CPUs gained math co-processors, then frequency multipliers, pipelines and instruction queues, multimedia extensions, etc etc. Video cards are going through the same changes, and I think the benchmarking community has been slow to adapt. Synthetic benchmarks may still help to understand how hardware compares in some ways, but they've become almost entirely useless for comparing overall gaming performance.

But they're still the easy way to go and people have a habit of taking the easy way when it's available and they can get away with it. Thus many sites continue to run generic timedemos and report the results in review after review after review...
 
You must log in or register to reply here.
Back
Top