ATI Radeon HD 3870 X2 @ [H]

Status
Not open for further replies.
Different sites different results. I don't see anything surprising there.

-Specifically The [H] was on Vista x64. DH was on Vista, unspecified 32 or 64.
-As far as COD4, DH does not clarify, as the [H] does on which texture setting it used. The fully maxed out "extra" setting would obviously stress the 256-bit bus much more than the 384-bit bus.

You're getting hung up on the numbers and not really thinking about how these sites got their numbers. General trends should not be hard to see. Don't get hung up on 5 or 10 fps here or there. What ALL reviews say is that sometimes the x2 wins, sometimes the ultra/gtx wins. So, you can't lose either way. Pick your poison and move on.

Agreed.
 
Thanks for feeding me further info, the effort is appreciated. I have now read the thread in addition to the article. Two questions remain (and no, I'm not going to register over there and needle them with questions that will undoubtedly be unwelcome):

First, what do they mean by "We play games for hours behind the scenes with new cards before we even start benchmarking. This is the best way to find or note anything unusual before we get into the indepth review testing."? If they mean the same thing [H] does by real-world testing, why do they draw a distinction between playing the game and "benchmarking" or "in-depth testing"?

Second, and this is really the reason why the first question arises at all, why is there still such a large difference in their results as compared to [H]'s on COD4? The hardware setups are not dramatically different, and yet DH is getting higher frames across the board with the same resolution, and with AA settings that [H] found unplayable on the 3870X2. Could it be because of the settings on the game options screen shot that DH has at normal? Maybe there are higher settings than normal that fell under [H]'s "Highest In-game Settings" catch-all. Or maybe it was dependent on which part of the game DH playtested, which wasn't specified.

Even if DH uses methods every bit as rigorous as [H]'s, somebody's wrong and somebody's right on this one--which I guess is the gist of what you said when you first linked to the review. It will be interesting to see how it plays out long-term. Given the 2900XT history especially.


Yep I agree something is strange, I posted my theory in this other thread someone started here about it: http://www.hardforum.com/showthread.php?t=1268095 I've been reading reviews and posting on forums all freakin day though and i'm about burned out with all this so i'm done for the night with the investigation LOL I did order one of these cards, looking forward to testing it myself. I suppose this will all flesh out eventually.

Cheers
 
Frankly, this is one hell of a risk for Kyle to take. If the reviewers are EVER caught with their hands in the cookie jar, this site will probably pull a Gamespot.

It is a bigger risk to fade into obscurity with the rest of the site's that no longer have any impact on buyer decisions in the industry. And quite frankly, integrity never has been a problem around here. It is one our cornerstones and everyone that works here knows that.
 
Timedemos are real-time and exact copies of actual gameplay (those sequences happening during the record process) when you record them. Depending on the engine (most of the new games released in the last year) these timedemos include the full physics and capabilities of the game, which is what you want to capture and then replay from both a consistency and "actual" gameplay perspective.
No, this isn't correct. Demo recording is real-time, yes, but timedemo playback is not real-time. Timedemo playback relies upon no aspect of internal synchronization nor is it fixed to any engine-wide tic rate.

For a real-world evaluation, you can't simply record a demo of gameplay and replay it. This yields consistent data, yes, and valid data from one perspective, but not the kind of data Brent's seeking. A timedemo playback may include physics calculations, but it will not include AI, sound, input, and various other aspects of game logic. While not related in any way to the video card, they are a part of actual gameplay, and they're to be factored in the results. That's the key difference.

If so, then how can you trust the results as the action will never be linear or exact each time. You cannot ensure consistency of the results using that methodology and therefore, the results are at best, an estimate of performance based on the actions of the user and game engine, not so much the video card depending on the variability of the action sequences.
No. There is correlation between two different data sets, even if the margin of error is high, as I've said before. It's typical thinking that anything other than an absolute rigid coherence to the scientific method yields unusable data, but that also describes the wrong kind of thinking. Don't try and extract too much from the graphs, because the data sets are not intended to be perfectly comparable. You should only observe the trend and nothing else.
 
Hi, Veridian3 (Stu) from Driver Heaven here. I just wanted to clarify my testing because there seems to be a bit of confusion about it.

We/I play through sections of each title and record the FPS with FRAPS. Like the guys here we spend a lot of time working out which section/level of the game should be used so as to make the results as useful as possible. (Years ago some graphics reviews on our site used timedemos etc but that was stopped long ago as we felt real world gameplay was more useful to anyone reading.)

If nothing else, actually playing through the games makes my job a bit easier... I get a feel for how well a product performs and it is much easier to notice problems with image quality or performance. Plus i think i might have to go insane if forced to watch timedemos over and over like some reviewers must do!
(To create the X2 review I would have had to watch about 528 timedemo runs!)

Also, to directly address the question about COD4 results being different on H and DH, I would put that down to different levels being used. In any game which has varied environments different cards will favour some levels to others. Unless sites start benchmarking each individual level in games no-one will get a clear picture of how each product performs. The ideal scenario is to look at a couple of site that use real world tests and then look at the results... you will get a feel for which card is faster but you should also pay attention to the lowest performance on each site... does the card that is losing still provide the FPS you need at the resolution you need? That’s probably the main question that should be asked/answered before buying a card.
 
Again, that is secondary to the evaluation. Primarily we play through several levels of the game, sometimes, the ENTIRE game even, that's what it takes, and that's what eats up the most time, and why we are the only ones to do it.

That's definitely the way to do it. For example, when the Crysis patch came out, there were posts upon posts that basically said "I ran the benchmark and gained 2 FPS!!1 :rolleyes:. OMFG USELESS CRYTEK lrn2patch!!"

I installed the patch and *played* through the entire game. At first, I was just gonna check out the first level, but the first level was way different in terms of performance compared to the rest of the game. So then I played through the 2nd stage, then the 3rd stage because it seems to be the most intensive. Before I knew it I was at the last stage. My conclusion: Improved perfomance across the board. It doesn't take numbers or graphs or a single canned benchmark to come up with that.

It's amazing how people use benchmarks to determine performance. I ran 3Dmark06 once in my entire life, and concluded that it was the biggest waste of time ever. All I got out of that was some numbers. No involvement, interaction, fun, or any kind of satisfaction. I can't imagine myself running that damned thing over and over with each driver release just to have some numbers spat back at me. Some people even go so far as to say "I installed this driver and lost 300 marks, back to X driver!" I guess these people look for the best drivers to run 3Dmark06 on because thats the ONLY thing that's guaranteed.
 
Hi, Veridian3 (Stu) from Driver Heaven here. I just wanted to clarify my testing because there seems to be a bit of confusion about it.

We/I play through sections of each title and record the FPS with FRAPS. Like the guys here we spend a lot of time working out which section/level of the game should be used so as to make the results as useful as possible. (Years ago some graphics reviews on our site used timedemos etc but that was stopped long ago as we felt real world gameplay was more useful to anyone reading.)

If nothing else, actually playing through the games makes my job a bit easier... I get a feel for how well a product performs and it is much easier to notice problems with image quality or performance. Plus i think i might have to go insane if forced to watch timedemos over and over like some reviewers must do!
(To create the X2 review I would have had to watch about 528 timedemo runs!)

Also, to directly address the question about COD4 results being different on H and DH, I would put that down to different levels being used. In any game which has varied environments different cards will favour some levels to others. Unless sites start benchmarking each individual level in games no-one will get a clear picture of how each product performs. The ideal scenario is to look at a couple of site that use real world tests and then look at the results... you will get a feel for which card is faster but you should also pay attention to the lowest performance on each site... does the card that is losing still provide the FPS you need at the resolution you need? That’s probably the main question that should be asked/answered before buying a card.

Thank you Stu for taking the time to explain your results. We like to hear both side of the story here too. It also nice to hear it from the person that did the testing ,instead of hear say.
 
Thank you Stu for taking the time to explain your results. We like to hear both side of the story here too. It also nice to hear it from the person that did the testing ,instead of hear say.



QFT The good guys have to stick together. :eek:
 
I appreciate anybody who is willing to go to the other side to explain himself and his way of testing.
 
Only one person seemed to think the INQ not including [H]'s 3870 x2 review with the others was significant.

I dunno, but I get the feeling its some sort of spite directed towards [H], maybe because they didn't follow the rest of the sheep with canned benches.
 
I only have one small bone to pick...

...That is disconcerting when you consider we are dealing with 640 stream processors on the Radeon HD 3870 X2 versus 128 stream processors on the GeForce 8800 GTX...

Now come on now guys! I expect you to be smarter than that, please. Any half wit should be able to figure out that what ATI calls a "Stream Processor" is different from what Nvidia is calling it. Specifically in ATI's case, they are counting multiple segments, that are capable of doing different things at once if the code supports it (which it almost never does....) as "individual stream processors". So if you do the math you discover that this ATI card with 2 GPUs has the same number of "real" SP as the 8800GTX (640/5=128).

What's impressive is that they appear to be getting close to 90-95% performance increase out of this implementation of Crossfire. Typically Crossfire, and SLi, see a performance increase in the range of 40-60%. Perhaps because they are in such close proximity their connection is working more efficiently?

I don't know, but I'm glad to see something approaching an alternative here... I'm just not sure if I want to try this out, or not.
 
Another good review from the [H]. And still more people complaining.:rolleyes:

Anyway, Great job Brent and the gang...


Ply
 
I think [H] should do a page of canned benchmarks, then a page of real world benchmarks. Just to make everybody happy. :)

:eek:

or not.
 
Can someone explain something a bit more to me.

I understand the "real" gameplay thing vs using cut scenes or timedemos/flybys included with games.

What would be the problem however, with recording a custom time demo (of the real intense stressful stuff) of the first run through on video card A, and then playing it back on video card B?

Is playing back a timedemo just not the same kind of stress on a video card as playing it in real time or something?

It would appear to me, not knowing any different, that this would allow for use of real gameplay testing, while eliminating the variable of different run throughs on different cards.
 
Can someone explain something a bit more to me.

I understand the "real" gameplay thing vs using cut scenes or timedemos/flybys included with games.

What would be the problem however, with recording a custom time demo (of the real intense stressful stuff) of the first run through on video card A, and then playing it back on video card B?

Is playing back a timedemo just not the same kind of stress on a video card as playing it in real time or something?

It would appear to me, not knowing any different, that this would allow for use of real gameplay testing, while eliminating the variable of different run throughs on different cards.

Considering that the rig with Video Card A is doing everything impromptu and that the rig with Video Card B is just following a demo that is scripted, I say that there is a world of difference. Think of it like this, a member of the speech club gives off a speech impromptu. He has emotion, he has charisma, but he makes grammatical mistakes and stuff. Teacher gives him a B. Member of the football team (this is just an example and completely in humor as I am not calling football players idiots) recites the same speech with the script in front of him. He is dull, boring, and uninspiring, but because he didn't mess up, he gets an A. In a game, when humans play we do things the computer does not anticipate. When was the last time you seen a bot teamflash in counterstrike. When was the last time you seen the same conditions again and again in a real modern PC game? Things always happen differently, and how a video card adapts and renders each scenario at said textures, frames, and resolution greatly affects my esteem of the video card itself.
 
Only one person seemed to think the INQ not including [H]'s 3870 x2 review with the others was significant.

I dunno, but I get the feeling its some sort of spite directed towards [H], maybe because they didn't follow the rest of the sheep with canned benches.

I emailed INQ about that; they replied (they are very good at getting back to me, so kudos to them) and said they didn't intentially just use "choice reviews." Seems weird to me that they could forget about HardOCP, but I'll give them the benefit of the doubt - there are a lot of review sites out there, and I'm sure they read through them pretty thoroughly before they post em on their site. So it is possible they just didn't get to it yet.
 
As for cursing , its not my cup of tea . You attract more flys with honey than viniger.

flies.png


Sorry, I couldn't resist :D Besides, this thread could use a touch of humor :p
 
Only one person seemed to think the INQ not including [H]'s 3870 x2 review with the others was significant.

I dunno, but I get the feeling its some sort of spite directed towards [H], maybe because they didn't follow the rest of the sheep with canned benches.

The INQ is quite pro AMD from what I've seen... ;)
 
We are used to it. :cool:

Why not do something about it then?

Your methodology and passion are both commendable, but listening to all the feedback this article has generated and judging by the fact that numerous 'review lists' are omitting the [H] review (the inq and dailytech come to mind) should be an indication that you might want to start listening too! Consider providing some baseline comparison (read: 'canned benchmarks') graphs in addition to your subjective analysis.. or at the very least some more insight into your methodology...


I don't think most of your devout followers would see any harm in either approach, but the vast majority of the people who have a problem with the way you do things would be either shut up or at least convinced you didn't receive defective hardware or use bad drivers (or worse yet, receive kickbacks from nvidia!)


Basically Kyle, without a basis of comparison to the other sites on the net, your review has totally alienated itself. I realize you're somewhat on a personal crusade to prove to the world that these 'canned benchies' can be deceiving, and that's great, but no one is taking you seriously because they don't see anything at all that's consistent right now with what everyone else is saying. I have some ideas though.. I don't want to just sound like another ranter because I too find great value in the no-holds-barred approach to getting down to the bottom line.. how do these cards PLAY? So... here's my basic thoughts..

I suggest keeping your reviews 90% the same as you do them now.. just make these minor adjustments:


1 - Try to keep different opinions in mind when you make your 'maximum playable settings' preference. Some people prefer FPS to IQ.. so maybe one without any AA and one with AA would suffice. It's great that you prefer to turn on AA when you spend a lot on a video card, but some people spend a lot so they can get the maximum FPS.. especially if they're heavily into multiplayer. Take it from a long-term competition-level FPS player.. FPS is EVERYTHING when you play competitively.. IQ be damned. Also consider running a few different resolutions.. a lot of people have 1900x1200 monitors, but a ton also have 1600x1200, 1680x1050, and even 1280x1024. Those users aren't going to be running @ a high res, but could definitely benefit from upping IQ or getting decent FPS in crysis :)

2 - I think I'm in the minority here, but I find huge value in your "play test graphs". I think these should remain but should be very well explained.. perhaps with commentary as to what's going on when there are significant changes ("Here we encountered a large number of enemies", "Here we went outside", "Here is the beginning of the last level", etc). Naturally in every review you need to make sure this graph is explained as being a direct plot of one unique gaming session at particular settings.

3 - Don't put up numbers and comparison graphs for different cards at different IQ settings. This confuses people.. make your subjective analysis and then make a separate (and less emphasized) apple-to-apples numbers only comparison. This will prove to people that your setup is fine compared to the other sites on the net and also will go a HUGE way to showing exactly what you're trying to show.. that those numbers don't necessarily help the bottom line - gaming performance. There should be at least one or two of these though for each game you're testing.. not just one game.


Granted, this is your site and you're welcome to do as you please, but I REALLY think if you could add the things I'm suggesting and take a little less time in learning each game you review so well, you might even gain enough time to review more games and more cards (consider adding new cards cumulatively to save time) and your reputation and credibility in general would skyrocket... not to mention the public's awareness of any real discrepancies between benches and in-game performance.

The ideas already mentioned to make a separate article specifically addressing any problems with real-world gaming performance compared to canned benchmarks are good too.. a substantial article on this subject alone with lots of good examples would be a fantastic way to usher in these other new methodologies.

Just some thoughts from a fellow enthusiast. Keep up the great work!
 
Found some pricing, keep in mind pricing might change over the next few days, or weeks.
From the Canadian site NCIX.com
(in CAD)
HIS 825mhz $494.99
Diamond 775mhz $479.99
ASUS 825mhz $499.99 (NOT the infamous 4xDVI one).
 
Can someone explain something a bit more to me.

I understand the "real" gameplay thing vs using cut scenes or timedemos/flybys included with games.

What would be the problem however, with recording a custom time demo (of the real intense stressful stuff) of the first run through on video card A, and then playing it back on video card B?

Is playing back a timedemo just not the same kind of stress on a video card as playing it in real time or something?

It would appear to me, not knowing any different, that this would allow for use of real gameplay testing, while eliminating the variable of different run throughs on different cards.
Considering that the rig with Video Card A is doing everything impromptu and that the rig with Video Card B is just following a demo that is scripted, I say that there is a world of difference. Think of it like this, a member of the speech club gives off a speech impromptu. He has emotion, he has charisma, but he makes grammatical mistakes and stuff. Teacher gives him a B. Member of the football team (this is just an example and completely in humor as I am not calling football players idiots) recites the same speech with the script in front of him. He is dull, boring, and uninspiring, but because he didn't mess up, he gets an A. In a game, when humans play we do things the computer does not anticipate. When was the last time you seen a bot teamflash in counterstrike. When was the last time you seen the same conditions again and again in a real modern PC game? Things always happen differently, and how a video card adapts and renders each scenario at said textures, frames, and resolution greatly affects my esteem of the video card itself.
what's stopping you from re-running the script on Video Card A? surely now it should be comparable to Video Card B, since now they both follow the script :confused:


btw Kyle, maybe a "AMD's ATI Radeon HD 3870 X2 Canned Version" outta help prove your point and quiet many of these conflicts
 
I've decided to retract my post as some looking through other posts it seemed I was rehashing old crap. Better to leave the horse alone.
 
what's stopping you from re-running the script on Video Card A? surely now it should be comparable to Video Card B, since now they both follow the script :confused:

Time demos are not the game, that is the entire point. It may be a "fair" comparison, yes, but it is not necessarily representative of actual in-game experiences.
 
Let me first put up a disclaimer-I am in no way trying to be disrespectful-I'm simply stating how I see things.

With that said-something stinks with these (Anand, Hard, Tom's) reviews. Someone, I'm not sure who, isn't being honest. I tend to side with HardOCP because I spend most of my time here, but who knows, I could be wrong.

But the bottom line is the results are not lining up. I understand that there are some basic hardware differences, but I don't think they'd make a night and day difference.

For example:
(images)

Now, I notice that Anand is using 32 bit Vista, while Hard is using 64 bit. Maybe this is the difference?

Or, it may be that both Tom's and Anand are using quad core processors instead of dual core?

Kyle and crew-I'm thinking your way might be the correct one, so don't crucify me please.

To the Anand and Tom's crew-I know you guys read these forums-throw your $.02 in.

The difference is presumably because the test anandtech and tom's used was getting fps rates from a pre-recorded demo / cutscene instead of averaging out FPS from real gameplay.
 

kinda, but right there. in every review, on every page with results.

note that most respectable sites explain how they get their numbers on every page too.. but when you're one of the only sites on the net doing things a certain way, you HAVE to go out of your way to explain why your results look different and how you got them.

the 'disclaimer' at the end of [H]'s review is a nice attempt at this, but it should be on every page, accompanying every graph... and personally I think it should be contrasted to the more readily accepted canned scores to show any discrepancies.
 
Forgive me for not plunging into the benchmarks/FPS/apples-to-apples/Nvida vs ATI debate(s0 w/ y'all, but I need something else on the 3870x2. What about HDMI? What about the onboard sound? I saw on one screenshot of the new 3450 (quite possibly the best new bargain for HTPC) a 10-pin sound connector so one could connect front panel audio connectors to the 3450, but I haven't seen it on any of the 38x0 cards. Running one cable up to my large flatscreen TV, playing a Blu-Ray movie @ 720 and then flipping over to some TF2 @ 40"+ and being able to plug in the gaming headset (w/ a big long coily cable, natch') sounds pretty sweet. Any results running HDMI? Any thoughts on the on-board sound? I kinda liked Tamale's suggestions to set the [H] reviews apart (constructive criticism is always good), and, I'd like to add that plenty of cards (new & old) do more than just game, and are connecting to more than just 22" widescreens, so how about at least a few paragraphs on all the other stuff?

With all due respect. I rarely let a day go by w/o browsing the [H] news.

Luke
 
kinda, but right there. in every review, on every page with results.

note that most respectable sites explain how they get their numbers on every page too.. but when you're one of the only sites on the net doing things a certain way, you HAVE to go out of your way to explain why your results look different and how you got them.

the 'disclaimer' at the end of [H]'s review is a nice attempt at this, but it should be on every page, accompanying every graph... and personally I think it should be contrasted to the more readily accepted canned scores to show any discrepancies.
No. That's a terrible idea. If someone reading an article needs to be reminded ON EVERY PAGE how the numbers were recorded, especially when [H] does it the same on EVERY one of their reviews...
Well, no one really cares what such a person thinks, as they probably cannot remember what was on the last page they read. If you (not YOU, but a general you) can't read the whole article...then your opinion means nothing anyway.
 
Why not do something about it then?
I suggest keeping your reviews 90% the same as you do them now.. just make these minor adjustments:


1 - Try to keep different opinions in mind when you make your 'maximum playable settings' preference. Some people prefer FPS to IQ.. so maybe one without any AA and one with AA would suffice. It's great that you prefer to turn on AA when you spend a lot on a video card, but some people spend a lot so they can get the maximum FPS.. especially if they're heavily into multiplayer. Take it from a long-term competition-level FPS player.. FPS is EVERYTHING when you play competitively.. IQ be damned. Also consider running a few different resolutions.. a lot of people have 1900x1200 monitors, but a ton also have 1600x1200, 1680x1050, and even 1280x1024. Those users aren't going to be running @ a high res, but could definitely benefit from upping IQ or getting decent FPS in crysis :)

2 - I think I'm in the minority here, but I find huge value in your "play test graphs". I think these should remain but should be very well explained.. perhaps with commentary as to what's going on when there are significant changes ("Here we encountered a large number of enemies", "Here we went outside", "Here is the beginning of the last level", etc). Naturally in every review you need to make sure this graph is explained as being a direct plot of one unique gaming session at particular settings.

3 - Don't put up numbers and comparison graphs for different cards at different IQ settings. This confuses people.. make your subjective analysis and then make a separate (and less emphasized) apple-to-apples numbers only comparison. This will prove to people that your setup is fine compared to the other sites on the net and also will go a HUGE way to showing exactly what you're trying to show.. that those numbers don't necessarily help the bottom line - gaming performance. There should be at least one or two of these though for each game you're testing.. not just one game.


Granted, this is your site and you're welcome to do as you please, but I REALLY think if you could add the things I'm suggesting and take a little less time in learning each game you review so well, you might even gain enough time to review more games and more cards (consider adding new cards cumulatively to save time) and your reputation and credibility in general would skyrocket... not to mention the public's awareness of any real discrepancies between benches and in-game performance.

The ideas already mentioned to make a separate article specifically addressing any problems with real-world gaming performance compared to canned benchmarks are good too.. a substantial article on this subject alone with lots of good examples would be a fantastic way to usher in these other new methodologies.

Just some thoughts from a fellow enthusiast. Keep up the great work!


Someone give this guy a medal. I was just about to further explain his post but he completely nailed it and it would just be redundant.

I've always liked hardocp's new methodology since they first introduced it - minor problems and all. While I haven't been keeping up in the last year on tech/computer news but clearly there's some backlash to your testing methods. Using the points Tamale made in future reviews, especially creating an article about why you test this way, would go a long way in helping turn the tide towards more realistic real world testing instead of alienating the site.. which for whatever reason it seems to be doing.
 
kinda, but right there. in every review, on every page with results.

note that most respectable sites explain how they get their numbers on every page too.. but when you're one of the only sites on the net doing things a certain way, you HAVE to go out of your way to explain why your results look different and how you got them.

the 'disclaimer' at the end of [H]'s review is a nice attempt at this, but it should be on every page, accompanying every graph... and personally I think it should be contrasted to the more readily accepted canned scores to show any discrepancies.

Er... why? The [H] review already has an order of magnitude more information and written text than any of the other sites. Anand's page on CoD4 had more graphs than sentences, and only a very quick, very brief, very hard to miss note on there that the results are from a freaking cutscene! By comparison, the [H] review has 6 paragraphs of text on their CoD4 page. And you have the nerve to say that they need MORE to bring them in line with other sites? WHAT?

Oh, and they even mentioned the sharp hit in performance from grenades:
In the graph you will notice the framerate declines sharply toward the end of our run-through; this happened while some grenades that were thrown right in front of us were kicking up a lot of smoke. It seems that the smoke from explosions like this can cause a large slowdown in performance at these settings tested. We didn’t see this happen as severely with the GeForce 8800 GTX.

I'm beginning to wonder if you read the same review I did...
 
... What about HDMI? What about the onboard sound? ...
Luke

Holly CRAP!!! An actual relevent question!!!!

Unfortunately this thread has become the "Testing Methodology Debate".

I hope [H] creates a thread that lets us chat about the 3850x2 and bans all other nonsense.
 
Status
Not open for further replies.
Back
Top