ATI Radeon HD 3870 X2 @ [H]

[Citizen86]

Take it to PM - Kyle

 

[FrgMstr]

If you play through a level with FRAPS running and generate those numbers, fine, but how do you do that consistently? You cannot, as the user or game engine will never generate the same results twice. Therefore, this testing methodology is not accurate and is nothing more than an estimation of performance within a single gameplay setting. If we hang our results on card A being 2FPS faster than card B with this method, then it is useless as the results will vary too much to generate meaningful numbers.

Who is to say the levels they play through are the right ones? Do they test multiplayer? I would just like to see the "actual" gameplay sequences they are using to report the numbers. Can I recreate that gameplay to see how well my system performs or do I continue to count on my seats of the pants feeling about what my system is doing or blindly trust the results. One good thing about built-in benchmarks is the ability to get 90% of the way there when it comes to setting up a system quickly or having fun comparing numbers against the systems of my friends, which usually results in a credit card being used a few minutes afterwards.

Please stop ignoring the methods and reasoning we have specifically told you. You are basing your argument on what YOU THINK is going on instead of listening to what the authors TELL YOU exactly the process is.

 

[Climber]

You don't make sense though.

I would agree with you had a way to get the same run through each time. However you don't.

I could understand using a time demo that you guys created to run through a section of the game. That way it would be the same each time. However since you don't your using two diffrent sets of data to come to a flawed conclusion.

I'm still debating whether your being willfully ignorant here or you just can't see the mean average of statistics.

When you poll a group of people, you choose from a variety of demographics to get an average. If you poll a 100 golf pros at the local country club about which sport is the best I'm sure it will differ greatly then if you were to ask 100 people down at the skate park. Real world game testing while not constant does simulate day to day situations that most games find themselves in. Synthetic benchmarks while though constant and show off the hardware capabilities does not provide the real world gameplay experiance you experiance when you sit down to play, you can at least agree on this correct?

There are pros and cons to each test, I do however believe that the methodology behind real world testing to be better then the synthetic benchmarks and the reviews here at hard generally line up with what we see as evidence down the road. For the record, I'm sporting in my three rigs a 9700 PRO, X800XT, and a 1950XT so I'm far from being an NVIDIA fan boy.

I go with the best price/performance offering at the time I need to upgrade. I'm currently building a new system and the video card/mobo were the only two things I had not yet purchased. Based on this review I decided to buy the 3870 x2 because [H] showed that the price performance was there and it was a solidly made card.

Just because the review doesn't show it knocking the crap out of Nvidia doesn't make it a bad card, the simple fact that you can't let this go screams ulterior motive to me. I held off upgrading to a 2900 a while back because of [H] review of it, Anand Tech almost swayed me , but I held off because of the review methodolgy and to be honest I'm glad I did.

Nothing inherently wrong with the way AT does their reviews, I just prefer real world benches over synthetics, I prefer that in pretty much everything I can think of.

 

[MrMike]

Thanks for the review Brent and Kyle.

Some of the canned benchmarks have as much physics and action as the rendering of a floor. Especially Crysis GPU Benchmark and that CoD4 intro. They could at least use the Assault benchmark in Crysis. I still believe real gameplay FPS recording is better, but I don't know who ever decided the GPU Benchmark gave a better indication of performance than the Assault Benchmark.

I honestly am a bit worried about ATi card performance with AA enabled possibly not being as good as it should be. I guess we can only hope for some improvements in drivers.

 

[FrgMstr]

Thanks for the reply Kyle. I do not think you actually answered my question. I fully understand you play through the game, you then decide to take a particular level or section of the game that best represents overall gameplay, setup the system, (insert my question below here), you chart that data, report it for those of us that like numbers, and then come to an objective/subjective conclusion based on how well the game played utilizing a variety of settings (resolutions, AA/AF, etc). That is all fine and dandy and the experiences commented on during actual gameplay is wonderful in my opinion.

However, my question is how do you capture the performance data to create those charts and FPS results? Do you create timedemos while playing the game, then replay them with the game engine while FRAPS is running to collect the performance data or do you (your staff) play through a particular level with FRAPS running and collect that data for reporting the numbers. If it is the latter, how do you ensure consistency between test runs, what variability do you see between each run, and are your FPS results then averaged for a composite score? If you use timedemos, then how are yours different than other sites that use the same testing methodology with FRAPS or the game engine to report results. I think answering these questions would clear up a lot of what has been said today.

NO YOU HAVE IT WRONG.

I fully understand you play through the game, you then decide to take a particular level or section of the game that best represents overall gameplay, setup the system, (insert my question below here), you chart that data, report it for those of us that like numbers, and then come to an objective/subjective conclusion based on how well the game played utilizing a variety of settings (resolutions, AA/AF, etc).

We play through the game, make notes of our experiences at specific settings then draw our conclusions. The graphed data you see has nothing to do with our conclusions. It is simply there as proof for our readers as to our experiences.

Now that I have said this for the umpteenth time, what are your questions?

 

[Climber]

If it is the latter, how do you ensure consistency between test runs,

...

Is it me or do you guys just keep ignoring everything they say?

 

[Citizen86]

However, my question is how do you capture the performance data to create those charts and FPS results? Do you create timedemos while playing the game, then replay them with the game engine while FRAPS is running to collect the performance data or do you (your staff) play through a particular level with FRAPS running and collect that data for reporting the numbers. If it is the latter, how do you ensure consistency between test runs, what variability do you see between each run, and are your FPS results then averaged for a composite score? If you use timedemos, then how are yours different than other sites that use the same testing methodology with FRAPS or the game engine to report results. I think answering these questions would clear up a lot of what has been said today.

This question was answered in this thread, even if it is buried in 30 pages of posts, it is there. Probably why Kyle and Brent are so tired of answering it over and over. The gist of it though was that the reviewer, in this case Brent, plays the game, plays it a lot, and when you know a level/area enough, even with AI making different decisions, etc. You can still run through an area and basically do the same thing. Shoot 1st korean, hide behind cover, shoot 2nd korean, run to another area, throw grenade, duck, etc etc. I believe that's what the FPS graph is there for, to show that they played the same areas. On every graph I've seen in these reviews, the FPS rises and dips in almost the same identical spots on all video cards tested. That shows that the same areas, and very similar battles/whatever are happening on both video cards.

Edit: If that's wrong then please correct me.

 

[Rock&Roll]

Wow. I can't believe how many people still havn't really gotten a grasp of the [H] methodolgy. Sounds like we need a major front page write up re-explaining things in full. I personally don't see what the mystery is, as Kyle has explained until blue in the face.

I realize the underlying differences that exist between ALL hardware review sites. In fact, I believe that non of them preach the absolute gospel. So I would advise reading everything with a grain of salt and a little discernment. I don't think your hand needs to be held to understand where a card lies against it's competition.

 

[Commander Suzdal]

Click the link that says discuss this in our forums at the end of the article


Here is the whole post, and there is some more information in that thread as well as in the review, each test is run three times per card and the results are averaged:



Thanks for the words of support today guys, much appreciated. Also be sure to check out the other reviews, we posted quite a few in the news today. I was a little disappointed with Hardocp and their stance today on their front page, stating that apparently everyone else is using "canned" benchmarks and they are the only ones who aren't.

We haven't been using premade benchmark scripts for a very long time (we also dropped 3dmark in reviews before hardocp did), stuart uses real game play. This is in fact how we noticed the minimum frame per second issue we noted (page 17 of the review), I don't think anyone else has even covered this in detail, take from that what you will! Not only that but we managed to get a statement direct from AMD explaining why. We have spent the last week working with AMD on the drivers, so I think its pretty fair to say we know what we are talking about.

That said, id rather not dwell on this, I hope our review has been informative, entertaining and educational.

As I said, I read the article. I saw the note that said they ran their tests three times and averaged the results. While that still is not specific as to how they test, it is the exact wording used to describe the canned benchmark method on most sites.

When he says, "oh no, we use real-world tests," does he mean they create their own time demos instead of using ones included with the game, or does he mean they FRAPS actual live gameplay like [H] does? There is a difference in the quality and accuracy of the results. For example, given how much higher the AA settings and framerates were on their tests for COD4 at the same resolution as [H] used, I would tend to believe that they used a testing method which did not capture all the elements of real gameplay.

 

[Climber]

On every graph I've seen in these reviews, the FPS rises and dips in almost the same identical spots on all video cards tested. That shows that the same areas, and very similar battles/whatever are happening on both video cards.

Apparently they just choose to overlook this fact. The AI doesn't chage drastically enough to even worry about this. The simple fact that I can sit down and play through a game a couple times and memorize the position and map of most levels makes this point so moot as to not even bring it up or worry about it in the least.

 

[jon67]

Must be the fastest growing thread since "Valve sucks!"...

Strange, as there is not much to discuss - there is no discrepancy between the H review and the reviews on other sites, there is only a question of different technical approach. The consumers must decide for themselves which approach is most relevant for their use.

Main X2 selling point for me is 4 DVIs out of one PCI slot, and price/performance ratio. Noise may be a drawback. Superior ultra-high resolution gaming performance is actually the last criterion on my list.

 

[bluehaze013]

As I said, I read the article. I saw the note that said they ran their tests three times and averaged the results. While that still is not specific as to how they test, it is the exact wording used to describe the canned benchmark method on most sites.

When he says, "oh no, we use real-world tests," does he mean they create their own time demos instead of using ones included with the game, or does he mean they FRAPS actual live gameplay like [H] does? There is a difference in the quality and accuracy of the results. For example, given how much higher the AA settings and framerates were on their tests for COD4 at the same resolution as [H] used, I would tend to believe that they used a testing method which did not capture all the elements of real gameplay.

It's in that thread man here I will quote it out for you:



I take it you aren't a regular, so ill break it down. We use real game play to test graphics cards, we have done so for a long time. if we used pre built benchmarks or time demos we wouldn't have found the minimum frame rate issue as the majority of sites who used these still haven't even found the issue. This is only seen with extended gameplay, or what id like to call "real life" gameplay. We play games for hours behind the scenes with new cards before we even start benchmarking. This is the best way to find or note anything unusual before we get into the indepth review testing. This is how we found issues that caused AMD to delay their NDA for a week. Issues we helped solve.

if you would like anything else explained please ask away.

Every single test in the review used real gameplay... i can confirm that as i was the one who tested the cards and wrote the article. We/I havent used timedemos, or built in benchmarks for years, and have never benched a cutscene in my life.

and heres a link to the thread:
http://www.driverheaven.net/news/153658-dh-review-ati-hd-3870-x2-2.html

 

[Runningflame570]

Specifically, does anyone have an idea how timerecorded demos might skew the results if at all? They don't exactly seem the same thing as cutscenes but at the same time I would image that it takes some of the load off of the GPU from various things (AI, physics, etc).

 

[btdvox]

Well that pretty much does it for me- After reading [H]'s review, Overclockers club review, PC Perspectives revew, extreme tech review and finalyl guru 3d's review and even Anands taken into count- this card in some certain games like Call of jaurez and Half life 2 in game will beat an 8800 ultra- but pretty much all the other games ive seen- its the complete opposite.

Take what you will- Im pretty amazed that G80's lasted this long- They are definitely the new ATI 9800...

btw. Noseguard- For calling "Nvidia Fanboys" out so much- you could be the biggest ATI Fanboy i have ever seen on the net....Please just stop and instead just debate over the cards. no need to call people out !

 

[Patman08]

ati needs to realize that no matter what they do to it R600 will never beat the G80. and get to work on the R700. mabey it'll be somewhat competive with the gt200 or whatever the next gen geforce will be

 

[Chris_B]

ati needs to realize that no matter what they do to it R600 will never beat the G80. and get to work on the R700. mabey it'll be somewhat competive with the gt200 or whatever the next gen geforce will be

Theyre already at work on r700 and have been for a while now, it was reported the chip taped out a while back and was rumoured for a late q2 release.

 

[XamediX]

WOW, 30 pages... I can make this easy for you folks whining and crying. Read Hardocp, then Anandtech, then <insert your favorite review site>. After that, take all that you read and use that goblet of information for your needs.

People who complain heavily about H's methods are just fuckin lazy. Read, you shmucks. Read more than one review site to garner your opinion about these complex technical devices.


/end thread


ps: thanks H. Because of you I bought a 8800GT for $230 and I have been happily gaming on my 24 and 22in lcds at my native resolutions.

pps: It's very funny , i know i said it many times before, but its funny to see AMD losing with AA/AF enabled. AMD is the reason H even started making a big deal about AA/AF performance. These young members that joined/lurked before the 9700 days are too young to be giviing opinions, lol.

 

[kush]

Take the water block talk to its own thread please, it has no place here discussing the review. Kyle

 

[Stanley Pain]

WOW, 30 pages... I can make this easy for you folks whining and crying. Read Hardocp, then Anandtech, then <insert your favorite review site>. After that, take all that you read and use that goblet of information for your needs.

People who complain heavily about H's methods are just fuckin lazy. Read, you shmucks. Read more than one review site to garner your opinion about these complex technical devices.


/end thread


ps: thanks H. Because of you I bought a 8800GT for $230 and I have been happily gaming on my 24 and 22in lcds at my native resolutions.

pps: It's very funny , i know i said it many times before, but its funny to see AMD losing with AA/AF enabled. AMD is the reason H even started making a big deal about AA/AF performance. These young members that joined/lurked before the 9700 days are too young to be giviing opinions, lol.

What about those of us that remember Mach64 4meg cards and Voodoo?

 

[gljvd]

The only reason [H] would need to show canned benchmarks would be to quieten down those that think there must be something wrong with the Hardware or config Kyle/Brent use.
The results [H] got speak for themselves.
Its even more work for them to include other benchmarks that dont show how it performs in games so I understand why they dont include them

they speak for themselves how ?

All I see is Kyle defending himself and his testing method that is getting drasticly diffrent results .


Why are they getting drasticly diffrent results and is thier way of testing really showing us a true example of how these cards will play games ?


I don't think his way is any more or less valid than canned tests , however with canned tests you can repeat them and they will be the same everytime you run them which gets rid of as many x factors as possible.


I mean everytime i hear about a new game its allways about some crazy new a.i that will make the bad guys act more human and react better than the older ones and each time you play the game you will experiance something new .

So tlel me how can benchmarking that with no control produce correct results ?

 

[Nenu]

they speak for themselves how ?

All I see is Kyle defending himself and his testing method that is getting drasticly diffrent results .


Why are they getting drasticly diffrent results and is thier way of testing really showing us a true example of how these cards will play games ?


I don't think his way is any more or less valid than canned tests , however with canned tests you can repeat them and they will be the same everytime you run them which gets rid of as many x factors as possible.


I mean everytime i hear about a new game its allways about some crazy new a.i that will make the bad guys act more human and react better than the older ones and each time you play the game you will experiance something new .

So tlel me how can benchmarking that with no control produce correct results ?

its clearly beyond you.

 

[Citizen86]

Why are they getting drasticly diffrent results and is thier way of testing really showing us a true example of how these cards will play games ?

Crysis GPU test = high FPS

Crysis Assault Map = low FPS

Which do I want to see how well the video card in question does? Assault....

Call of Duty 4 "Opening Cutscene" = high FPS, who cares?

Call of Duty 4 "The Bog" level, one of the most demanding in the game = low FPS

I want to know how hard the video card is going to be hit.... not how fluid that bastard's smoking is going to look.

This keeps getting explained to you, but you refuse to accept the answer. Play even 5 minutes of a level through on almost any game. Even if you don't shoot someone as fast as you did the first run through, you can still basically mimic the gameplay, where you go, what you see.

You use the advanced AI as an excuse that you can't replay a level the same way twice. Although that's what the developers want you to believe, how often is that actually the case? Bioshock had incredible life-altering AI?

 

[Simpson5774]

http://www.theinquirer.net/gb/inquirer/news/2008/01/28/review

I don't know if this post has been mentioned yet, but it basically is INQ showing off links to a whole bunch of 3870 x2 reviews, but it lacks the HardOCP review, and I seriously have to wonder why. HardOCP is as big of a name as any of the other 20 something reviews on there, but it seems the be the only one using a testing methology that works.

 

[V99]

No trolling - Kyle

 

[Citizen86]

http://www.theinquirer.net/gb/inquirer/news/2008/01/28/review

I don't know if this post has been mentioned yet, but it basically is INQ showing off links to a whole bunch of 3870 x2 reviews, but it lacks the HardOCP review, and I seriously have to wonder why. HardOCP is as big of a name as any of the other 20 something reviews on there, but it seems the be the only one using a testing methology that works.

I sent him an email asking him why no love for the [H], let's see what he says

 

[gljvd]

Crysis GPU test = high FPS

Crysis Assault Map = low FPS

Which do I want to see how well the video card in question does? Assault....

Call of Duty 4 "Opening Cutscene" = high FPS, who cares?

Call of Duty 4 "The Bog" level, one of the most demanding in the game = low FPS

I want to know how hard the video card is going to be hit.... not how fluid that bastard's smoking is going to look.

This keeps getting explained to you, but you refuse to accept the answer. Play even 5 minutes of a level through on almost any game. Even if you don't shoot someone as fast as you did the first run through, you can still basically mimic the gameplay, where you go, what you see.

You use the advanced AI as an excuse that you can't replay a level the same way twice. Although that's what the developers want you to believe, how often is that actually the case? Bioshock had incredible life-altering AI?

Dude , get over the cod4 thing. He is not the only one showing diffrent results . There have been links to many other sites that show vastly diffrent scores including driverheaven

As for bioshock . I've played the game 3 times and each time I experianced diffrent things . Sometimes i'd run into no big daddys in a certian spot , sometimes there would be 2 plus a bunch of slicers .

Other games like cod4 are extremely diffrent for me . I've played through that twice on the 360 (more friends on live than pc gaming now adays )

 

[btdvox]

YES The new ATI card is better than the ultra. Its beats the Ultra in price/performance and performance in general

Stop trolling man. No IT MOST DEFINITELY is NOT!!!

Yes the card beats the ultra in certain games and the ultra wins in other games and they tie in a couple games. dose taht make the new ati card a winner ? I DONT THINK SO. -

Also Please take this last TID BIT in account....
OVERCLOCKING!!!!

Do you not overclock? Weird last time i checked ALOT OF USERS overclock. And You can easily overclock an 8800 ultra to 675/1134 where as 3870x2's overclock to a measly 860/900 from most reviews seen. Thats not even a 40 mhz overclock....

But then again you'll prob say no one overclocks....

So when you put up overclock results such as Overclockers club did- id assume that things would change quite a bit

 

[FrgMstr]

Sorry Kyle , I just don't agree with you. You've voiced your opinon many times.

Yes we understood that the first time you refused any refuting logic to your explanation. Now you can back to your forums and declare victory. I am good with that.

And a little cursing never hurt anyone. Sorry if it is not your style. Now have a big cup of STFU.

Unless you have something new to say in your next post, I will just disable your posting ability. So let's move on shall we.

 

[bluehaze013]

Also Please take this last TID BIT in account....
OVERCLOCKING!!!!

http://www.youtube.com/watch?v=VeHgoyGRbuI

 

[ChakkaSol]

If not said before (30 pages in a day, is that a record?) - the real benefit of today's release is that the 3870x2 is seen by a rig as 1 card so you can run 2 gpus on any motherboard...hopefully game makers will write new games based on multi-cpu/gpu systems giving us all smooth and high quality gaming at a cheap price.

It seems to me that the future is in multi-gpus, cause I would like to see 4 cores on a single card similar to the latest quad core cpus...if only we could see 4 cores under $500 too...

 

[btdvox]

http://www.youtube.com/watch?v=VeHgoyGRbuI

Thanks for showing a watercooled block card. Do you want me to show you some Dry ICE tests now too? Oh and please dont show me 3dmark 06 tests - IT MEANS NOTHING TO ME! I dont like watching loops Id rather play thanks

My 8800 ultra with a swiftech stealth waterblock gets me some nice numbers too OC'd

 

[cygnus_x_1]

jeez, 15 pages of whiny bitches over a review on how most users will use the card (unless you are just a bencnmark bitch). Do I agree with the way [H] does their GPU reviews? well, its a different perspective than the clones that just run timedemos, and straight benchmarks. While its not easily comparable, you do get a good feel on the upper limits of the cards. Is every one running 24-30" LCD's that will stress this hardware, last time i check, no. The average user is running 19-22" so the gameplay, and visuals should be pretty decent for the average hardware.

I know myself being a reviewer on another site, I don't read reviews for the benchmarks, or gameplay, I'm there for the writers editorial content. I like to see how real users are faring in the forums with their hardware which is MUCH more accurate than any review/reviewer will EVER be.

Kyle, I don't know how you do it with out freaking the F out...

 

[gorilla show]

I'm glad you've heard me . I was responeding to others that were quoting me.

As for cursing , its not my cup of tea . You attract more flys with honey than viniger.


As for something new to say. How about a question


Can you run an x2 and a 3870 in a tri crossfire type deal ? I know tha the x2 has the slower ram but could they could handle that in software .

what's viniger

I just read through the review and was mostly unpleased-- I was expecting the 3870x2 to do some major asswhooping on the competition. It still looks like a solid card, but I am going to wait a bit longer as my X1900XT still proves worthy of gaming (with Crysis being the obvious exception).

Other review sites may have different results, but the time that I have spent reading [H]'s articles and reviews has shown me that Kyle, Brent, and Co. put quite a bit more effort into this. The [H]'s real-world testing on video cards have helped me in my purchases. I trust the [H] because there has never been any reason for me not to. I trust other sites as well, so this has been rather challenging for me and others.

Whether or not one chooses to rely on one review or another (or multiple) is strictly within their own and should not be made into a hissy forum fit.

 

[FrgMstr]

Can you run an x2 and a 3870 in a tri crossfire type deal ? I know tha the x2 has the slower ram but could they could handle that in software .

The configuration is limited to 4 GPUs total now for CrossFireX. So four R670 or two R680.

 

[jaegertech]

I'm disagreeing and you have yet to explain to me how your able to get acurate results from something as random as playing through a level.

Basic probability works rather nicely. Three "gameplay" runs through the game with Card 1. Three "gameplay" runs through the game with Card 2. There is no way for any of these runs to be identical to any other, but the reviewer is (presumably) attempting to make them similar. Now, if you presume that the reviewer is unbiased, anomalies will happen RANDOMLY throughout the runs. Run1/Card1 might have a couple guys popping grenades all at once thirty seconds in, but it is equally likely that Run3/Card2 will have a smoke grenade going off three minutes in. As these anomalies are random, they can be accounted for by taking enough samples. In [H]'s case, the samples appear to be several minutes of gameplay, several times. Law of large numbers at work here: you take enough samples, you get a meaningful answer that averages out anomalies. It is statistically possible to still end up with a meaningless result, but it is much less likely. I assume it is part of the reviewers job to interpret the results and judge if more testing is necessary.

This methodology is ENTIRELY dependent on reviewer integrity, but it is also less susceptible to manufacturer tampering. How do you know Graphics Company #3 didn't spend man hours spanking their drivers to render water a little faster during one part of a canned flyby?

Frankly, this is one hell of a risk for Kyle to take. If the reviewers are EVER caught with their hands in the cookie jar, this site will probably pull a Gamespot.

 

[jstnomega]

Another fact to remember is that gaining performance by adding GPUs is a process of ever-diminishing returns. I suspect that combining two of these cards in a CrossFire arrangement (effectively 4-way CrossFire) will not be as efficient as adding two single-GPU NVIDIA cards together in an SLI arrangement. Plus, there’s nothing stopping NVIDIA doing exactly the same with one of their recent cores, such as the 8800 GT. However, despite having already dipped its toes into the waters of dual GPU solutions, this approach does go against the flow where NVIDIA is concerned. Don’t forget that NVIDIA absorbed 3dfx, which faltered in part from developing multi-GPU solutions back in 1999. Such solutions result in extremely complex and expensive boards. NVIDIA’s instinct has been to move in the opposite direction and reduce board complexity while maintaining performance. The latest 8800 incarnations have been a prime example. Whether this consolidation of components is a prelude to squeezing two GPUs onto one board, only time and the rumor-mill will tell.

http://accelenation.com/

 

[POPEGOLD]

Basic probability works rather nicely. Three "gameplay" runs through the game with Card 1. Three "gameplay" runs through the game with Card 2. There is no way for any of these runs to be identical to any other, but the reviewer is (presumably) attempting to make them similar. Now, if you presume that the reviewer is unbiased, anomalies will happen RANDOMLY throughout the runs. Run1/Card1 might have a couple guys popping grenades all at once thirty seconds in, but it is equally likely that Run3/Card2 will have a smoke grenade going off three minutes in. As these anomalies are random, they can be accounted for by taking enough samples. In [H]'s case, the samples appear to be several minutes of gameplay, several times. Law of large numbers at work here: you take enough samples, you get a meaningful answer that averages out anomalies. It is statistically possible to still end up with a meaningless result, but it is much less likely. I assume it is part of the reviewers job to interpret the results and judge if more testing is necessary.

This methodology is ENTIRELY dependent on reviewer integrity, but it is also less susceptible to manufacturer tampering. How do you know Graphics Company #3 didn't spend man hours spanking their drivers to render water a little faster during one part of a canned flyby?

Frankly, this is one hell of a risk for Kyle to take. If the reviewers are EVER caught with their hands in the cookie jar, this site will probably pull a Gamespot.

whats throwing me is DRIVER HEAVEN claims to have run thier test the same way as KYLE does ...and in cod4 at similar settings the numbers are way off

the 3870x2 slaps up the ultra on DH.... but loses to the gtx at the same rez on [H]

 

[gljvd]

http://accelenation.com/

Nvidia is putting out a dual chip single core .


Look your not going to get %100 improvement but its going to be the easiest way to improve performance for a long time. As you add more chips you get a smaller gain each time , but you still should get a gain and for some thats enough.

Personaly i'd rather add 2 cards to my pc with 4 cores on it than 4 cards in my case , so even if they make more of these on both sides of the fence , i'm happier than them doing it with single cards

 

[Commander Suzdal]

It's in that thread man here I will quote it out for you:



I take it you aren't a regular, so ill break it down. We use real game play to test graphics cards, we have done so for a long time. if we used pre built benchmarks or time demos we wouldn't have found the minimum frame rate issue as the majority of sites who used these still haven't even found the issue. This is only seen with extended gameplay, or what id like to call "real life" gameplay. We play games for hours behind the scenes with new cards before we even start benchmarking. This is the best way to find or note anything unusual before we get into the indepth review testing. This is how we found issues that caused AMD to delay their NDA for a week. Issues we helped solve.

if you would like anything else explained please ask away.

Every single test in the review used real gameplay... i can confirm that as i was the one who tested the cards and wrote the article. We/I havent used timedemos, or built in benchmarks for years, and have never benched a cutscene in my life.

and heres a link to the thread:
http://www.driverheaven.net/news/153658-dh-review-ati-hd-3870-x2-2.html

Thanks for feeding me further info, the effort is appreciated. I have now read the thread in addition to the article. Two questions remain (and no, I'm not going to register over there and needle them with questions that will undoubtedly be unwelcome):

First, what do they mean by "We play games for hours behind the scenes with new cards before we even start benchmarking. This is the best way to find or note anything unusual before we get into the indepth review testing."? If they mean the same thing [H] does by real-world testing, why do they draw a distinction between playing the game and "benchmarking" or "in-depth testing"?

Second, and this is really the reason why the first question arises at all, why is there still such a large difference in their results as compared to [H]'s on COD4? The hardware setups are not dramatically different, and yet DH is getting higher frames across the board with the same resolution, and with AA settings that [H] found unplayable on the 3870X2. Could it be because of the settings on the game options screen shot that DH has at normal? Maybe there are higher settings than normal that fell under [H]'s "Highest In-game Settings" catch-all. Or maybe it was dependent on which part of the game DH playtested, which wasn't specified.

Even if DH uses methods every bit as rigorous as [H]'s, somebody's wrong and somebody's right on this one--which I guess is the gist of what you said when you first linked to the review. It will be interesting to see how it plays out long-term. Given the 2900XT history especially.

 

[Rock&Roll]

whats throwing me is DRIVER HEAVEN claims to have run thier test the same way as KYLE does ...and in cod4 at similar settings the numbers are way off

the 3870x2 slaps up the ultra on DH.... but loses to the gtx at the same rez on [H]

Different sites different results. I don't see anything surprising there.

-Specifically The [H] was on Vista x64. DH was on Vista, unspecified 32 or 64.
-As far as COD4, DH does not clarify, as the [H] does on which texture setting it used. The fully maxed out "extra" setting would obviously stress the 256-bit bus much more than the 384-bit bus.

You're getting hung up on the numbers and not really thinking about how these sites got their numbers. General trends should not be hard to see. Don't get hung up on 5 or 10 fps here or there. What ALL reviews say is that sometimes the x2 wins, sometimes the ultra/gtx wins. So, you can't lose either way. Pick your poison and move on.