Beyond3D Forums Close & Delete Benchmarking Discussion.

digitalwanderer said:
I don't get your point Kyle. Should I go dig up the thread where you banned Dave or any of a half a dozen others during the FX era to point out to you for some reason? :confused:
Click to expand...

Feel free to do what you want to. Sounds like a fun time to me, and we all know that you certainly have the spare time to invest in such a project.

I know you have a hard time getting the meaning of things stated sometimes, so please let me help you out. Reread this line.

Doomtrooper said:
I've said this many times, and it will ALWAYS be the graph with the highest numbers that the websites BRAG about, no matter if its blurry and banding throughout the review.
Click to expand...

I pointed this out, because Doomtrooper was one guy that would rake HardOCP over the coals constantly. One of the things being discussed here is how HardOCP has changed how video cards are "reviewed," at least for us. The fact of the matter is that HardOCP made true one of the things that Doomtrooper noted needed to be fixed four years ago. He was right in pointing out that is was mucked up, but it is funny to me that HardOCP was the first site to actually do something about it, when he used to / still does? despise us so much.

Now all of that said, I usually make it a point to ignore you because you usually contribute nothing to any conversations that I have seen you be a part of. But in this one instance your ignorance has shown me that maybe my hint to Doomtrooper's 4 year old post was bit too subtle for some. Thanks for showing me the way.
 
phide said:
Doodle said:
1) They do not seem to be consistent with "non-performance" issues: i.e. their take on the unavailability of ATI X1800 vs unavailability of NV 7800GTX 512 (NV got a much easier ride from [H] than ATI did IMO).
Click to expand...

At the time, the GTX 512 was (seemingly) plentiful. If you recall, it had actually been available a number of days before NDAs were lifted. The evaluation was written prior to the release, and thusly, no thoughts on releases could be given at the time. Could the article have been updated? Possibly, though I don't imagine what purpose that would really serve.
Click to expand...

The X1800XT too was available, before the launch, by a few days. And was always available, after it came out. The X1800XL was two days late, and always available after it came out. The 512MB GTX too was available a few days before launch, and a few days after. However, after that, it was MIA. And the very, very few times it did come into stock, it was well above MSRP.

I agree with Doodle, NV got a much easier pass on availability problems to me. ATi took constant shots on the front page, where as NV's didnt. At least thats how I remember it.

But more on topic, I am very glad [H] is going to use WS resolutions in upcoming evaluations. And I do like that they use TRAA/AAA where as most do not, as well as other settings. Nor does [H] use useless 1024x768 number, or 3dmark, which I also find very misleading. Especially '06. I like [H]'s, FiringSquads, and Rage3D's (<--- very good) reviews the best. And then B3D to just boggle my mind after Ive read the others. :eek:

edit, I also like how Brent is generally available on these forums, if you have a question about an evaluation. And how he has ran something for us on the forums, just because we asked. That reaching out to your readers, the way I dont think any others normally do. And he does it all the time.
 
BoogerBomb said:
I had never really thought about it until you mentioned it but I dont recall too many sites with that information. That might even be more important than max fps and perhaps more reflective of what the GPU is capable of. It it has really low min FPS means that the GPU may not be as good as the other GPU that has higher min fps.
Click to expand...

I'd say the minimum fps is the most important, followed closely by a consistent average. I don't want my framerates going from 200 fps to 30 to 60 to 120 to 25, that rapid change in framerate is very negative for smoothness, you feel the change in framerate.

There really is no other way to get this information than going in the game and playing the game to see what the framerate is doing.
 
Dang Kyle...I thought it was just me!

But now I see you also have the ability to fall into heated forum posts. :D

BTW: I think a max playable with best IQ comparison on real games is very valid...it's how I decide if I like a card or not myself. That and things like being able to run Doom3 Editor (which ATi sucks at now, BTW)
 
BBA said:
Dang Kyle...I thought it was just me!

But now I see you also have the ability to fall into heated forum posts. :D

BTW: I think a max playable with best IQ comparison on real games is very valid...it's how I decide if I like a card or not myself. That and things like being able to run Doom3 Editor (which ATi sucks at now, BTW)
Click to expand...

Many folks have issues with my lack of tact coupled with my brutal honesty. I has worked out well for us over the years, so I am sure not going to change now. Besides it is fun to see the truth get so much drama flowing.
 
Kyle this was an interesting thread today it got a lot of hits. Very entertaining.

Tomorrow start a thread on Vista in regards to the graphics interface and open gl.

I think it would be an eye opener for a lot of readers in regards to the fact that there will be no more optimations by either ATI or NV as DX10 and vista will run the whole show.

With DX10 the next cards out of the box I think its time everyone understands after these cards and vista are out . The Words (The way its meant to be played) is no longer going to be a joke. Is open GL in the future of microsofts planes ? I think not. So I see a few people going to that other OS . I hope they have fun
 
Sometimes I feel as if Kyle/Steve is being slightly biased agianst ATi but never to any point close enough to make the review null and void. I still read his conclusion and agree most of the time. Beyond 3D is a great fourm with great people I have read that fourm for the longest time and I have learned so much, and the same applies to the [H]. Diffrences make the world go round. And as for my Kyle/Steve being slightly biased towards the N I feel that my claim is superficial mainly since I dont know what goes on behind the scenes at the [H] still so far its been my Opinion and I dont mean any offense to you for it Kyle/Steve.
 
Are we having fun yet guys? :)

I tend to think Kyle got bored and decided to have some fun with a mini-site-war to liven up a Monday! After all, what I like about him is he is very much of the community, "one of us" going way back. For good and ill. ;)

Tho, since I'm here, I see that you're adding WS res, for which this 2405FPW owner thanks you!
 
Doodle said:
...
1) A chart along the lines of "max resolution and AA / AF game can be played at while achieving a minimum XX FPS" similar to their chart of comparable settings...
Click to expand...

That would be perfect. Taking out some of the useless apples to oranges comparison charts and adding these would be a godsend.
 
$BangforThe$ said:
Kyle this was an interesting thread today it got a lot of hits. Very entertaining.

Tomorrow start a thread on Vista in regards to the graphics interface and open gl.

I think it would be an eye opener for a lot of readers in regards to the fact that there will be no more optimations by either ATI or NV as DX10 and vista will run the whole show.

With DX10 the next cards out of the box I think its time everyone understands after these cards and vista are out . The Words (The way its meant to be played) is no longer going to be a joke. Is open GL in the future of microsofts planes ? I think not. So I see a few people going to that other OS . I hope they have fun
Click to expand...

Thats a common misconception, microsoft is providing a opengl library that just translates a basic feature set of opengl code (opengl 1.1 I think) into directx. They also disable whatever special gui features of an opengl window for some reason Im unaware of and dont care to lookup right now.

No one in their right mind would use microsoft's opengl library for gaming. Ati and nvidia already provide you with their own opengl implementation today and will continue to do so with vista. I doubt that you will regain the fancy aero gui features with opengl apps running in window mode, but it will not be dog slow like microsofts implementation. (May it be todays or Vistas)
 
Brent_Justice said:
I'm in love with widescreen gaming now.
Click to expand...
You and I and everyone else should start mentioning that in our reviews more - most games do not support widescreen resolutions and need hacks to work; and that doesn't apply for every game. Widescreen gaming is here to stay imo.

As for optimizations (since I saw the topic mentioned), it's a very long discussion, which can be summarised to this simple sentence: They're fine, as long as I don't know they're there.
 
Kombatant said:
As for optimizations (since I saw the topic mentioned), it's a very long discussion, which can be summarised to this simple sentence: They're fine, as long as I don't know they're there.
Click to expand...

Ignorance is bliss
 
Hardocp's reviews are pretty much useless now because of the format. I think most people know that. However he's not going to change so I'm not going to cry about that.

The whole way they treated the 512 7800 GTX, one of the most blatant press edition's in video card history, is one thing I disagree with.

The 512 was made to win benchmarks and was never available widely. I feel Nvidia should have really gotten called out on that. However they weren't.

All in all I think ATI should stop sending cards, let alone info, to Hardocp. This would be great. Let them fend for themselves how they want, talk this nonsense about only reviewing available cards (unless they're Nvidia) if they like. I'll go read the sites that put the information out there. If a paper launch card is reviewed by one site, guess what I'll read it cus I'm interested!
 
Kombatant said:
Nothing to do with ignorance, everything to do with mathematically equal optimizations. There's a difference.
Click to expand...

Thefore you just negated what you stated above then.
 
Lets say 7900GTX is made available for review tomorrow. However it will not be in stock for one week.

Anand or Hard or whoever puts up the full review. The other site says we will wait until it's available. Are you going to read the review? Or wait the week? We know the answer.
 
Nothing to do with ignorance, everything to do with mathematically equal optimizations.
Click to expand...

One should never think of mathematics when playing games.
 
Coldtronius said:
Thefore you just negated what you stated above then.
Click to expand...
Umm, no. When you have an optimization that produces the same result as the original function, you don't see a difference and thus you don't care whether optimizations exist in the game or not - you merely concentrate on gameplay. I didn't negate anything and I really don't understand your way of thinking; perhaps you could elaborate?
 
Wow, this thread and the referenced thread at B3D are fascinating. It was like watching a train wreck, you don't want to watch but you can't help yourself... I believe both sides are not quite correct in fending for "their" benchmarking. Purely subjective reviews are useless. What I subjectively consider good another might consider not quite that good, and vice versa. The bottom line is that with everyone cheating, nVidia & ATI, pure benchmarking is not going to work either. I don't have a problem with game optimization but I did have a problem with the 3DMark bull that both ATI & nVidia pulled. Bottom line is that video card performance evaluation has become a lot more difficult for reviewers, and even more difficult for consumers.
 
Weazmeister said:
Bottom line is that video card performance evaluation has become a lot more difficult for reviewers, and even more difficult for consumers.
Click to expand...
I will definitely agree to that, and to add insult to injury, I still see computer magazines for instance benchmarking graphics cards like we are still in the 2001-era. This has definitely got to stop.

I agree that you can't judge the way a card performs by running a timedemo - but, on the other side, if you want to do it [h]-style, how can you be sure that the certain combination of eye candy settings you propose is the one common folk use? Perhaps for me soft shadows are indifferent, so I disable them and I can immediately play the game at 4xAA instead of 2xAA - and at that resolution/eye candy settings, card A is better than card B (and not the other way around it was when i had soft shadows on). Games today have so many different variables you can alter and influence performance, that I believe you can never have a proper evaluation that way.

What's really important, in my opinion, is to find out the "worst-case" scenario for each game. That comprises of two things, playing the game in order to find out the part where it is the most taxing for the GPU, and benchmarking at that point. Then making people know that, what they see is how the card(s) you benchmark perform under the worst possible conditions a certain game has to offer. This will mean that, in general, performance in that game can't get lower than that with a certain card and certain AA/AF combinations. For me personally, it's totally irrelevant if card A gets 240fps and card B gets 260fps. But I want to know how they perform under the worst-case scenario of the game (too many baddies on screen for instance, lots of gunfight, etc) because that's when I am going to need my card's power. Not when I am staring at the wall and walking in empty hallways.
 
Kombatant said:
I will definitely agree to that, and to add insult to injury, I still see computer magazines for instance benchmarking graphics cards like we are still in the 2001-era. This has definitely got to stop.

I agree that you can't judge the way a card performs by running a timedemo - but, on the other side, if you want to do it [h]-style, how can you be sure that the certain combination of eye candy settings you propose is the one common folk use? Perhaps for me soft shadows are indifferent, so I disable them and I can immediately play the game at 4xAA instead of 2xAA - and at that resolution/eye candy settings, card A is better than card B (and not the other way around it was when i had soft shadows on). Games today have so many different variables you can alter and influence performance, that I believe you can never have a proper evaluation that way.

What's really important, in my opinion, is to find out the "worst-case" scenario for each game. That comprises of two things, playing the game in order to find out the part where it is the most taxing for the GPU, and benchmarking at that point. Then making people know that, what they see is how the card(s) you benchmark perform under the worst possible conditions a certain game has to offer. This will mean that, in general, performance in that game can't get lower than that with a certain card and certain AA/AF combinations. For me personally, it's totally irrelevant if card A gets 240fps and card B gets 260fps. But I want to know how they perform under the worst-case scenario of the game (too many baddies on screen for instance, lots of gunfight, etc) because that's when I am going to need my card's power. Not when I am staring at the wall and walking in empty hallways.
Click to expand...

EXACTLY.
 
Kombatant said:
Umm, no. When you have an optimization that produces the same result as the original function, you don't see a difference and thus you don't care whether optimizations exist in the game or not - you merely concentrate on gameplay. I didn't negate anything and I really don't understand your way of thinking; perhaps you could elaborate?
Click to expand...

Actually there is no way to do this, this is why we can pick up on these optimizations. And there really is no way around making two different algo's to end up looking the same, mainly because if the API calls, programmers have no control on what the calls do behind the scene so what ever is used will always produce differences.

Edit:

The only way optimizations will produced mathmatically equal results is if the orginal program isn't optimized. And this is very rare. Also this is why all IHV's have different outputs, the hardware just do things in different ways.
 
razor1 said:
Actually there is no way to do this, this is why we can pick up on these optimizations. And there really is no way around making two different algo's to end up looking the same, mainly because if the API calls, programmers have no control on what the calls do behind the scene so what ever is used will always produce differences
Click to expand...
Allow me to disagree.
 
razor1 said:
well the only other way to do it is if we scrap the API's ;)
Click to expand...
It depends on what you consider optimization - I mean, in certain platforms, merely re-arranging instructions without actually altering them results in better performance because the pipelines are constantly fed. It's (more or less) the same with CPUs and GPUs really - I've done optimizations for CPUs using VTune so I think have a pretty good idea on how these things actually help :)
 
This thread encapsulates what I don't like about the video card review industry.

The emergence of the video card as the standard by which computers (and their owners) are judged has given rise to a stratified hierarchy of users and reviewers.

While I'm sure the B3d forum users feel like they're all somehow now game developers or hardware engineers because they've learned to connect the dots between white paper specifications and the resulting performance in video games, for the most part, they are not.

I would even question the value of this rote memorization style of posting and review, as in the end none of us know how hardware will be used by future software, and for the most part we can assume current software won't be satisfactory on games released a year later.

I like H's style of review. It's far more usefull to me to have minimum framerates at maximum IQ settings that are playable than know a card will run FEAR at 13fps at 16X12 4X8X.

I want to know what settings I can expect to play current games at, and that may be 16X12 2X MSAA 8X HiQ. HardOCP saves me the trouble of testing (read: wasting my time) playing at a variety of settings to see what works. I get a pretty good idea by starting here.

The "normalized settings" will always be a moot point due to differences inherent in rendering methods.

I'm not saying the paraphrased press kits/white papers don't have their place, they're good to read if you're so inclined. This notion of the people who do so being somehow "Elite" and taking on a condescending, paternalistic attitude is what annoys me.

<end rant>
 
Kombatant said:
It depends on what you consider optimization - I mean, in certain platforms, merely re-arranging instructions without actually altering them results in better performance because the pipelines are constantly fed. It's (more or less) the same with CPUs and GPUs really - I've done optimizations for CPUs using VTune so I think have a pretty good idea on how these things actually help :)
Click to expand...


Ok yes that is very true :) rearraigning instructions is 100% ok in my book :D. Didn't really concider that an optimization, guess i was thinking in the lines of shader replacement ;)
 
You really want to do something useful for the community (and, yeah, it's even more work), start adding a greater variety of cpu's to the benchmarks. If not for every game tested, then at least a special section with a representative selection with a mid-range cpu and a lower-end mainstream cpu. For every guy who is itching for a new gpu and has a FX60 to pair it with, there are probably five guys who are itching for a new gpu but have something signficantly less exalted to pair it with.

It seems to me that a greater range of cpu's used to me more common in viddy card testing, but has been drying up the last few years.
 
Rollo said:
While I'm sure the B3d forum users feel like they're all somehow now game developers or hardware engineers because they've learned to connect the dots between white paper specifications and the resulting performance in video games, for the most part, they are not.
Click to expand...

Well, no, just the ones who actually are game developers or hardware engineers --and probably the highest percentage (tho still low) on any major graphics forum around. Why do you think it rated three AEG'ers? (your post count doesn't qualify you as a regular to be #4).

It is a pity that it seems so difficult for some people to appreciate the strengths of a given methodology without succumbing to the temptation to throw other methodologies with their own strengths under the bus.

I find value in [H]'s reviews, I just wouldn't care to have B3D emulate it --nor, for that matter, for [H] to emulate B3Ds. Appreciate the mosaic, people, it's one of our strengths as a community.
 
geo said:
It is a pity that it seems so difficult for some people to appreciate the strengths of a given methodology without succumbing to the temptation to throw other methodologies with their own strengths under the bus.

I find value in [H]'s reviews, I just wouldn't care to have B3D emulate it --nor, for that matter, for [H] to emulate B3Ds. Appreciate the mosaic, people, it's one of our strengths as a community.
Click to expand...

Very well said ;)
 
geo said:
Well, no, just the ones who actually are game developers or hardware engineers --and probably the highest percentage (tho still low) on any major graphics forum around. Why do you think it rated three AEG'ers? (your post count doesn't qualify you as a regular to be #4).

It is a pity that it seems so difficult for some people to appreciate the strengths of a given methodology without succumbing to the temptation to throw other methodologies with their own strengths under the bus.

I find value in [H]'s reviews, I just wouldn't care to have B3D emulate it --nor, for that matter, for [H] to emulate B3Ds. Appreciate the mosaic, people, it's one of our strengths as a community.
Click to expand...

You've got the right idea here. I personally do not care for [H]'s GPU reviews. I give them a look every time they come out.. but I am more partial to the apples to apples approach. Usually, I find another site to get my benchmarks from, and from [H] I stick to the first and last pages simply to get the editors overall opinion of the card.

I respect [H] and it's opinions, and believe they know their shit when they say I card is good... I just don't like the way they represent it in the benches ;). To each his own.
 
As im sitting here looking back through the thread, a couple of thoughts come to mind.

Kyle, how much do you charge average joe, to come and look at reviews put together by you and your team?
How much is the forum subscription? (Gen May not included)

It all boils down to this, everyone has their own opinion. Kyle you have been at this for a long time now, and I've been here most of it. And when your successful at something, everyone wants to see it fail. Envy? Jealousy? probably have allot to do with it. When you pay someone who works for you to bust their ass to do reviews, pay for bandwidth, so people can view it for free, and then they want to bash the work your team put into it. I say the hell with them. Keep up the good fight. If the nay-sayers could do better they would have their own websites that stood up for the community.
 
razor1 said:
Actually there is no way to do this, this is why we can pick up on these optimizations. And there really is no way around making two different algo's to end up looking the same, mainly because if the API calls, programmers have no control on what the calls do behind the scene so what ever is used will always produce differences.

Edit:

The only way optimizations will produced mathmatically equal results is if the orginal program isn't optimized. And this is very rare. Also this is why all IHV's have different outputs, the hardware just do things in different ways.
Click to expand...

This is exactly why I sugjested that Kyle start a DX10 Vista thread . Although all the facts aren't known . Vista's API will run the show on the DX10 cards . Optimsing a card running in DX10 mode will have zero affects. Another poster replied that Open gl will run on vista which is true but not in DX10 interface it will run the same as the present ati. nv cards will.

Vista API DX10 will change how the Game is played . MS says THE WAY GAMES ARE MEANT TO BE PLAYED.
 
HI Rollo
Thats interesting view you have . I agree to a point . B3D is a more technical review site and forum . But it is still very interesting review site. There are want to be's there as you stated . There is however some Brillant people posting at the site. Many times I get lost in the conversations and many times its redundent. But there are people there Rollo who are far more capable of just connecting the dots.

The post I posted just above this one would take off and really fly at B3D here it will just lie there and die .

You did make some valid points though.
 
$BangforThe$ said:
HI Rollo
Thats interesting view you have . I agree to a point . B3D is a more technical review site and forum . But it is still very interesting review site. There are want to be's there as you stated . There is however some Brillant people posting at the site. Many times I get lost in the conversations and many times its redundent. But there are people there Rollo who are far more capable of just connecting the dots.

The post I posted just above this one would take off and really fly at B3D here it will just lie there and die .

You did make some valid points though.
Click to expand...

I have nothing against B3d/the people there beyond what I see as a patriarchal attitude toward those who don't share their love of trying to understand the details of interplay between GPU/driver/software to the extent they do.

I enjoy reading that sort of thing at times, but for the most part, I want to know what kind of performance and settings I can expect from a card, compared to other cards. The highlights of new features are great as well.

While I realize many of the posters there are industry professionals and respect them for posting there, my point in posting here is that there was no need to attack HardOCPs style of review in that thread.

Side note: AEG members do not get assigned to a board by it's relevance, we post where we like to post. The fact there are three AEG members posting at B3d probably has more to do with the type of discussions the three members like to partake in than anything else. (as I'm sure they'd tell you)
 
mashie said:
I hope Santa Kyle brought you a Dell 3007 so you can test all resolutions ;)
Click to expand...

Those are a bit pricey, and honestly, how many people have those right now? That's a niche of a niche of a niche.

Nice LCD for sure, but only the elite of elite are going to be able to afford one, I don't think we'll see many at LAN parties either :p
 
You must log in or register to reply here.
Back
Top