does 3dmark really mean nothing?

[joethemole]

i read somewhere that a 9600xt is better than an x300, yet the x300 scores like 2500 in 2k5, while my oc'ed 9600xt is at 1700.

but then again my 9600xt is higher than the 5950 from what i heard...

 

[Brent_Justice]

it really means nothing

except for maybe system tweaking and stability testing, on your own system

 

[legcramp]

a x300 does not.. i repeat not score 2500 in 3dmark2005.... a 9800pro or higher gets 2500.... not a x300.. maybe in 2003 it gets 2500 and u bench ur 9600xt in 05 instead of 03...


http://www.xtremecomputing.co.uk/review.php?id=100&page=2

scrool down and u'll see that the x300 ONLY gets 1142 in 3dmark2005.. and around 2500 in 2003.... u are confusing ur 3dmarks and comparing the wrongthing.. since u got 1700 in 3dmark2005 and teh x300 only gets 1142.... ur 9600XT>x300 OK? hehe enjoy

 

[Scali]

3DMark is very meaningful, as long as:
1) You compare results from the same version of 3DMark.
2) You compare only between hardware from the same generation (eg, don't compare the score of a GeForce256 with a GeForce 6800 in 3DMark03, because they don't run the same set of tests).
3) You use drivers without any form of application-specific optimizations.

I think that point 3) has caused the biggest problems for 3DMark... It seems to be more or less under control these days, though.

 

[CleanSlate]

I always use 3dmark2k1se as a benchmark as to how well a computer can perform compared to mine or at least I used to back when I had a gf 2, now a days I use 3dmark2k3 because it uses about the same set of gfx tools I play most games with... now 2k5... I don't know what that's all about but that's the worst 3dmark I've ever seen, not only does it all look like junk but it seems to be slow on my 9800 for no reason, no elaborate shades, no great triangular masses, it honestly just boggles my mind how 2k5 can tax my 9800 so badly.. but anyhow yea it also is good for tweaking.

~Adam

 

[SoLiD_MasteR]

what 3dmark version you are talking about !?

also please consider some tests may not work on some cards because they don't support dx9 (that's why some fast-old cards get lower score than some new-slow cards...)

 

[Scali]

I don't know what that's all about but that's the worst 3dmark I've ever seen, not only does it all look like junk but it seems to be slow on my 9800 for no reason, no elaborate shades, no great triangular masses, it honestly just boggles my mind how 2k5 can tax my 9800 so badly.. but anyhow yea it also is good for tweaking.

~Adam

3DMark05 obviously has incredibly high polycounts (just look at the detail on the airship, or the size of the hangar, and the detail on every part, and also the characters), far higher than any game.
And 3DMark05 uses shadowmaps extensively, which is a rather slow method of shadowing (but with some nice advantages, will be the shadowing method of the future).
Just look at the second game test, how every leaf and blade of grass in the forest is shadowed correctly, and with great detail.

So that's why 3DMark05 taxes your 9800 so badly.

 

[GregP24]

It's kinda hard on X800s and 6800s, too.

 

[pxc]

Unbelievably, 3DMark got even more worthless in the last version (05).

My benchmarks:
x300SE: ~1100 3DMarks
9600 Pro: ~1200 3DMarks

Reality: my 9600 Pro simply smokes my old x300SE (64-bit memory @ a much slower speed) in *everything*. Conclusion: 3DMark is worthless to predict performance.

 

[tsuehpsyde]

yea, I got 5112 on 3dmark05, and I noticed how horrible the green light demo beat up my system....freaking rediculous. I mean, it's the bench for '2005' but yet cards right now (weeks from 2005) get gang-banged....oh well, maybe the next revision of cards will run better (SLi is not a new revision lol, it's just more raw power).

Who else here thinks SLi is going to be a replacement for 6 month cycles and make them much longer? If not, I think it's going to extend the current generation myself.

 

[bonkrowave]

Its kinda hard to get a straight answer when you post in such a bias fourm such as this.

 

[arentol]

Its kinda hard to get a straight answer when you post in such a bias fourm such as this.

...Says the guy who doesn't actually try to give any answer at all.

 

[Scali]

Unbelievably, 3DMark got even more worthless in the last version (05).

My benchmarks:
x300SE: ~1100 3DMarks
9600 Pro: ~1200 3DMarks

Reality: my 9600 Pro simply smokes my old x300SE (64-bit memory @ a much slower speed) in *everything*. Conclusion: 3DMark is worthless to predict performance.

The 9600Pro gets a 10% higher score in 3DMark05 aswell, so it 'smokes' the X300SE aswell. Which is not bad, considering the fact that 3DMark05 is mostly geometry-limited and not bandwidth-limited on such cores with only 2 vertex units (it is aimed at high-end cards, which have 6 vertex units).

Ofcourse, results are worthless if you don't know how to interpret them.

 

[arentol]

3dMark is an awesome tool.... For determining whether a particular tweak to your system made a difference in graphics power.

3dMark is an awesome tool.... For comparing the performance of two systems with minor differences.

3dMark is an awesome tool.... For estimating how much average speed you will gain in games if you change a particular piece of hardware.

3dMark is not an awesome tool for more or less anything else. It's kind of a guide to the overall ability of various Videocards, but it's not accurate as a battery of Real Life games you actually play being run on the same system with each of the cards you are comparing.

Also keep in mind these ATI facts:

XT cards are the cream of the crop for their chipset.
SE's are the crap version of that same chipset.
__'s or pros are middle of the road for that chipset.

The x series (x300, x700, x800, etc.) are really only 1/2 of a generation newer than the 9600 and 9800 cards.

So the 9600xt is the high end version of the middle line card of 1/2 a generation ago while the x300 is the middle version of the low end card of the current generation. Overall low end cards usually need an entire generation to catch up with the old middle line cards, and sometimes even that isn't enough.

 

[pxc]

Of course, results are worthless if you don't know how to interpret them.

LOL

Yes, because a less than 10% difference in 3DMark translates to a 30-50% performance difference in games. 3DMark apologists are funny. It's a junk synthetic benchmark, deal with it.

 

[jon67]

it really means nothing

except for maybe system tweaking and stability testing, on your own system

no reason to use it in reviews then:

http://hardocp.com/article.html?art=Njk4LDU=

Video cards ranking by 3DMark performance is mostly consistent with ranking in GPU-limited games, so of course it means something.

 

[bonkrowave]

...Says the guy who doesn't actually try to give any answer at all.

I didn't give an answer beause I knew I would get flammed if I even so much as hinted 3dmark had some value at all. The facts are there are so many fan boys here who idolize kyle so much that they will support his idea that 3dmark is worthless, to the death, without even thinking for themselves.

I fail to see how everyone loves the hardocp benchmarks. Well guess what people thoose are made with an in-game run not some demo you can compare to. How is using a demo you dont have and cant compare against, any more valuable then a benchmark tool that uses many DX paths and effects yet does not utilize an actualy game engine ?

 

[Scali]

3dMark is not an awesome tool for more or less anything else. It's kind of a guide to the overall ability of various Videocards, but it's not accurate as a battery of Real Life games you actually play being run on the same system with each of the cards you are comparing.

This is true... two common mistakes that people make with interpreting 3DMark results are:

1) They think that the average framerate of the 3DMark tests will correspond with the framerate in their favourite game.

2) They think that a card performing well in 3DMark is a guarantee for the card performing well in their favourite game, or vice versa.

For 1) it should be obvious that while 3DMark aims to simulate a variety of game engines, the 3 or 4 tests they offer, can only cover so much ground. Firstly, 3DMark always aims at the next generation of games, so the framerates will generally be lower than current generation games. Secondly, not all games of a certain generation will perform the same. Some games simply get higher framerates than others, because of the way the engine renders things. And ofcourse, these framerates can differ from card to card. Half-Life 2 and Doom3 are two nice examples. Half-Life 2 will perform slightly better on ATi cards, and Doom3 will perform slightly better on NVIDIA cards. While some people think this is a conspiracy, it's simply a result of the fact that one brand performs better on feature A, and another brand performs better on feature B. Whichever feature is used most by a game, will determine which brand will perform better in that game.

2) is pretty much a result of this. Just because your favourite game performs well on a certain card, doesn't mean all games, or 3DMark for that matter, have to perform well on that same card. It all depends on the way things are rendered. Games that work in a similar way, will get similar performance. That's the idea behind 3DMark's game tests anyway. They try to make the tests work similar to a few popular future games. In 3DMark03 it should be apparent that Game Test 2 was based on Doom3, and that the performance of Game Test 2 was quite a good indication of performance in Doom3 (assuming Doom3 is configured to highest quality, shadows enabled etc).
Since there is no game available yet, on which 3DMark05's game tests are based, we can't really say whether 3DMark05 gives a good indication or not. Personally I see no reason to doubt the Futuremark team though, their choices for high polycount, long shaders and shadowmapping seem to make sense for the future of gaming.

 

[pxc]

no reason to use it in reviews then:

http://hardocp.com/article.html?art=Njk4LDU=

Sadly, 3DMark2001SE (with it's flaws) was the last useful version.

And if you look at what Morry is measuring (not video performance vs other video cards... he's using it to test system vs system performance), the choice makes sense.

 

[Scali]

LOL

Yes, because a less than 10% difference in 3DMark translates to a 30-50% performance difference in games. 3DMark apologists are funny. It's a junk synthetic benchmark, deal with it.

Apparently you didn't read my whole post, or didn't understand it. So let me try and explain it again:
The polycount in 3DMark05 is not representative for any current game. Try 3DMark03 for a lower polycount, and more or less representative for current games like Doom3, Half-Life 2 and FarCry.

Or, in other words... as polycount gets higher, performance differences between 9600Pro and X300SE get smaller, since memory speed becomes less of a factor in performance.
Conversely, as polycount gets lower, differences get larger.

 

[pxc]

The polycount in 3DMark05 is not representative for any current game.

That's exactly why it's worthless. Not only that, but the engine used doesn't represent any game currently used or likely to be used in the future.

Try 3DMark03 for a lower polycount, and more or less representative for current games like Doom3, Half-Life 2 and FarCry.

Nope, each of the 3 games you listed runs better than the synthetic tests (Proxicon/nature) suggests. And it doesn't even give accurate relative performance between video cards.

3DMark03 and 3DMark05 blew it big time. Both are worthless for predicting game performance, or even relative game performance between different video cards. It's not just [H] that doesn't use those, 03 and 05 are far less common in reviews than 2001 was. Most people recognize the flaws since 03 that were continued with 05. But I guess a few people haven't.

 

[Scali]

That's exactly why it's worthless. Not only that, but the engine used doesn't represent any game currently used or likely to be used in the future.

Care to explain why not? I gave arguments to the opposite in my previous post.
If you don't even bother challenging those, I see no reason to take your word for it.

Nope, each of the 3 games you listed runs better than the synthetic tests (Proxicon/nature) suggests. And it doesn't even give accurate relative performance between video cards.

You seem to have fallen in one of the traps that I explained in my earlier post. And I also think you misunderstood what I said. I said that the polycount of 3DMark03 is more or less representative to those games, not the framerate (should be obvious anyway, since the framerates between those 3 games tend to differ quite a bit aswell). The absolute framerate is not important anyway, only the relative framerate is.

 

[pxc]

Care to explain why not? I gave arguments to the opposite in my previous post.
If you don't even bother challenging those, I see no reason to take your word for it.

I saw your opinion, but no arguments. Carry on.

There's no point discussing this with you because you believe 3DMark is great, and you don't even read my posts or look at the relative results of video cards in 3DMark vs games.

 

[Scali]

I saw your opinion, but no arguments. Carry on.

Let me quote myself to make sure we are talking about the same thing here:

Since there is no game available yet, on which 3DMark05's game tests are based, we can't really say whether 3DMark05 gives a good indication or not. Personally I see no reason to doubt the Futuremark team though, their choices for high polycount, long shaders and shadowmapping seem to make sense for the future of gaming.

There's no point discussing this with you because you believe 3DMark is great, and you don't even read my posts or look at the relative results of video cards in 3DMark vs games.

Apparently you're the one not reading my posts, and not giving any arguments to support your view. I give arguments to support mine. There's no discussion if you don't present arguments. And I never said that I believe 3DMark is great, I just gave some tips on how it should or shouldn't be used.

 

[pxc]

Apparently you're the one not reading my posts, and not giving any arguments to support your view.

I did read your posts and it's just your personal opinions. The fact is that I gave data and you did not.

BTW, there were no 3dMark03 game engines... ever. I wouldn't hold my breath for games based on the 3DMark05 game engine. If anything ran as poorly as 3DMark03/05 does, no one would buy the game(s). LOL

 

[bonkrowave]

I saw your opinion, but no arguments. Carry on.

There's no point discussing this with you because you believe 3DMark is great, and you don't even read my posts or look at the relative results of video cards in 3DMark vs games.

Now thats an ironic statement... You on the opposite side, are a complete fan-boy who is totally unwilling to give any credit to 3dmark at all. Just the fact that you say 3dmark is worthless and is of no use at all, sure does wonders for your credibility /sarcasm.

 

[Scali]

I did read your posts and it's just your personal opinions. The fact is that I gave data and you did not.

I didn't have to present data, I used the data that you presented, and explained how to interpret it.

 

[jon67]

Sadly, 3DMark2001SE (with it's flaws) was the last useful version.

And if you look at what Morry is measuring (not video performance vs other video cards... he's using it to test system vs system performance), the choice makes sense.

system vs system, mobo vs mobo, video card vs video card, kramer vs kramer.... 3DMark is useful for comparison purposes, hence its output means something, period!

 

[rcf1987]

My 5600 ultra gets 4500 average in 3dmark03 it sucks. My Ti4600 gets like 1600 average and it kicks the 5600 ultra's butt anyday of the week.

 

[rcolbert]

3DMark is just a mine is bigger than yours number, but it's still useful to that end. Just like any measurement it doesn't take into account how you're going to use it. Although I don't think your woman is going to be too happy when you drop your drawers and demonstrate your 4" score on PenisMark05, no matter what you intend to do with it.

 

[Scali]

My 5600 ultra gets 4500 average in 3dmark03 it sucks. My Ti4600 gets like 1600 average and it kicks the 5600 ultra's butt anyday of the week.

You have also fallen in one of the traps I've mentioned earlier in this thread.

 

[pxc]

Now thats an ironic statement...

don'tchathink?

I said it's useless:
1) for predicting real game performance
2) for comparing different cards' relative performance

03 and 05 are almost entirely GPU benchmarks, at least for the reported "3DMarks" score. It lost the one good thing that made 3DMark2001SE useful, which was flawed from a GPU load perspecive on later/faster video cards but worked fine in its original time, as a whole system benchmark.

As you can see from modern games, a system at low resolution (1024x768) scales with 2 thing: system performance (faster CPUs and memory), and faster video cards. Once futuremark realizes that mistake, a future 3DMark may once again be useful.

The purely synthetic nature of the benchmark will be harder to overcome unless FM can license their engine again. But judging by the poor performance compared to real game engines, that's probably a lost cause.

 

[Scali]

03 and 05 are almost entirely GPU benchmarks, at least for the reported "3DMarks" score. It lost the one good thing that made 3DMark2001SE useful, which was flawed from a GPU load perspecive on later/faster video cards but worked fine in its original time, as a whole system benchmark.

As you can see from modern games, a system at low resolution (1024x768) scales with 2 thing: system performance (faster CPUs and memory), and faster video cards. Once futuremark realizes that mistake, a future 3DMark may once again be useful.

3DMark05 is already more CPU-intensive... we will have to see what happens with the next generation of videocards... it's rather hard to get CPU-limited when you're rendering < 15 fps.

The purely synthetic nature of the benchmark will be harder to overcome unless FM can license their engine again.

This makes absolutely no sense to me. In what way does a license change anything about the engine at all?
Oh and by the way, the early 3DMarks were built on the MAX-FX engine which was licensed FROM Remedy, BY Futuremark, not the other way around.

 

[pxc]

This makes absolutely no sense to me. In what way does a license change anything about the engine at all?

A benchmark based on a purely synthetic "gaming" engine measures nothing except the performance inside that gaming engine.

A benchmark based on or using a game engine that is used elsewhere, if properly written (i.e. no major changes), shows a more meaningful result since the same engine code is used in real game(s).

Make more sense now?

Maybe you can point out a game that gets in the low 30-fps range at 1024x768 so we can try to figure out exactly what 3DMark05 is attempting to represent? All I see is that Futuremark is attempting suicide. Twice.

 

[Scali]

A benchmark based on a purely synthetic "gaming" engine measures nothing except the performance inside that gaming engine.

A benchmark based on or using a game engine that is used elsewhere, if properly written (i.e. no major changes), shows a more meaningful result since the same engine code is used in real game(s).

Make more sense now?

No. I don't see why the result is suddenly meaningful when the SAME engine code is used in a game. It's the same code, so nothing changed. So I don't see how it can suddenly be meaningful.

Maybe you can point out a game that gets in the low 30-fps range at 1024x768 so we can try to figure out exactly what 3DMark05 is attempting to represent? All I see is that Futuremark is attempting suicide. Twice.

Perhaps the 3DMark05 name is rather subtle, so I'll give you a hint: It's not 2005 yet. The company name is another hint: Future-mark.

 

[coz]

Perhaps the 3DMark05 name is rather subtle, so I'll give you a hint: It's not 2005 yet. The company name is another hint: Future-mark.

Are you saying that the 3DM05 engine is in any way representative of the performance we'll see in the 3D game engines of 2005 (which is about 17 days away) or even 2006? I doubt we'll see any 2005 games running at 22fps average on a 6800U in 1024x768 with no AA.

As a benchmark it seems pretty useless because it doesn't tell us how well our hardware will perform in actual games - and isn't that the whole *point* of a benchmark? All this benchmark really tells us is how well the latest hardware runs 3DMark05 and how well ATi and NV have optimized their drivers for it (which they both do, bigtime).

It's useful for stress-testing though. Plus, it keeps hardcore tweakers/benchers occupied and amused for months!

 

[0ldman]

Considering the general masses used the battle of proxycon (whatever the name is) as the benchmark for whether you can play Doom3 or not for MONTHS, I do consider 3dmark to be near worlthless as a test of overall performance.

Considering I can play Doom3 nicely and still have yet to see that damn proxycon demo right...

 

[robberbaron]

Considering the general masses used the battle of proxycon (whatever the name is) as the benchmark for whether you can play Doom3 or not for MONTHS, I do consider 3dmark to be near worlthless as a test of overall performance.

Considering I can play Doom3 nicely and still have yet to see that damn proxycon demo right...

How could it at all be comparable to Doom3? It used a totally different engine. What, it had an outerspace theme?

 

[CleanSlate]

How could it at all be comparable to Doom3? It used a totally different engine. What, it had an outerspace theme?

ROFL, dude proxycon and doom3... that's like saying because I don't ike apples I won't eat oranges. No one thought that, at least no one I know.

~Adam

 

[ATW]

Sadly, 3DMark2001SE (with it's flaws) was the last useful version.

Agreed, 01 is the only version I use, and only do that to see the results of adjustments I've made. I've run 03 and 05 about two times apeice just to see where I fit in among the e-penises and uninstalled them.