Finally,objective 7970CF vs GTX 690 microstutter results

X

xoleras

2[H]4U
Joined
Oct 11, 2011
Messages
3,551
http://techreport.com/articles.x/22890/7

bf3-fps.gif

bf3-beyond-50.gif

bf3-beyond-33.gif

bf3-beyond-16.gif


Objective data on MS on both setups.
 
Interesting, their data analysis sucks balls though. If I could get my hands on the frametime vs frame number data in a text file I could produce some more interesting and meaningful plots. The most interesting ones to me (of what they presented) are just the time series, the width of the frametime vs framenumber plot is showing how much the frametime is oscillating, so how much microstutter there might be. The 680, 7970, 680 SLI and 690 have nice narrow bands so frame rendering time is more consistent, compared to the 7970 CF, 6990 and GTX590 which have thicker bands and so less consistent frametimes, despite maintaining a good mean framerate. This would mean the latter cards "may" have good framerates, yet be less playable at those framerates.
 
Tudz said:
Interesting, their data analysis sucks balls though. If I could get my hands on the frametime vs frame number data in a text file I could produce some more interesting and meaningful plots. The most interesting ones to me (of what they presented) are just the time series, the width of the frametime vs framenumber plot is showing how much the frametime is oscillating, so how much microstutter there might be. The 680, 7970, 680 SLI and 690 have nice narrow bands so frame rendering time is more consistent, compared to the 7970 CF, 6990 and GTX590 which have thicker bands and so less consistent frametimes, despite maintaining a good mean framerate. This would mean the latter cards "may" have good framerates, yet be less playable at those framerates.
Click to expand...

If you bothered to read their analysis, that's precisely what they point out with their 99th percentile graph as well as the Frame latencies by percentile graph... and they even put it into words in case the graphs were too cryptic for a 5 second glance.
 
Sorry mate, but it is still not possible to measure frame metering with fraps. There was an Nvidia Q&A yesterday on a German website and I asked their rep specifically that. So this analysis is of little value.

Edit:
And what rincewind said. The 99% percentile are...suboptimal if you want to look at microstutter. Raw data is much better.
 
Last edited:
Not sure why you linked those particular graphs, the frametimes over time highlights the microstutter/variance much better. As a final piece of data those are really nothing more than 'amount of time spent where framerate was below a given target', which is just an overall performance metric without the big picture of when these drops occurred. 30 seconds at 60fps followed by 30 seconds at 30fps would mean that 50% of the time (and 1/3 of the frames) did not meet 16.7ms target, but there would be no microstutter there.

I think their final data would be better presented as number of frames where the frametime deviated from the previous one by more than X.
With standard vsync enabled you could simply record the 'framerate' as 60/30/20 etc and record fluctuations based on this.
 
boxleitnerb said:
Sorry mate, but it is still not possible to measure frame metering with fraps. There was an Nvidia Q&A yesterday on a German website and I asked their rep specifically that. So this analysis is of little value.

Edit:
And what rincewind said. The 99% percentile are...suboptimal if you want to look at microstutter. Raw data is much better.
Click to expand...

Either way it underscores the need for an objective measure of MS. And there does seem to be a large disparity from last gen to current gen in terms of frame latency, and that is consistent. I'm sick and tired personally of hearing subjective thoughts on it - some swear its there, others not so much. I have not seen MS in years, since the 5k series. Have not seen it with 580 sli, 7970cf, 680 sli, Not with vsync enabled, anyway.

Wouldn't you agree that an objective measure that tells absolute factual measures of microstutter is desirable? Maybe these tests aren't it, but we need it.
 
Would you rather they don't say a word about it when there actually is/might be a difference? But lets hope Kyle and Nvidia come up with a way to measure this. But then people might say, it is biased in Nvidias favor since they have a hand in it...
 
boxleitnerb said:
Would you rather they don't say a word about it when there actually is/might be a difference? But lets hope Kyle and Nvidia come up with a way to measure this. But then people might say, it is biased in Nvidias favor since they have a hand in it...
Click to expand...

Kyle and nvidia come up with a way to measure it? What?
 
xoleras said:
Kyle and nvidia come up with a way to measure it? What?
Click to expand...

This page is much better at showing microstutter. The way he summed it up was foolish. He needed to use a "derivative approach"

http://techreport.com/articles.x/22890/3

However, as stated, there is no easy method when the frame actually gets loaded into the output buffer with multi GPU. Kyle and crew are workng with nVidia to get an absolute frame time. However, if nVidia would just give one pin on the BGA to pulse when a new frame is loaded to a monitor...a simple microcontroller would be enough. However, that is extra silicon..and as a share holder I would be pissed if they would leave in "non useful" silicon and hardware. I'm gong to guess is that what they are working on is something that looks at the video output stream and looks for start of frame when being pushed to the dispaly. This would require something that is "very fast" and thus why the "not cheap" came out off Kyle.
 
Last edited:
Trepidati0n said:
Kyle and crew are working with nVidia to actually measure "absolute frame time" for all frames in a bench (run through). This will not be cheap unfortunately.

You would think though that nVidia would have just one damn pin on the BGA which pulses when a new frame is loaded into the buffer. Then it would require nothing more than a cheap ass microcontorller with a USB interface and a timer/counter to get that data.
Click to expand...

Gotcha. I'd welcome any sort of test that is objective.
 
to this day i have never experienced micro stutter on any of my SLI systems i have owned over the years. i have seen video of it and know what it is but i have never personally seen it on my machines.


i think there are other issues when someone experiences micro stuttering going on in the machine.
 
Since we're not entirely confident in our numbers, I'll offer a few subjective impressions from my testing, as well. I started the process with the dual 7970 CrossFire team, and I chose the image quality settings for testing based on how well that config could run each game. My goal was to stress the multi-GPU solutions appropriately without cranking up the image quality options so far the games would be unplayable on the single-GPU cards.
Click to expand...
They dumbed down the test
 
Intel_Hydralisk said:
If you bothered to read their analysis, that's precisely what they point out with their 99th percentile graph as well as the Frame latencies by percentile graph... and they even put it into words in case the graphs were too cryptic for a 5 second glance.
Click to expand...

I did read through, just saying that with some better data analysis they could have shown it more clearly rather than having to explain it with abstract graphs of time spent at different frame delays and integral of percentage vs frame time ;)
 
So much for objective...but in line with what [H] has been saying about SLI and CF.

The GTX 680 SLI config and the GTX 690 came next, and subjectively, the experience offered by the two was indistinguishable. Both were obviously faster in places where the GTX 680 felt strained, and I'd say they offered a better experience overall—and thus the best of any config I tested. However, it seemed like they'd still run into occasional, brief episodes of sluggishness that one didn't experience on a single GTX 680.
Click to expand...
 
While this can be quantified as much as you want, user preference and sensitivity are also key attributes. Unfortunately the only way to really know if it affects you is to actually try it out :(
 
More like awareness of the issue. People who don't know what it is will probably just take it for low fps/poor performance, and people who think they know what it is may blame it for all other sorts of 'stutter'.

But I've little doubt that most anyone, whether aware of the issue or not, would prefer the gaming experience given by a single-GPU setup running at ~40fps to that of a multi-GPU setup running at ~40fps if they got to try it and watch it in a side-by-side comparison.
 
AlexRiker said:
Frametimes are NOT a valid measure of microstutter, see:

http://techreport.com/articles.x/21516/11

Be nice if they read their own articles, tho.
Click to expand...

Hmm, reading through that it sounds like frame times is a valid measure, just not the frame times as seen by FRAPs. Though I don't quite understand the reasoning for that, does FRAPs measure the frame time before the GPU flips to the new buffer, meaning the user is seeing a frame delay different to what FRAPs is reporting?
 
I think the "time spent beyond X ms" is still valid as a measure of perceived FPS, whether it's impacted by microstutter or macrostutter (aka sudden drops of framerate). It just doesn't isolate microstutter well because it counts prolonged drops in FPS as wells as drops between individual frames (and apparently, doesn't account for frame metering).

If you wanted to really measure microstutter alone you'd want to take the average frame time difference between adjacent frames (for dual GPU) or adjacent frame triplets (for triple GPU).
 
You might check a bit more into exactly how TechReport collects its frametime data. There are a bunch of us that do not believe the data shown is how exactly the user actually sees it. At least the last time I checked.
 
I find the results a little odd when they say that 680 SLI performs better than a single 680 (in terms of micro-stutter).
 
You must log in or register to reply here.
Back
Top