Frame Latency - Why you should stop talking about FPS
- Thread starter mkenyon
- Start date
Yep, that's it.Interesting write up, didn't even know such a thing was a thing(?) So this is how long it takes for 1 frame to render?
Is this the graph? 5min match of BF3, all settings max, 64players.
If you are calculating the standard deviation, you already have your mean. And if the standard deviation is 0 in both cases, then of course we'll take the 60 fps games over the 50 fps.
This answer goes for both, here's Skyrim's frame latency with vsync turned on (is there any other setting :/) running at an indicated 60 fps. The worst of all is that AFAIK Skyrim does limit frame latency by rendering 3 frames ahead which is what causes some input lag.
As you can guess it is jerky to play. I tend not to notice the big, but short spikes, but a thick line is really awful to play with. Vsync that constantly switched between 17 and 33 ms frame time was especially horrible when compared to a different game running the same frame rate (40ish), but without vsync. A constant 33ms is much better than that shit.
It depends on the config and like mkenyon said SLI is more consistent. When it's bad it looks like the aforementioned horrible vsync jitter. Techraport checked it out in their first article on frame latencies.
Hope this new method of benchmarking games takes off; Tom's Hardware plans to feature it in their reviews going forward, along with their usual average fps data.
More info by TechReport for those interested: Inside the second: A new look at game benchmarking
More info by TechReport for those interested: Inside the second: A new look at game benchmarking
Tom's is doing a shit job of representing the data though. They're either purposefully dropping some really key information from their charts (95th percentile? seriously?) or don't understand why it's important. Then they fuel the fires of the AMD vs. Intel CPU wars and get a bunch of hits. Really really questionable stuff.
Heh that's true, I just stumbled upon their initiative while I was looking for TechReport's old article, which your OP reminded me of.
Anton668
Member
so tempted to add this to their comments to see if they reply
Using the %iles is a way to keep their articles neat and clean while still conveying relevant data.
I have a feeling as we get more and more data on this issue, people will begin to figure out which hardware choices will improve their desired gaming outcomes. It's not as if you can take this data and shop around to reduce bad latency times beyond the current method.
This answer goes for both, here's Skyrim's frame latency with vsync turned on (is there any other setting :/) running at an indicated 60 fps. The worst of all is that AFAIK Skyrim does limit frame latency by rendering 3 frames ahead which is what causes some input lag.
As you can guess it is jerky to play. I tend not to notice the big, but short spikes, but a thick line is really awful to play with. Vsync that constantly switched between 17 and 33 ms frame time was especially horrible when compared to a different game running the same frame rate (40ish), but without vsync. A constant 33ms is much better than that shit.
Wow, that's disappointing.
The only way that would be happening is if the video card ends up outputting more than 1 frame in a 1/60s time slice to make up for the missed frame. I wonder if that's how they achieve higher benchmark numbers...by essentially outputting useless frames.
Yes, but they are choosing 95% instead of 99%, thus covering up that 2% (compared to 99%) of data where things are going wrong.
Look at what this chart would indicate if the top 2.5% was thrown out as compared to the top .5%.
Oh wow. That response clearly demonstrates they don't even know what the data means.
The 95% throws out the top and bottom 2.5% of data to say 'What is the frame time that I can expect 95% of all frames to be rendered at?'
It's a metric long used to assess server performance.
No, what you are referring to is 2 standard deviations away from the mean = 95% of the area under the distribution starting from the mean in both directions.
95%ile is the top 5%.
phosphor112
Banned
When I say FPS, I mean average.
Here's why:
In recent comparisons, AMD gets better "FPS" but gets higher micro-stuttering.
nVidia have been getting lower "FPS" but gets much less micro-stuttering.
Either way, I don't care. I just want high frame rates.
Here's why:
In recent comparisons, AMD gets better "FPS" but gets higher micro-stuttering.
nVidia have been getting lower "FPS" but gets much less micro-stuttering.
Either way, I don't care. I just want high frame rates.
Really glad this thing has reared its ugly face, now every time I mention the word "SLI", 15 seconds later there's a "OMG LATENCY" comment, really is beautiful.
Especially because I can't feel the difference whatsoever. You could sit me down in front of a new generation single and SLI PC and promise me eternal wealth and I would never be able to tell.
Especially because I can't feel the difference whatsoever. You could sit me down in front of a new generation single and SLI PC and promise me eternal wealth and I would never be able to tell.
Hrm. Well, where I get my argument from is TechReport:
http://techreport.com/review/21516/inside-the-second-a-new-look-at-game-benchmarking/3
Not everyone can be a pro gamer Sethos, it's okay.
They are saying the same exact thing Tom's website is saying. 99%ile is looking at the 1%.
zephervack
Member
So in a perfect world we would like to shop for a card that gets a +60fps on the 0.1% and 1% of this graph, right???
Also is this what is commonly called "microstuttering", I remember having my crossfire setup saying I was hitting 100fps yet it felt more like 25fps some times.
Also is this what is commonly called "microstuttering", I remember having my crossfire setup saying I was hitting 100fps yet it felt more like 25fps some times.
Not everyone can be a pro gamer Sethos, it's okay.
Which is funny because I'm an excellent player when it comes to FPS games and most people wouldn't notice SLI frame latency whatsoever. However, now it's suddenly a 'thing' and there's graphs and write-ups everywhere, now everybody suddenly feel a 3 day delay and are professors in frame latency.
Felix Lighter
Member
I've noticed something very frustrating but common and I've mentioned it quite a few times in many threads. I often get stutter when in full screen mode but when I switch to windowed mode it vanishes. Is there a known link between full screen mode and frame latency?
Maybe I'm not understanding how this: "(Ruling out that last 1% allows us to filter out any weird outliers.)" equates to "focusing on the 1%". That seems to indicate the opposite.
Stallion Free
Cock Encumbered
Someone pass the word on to Digital Foundry.
Edit: I wasn't clear with my wording from the previous post.
It indicates the opposite because they are using the term "percentile" incorrectly. Even if it was reversed (which it is not), there is no cutting off of the 5% at 2.5% on each end.
95%ile means - top 5%
99%ile means - top 1%
The copy machine people are looking for the number X over the shortest 99% of cases. That is not the 99%ile.
Returning to your graph:
The 95% would include all data from the 95% mark to "100%"
Yeah, they can add this information to their 720p -> 360p downsampled screenshots.
*edit*
okay, I get what you are saying now. What's the accurate term then for looking at the 99% of results?
*edit 2*
Actually, isn't it more accurate to say 99% = the number separating the 99% from 1%? Or, the number separating 95% from the 5%? It would be the number right on the line, correct?
Stallion Free
Cock Encumbered
"Our budget PC had trouble rendering out the graphs for the Frame Latency so they couldn't be included in this comparison"
GHG
Member
I'm exactly the same so unless data like this pulls out some really ugly results I don't really care as it makes no difference to me.
Well, I'm not sure what the technical term for looking at the bottom 99% of the data is, lol.
You wouldn't want to use a number or data point to separate the data because then you wouldn't have any place for that number to go. I suppose if we had the numbers 1-100, the 95%ile numbers would be 96 and up. But then 95 would be part of the lower group.
If we had 40 frames of data, the top 10 frames with respect to latency would make up the 75%ile.
Okay, I think we are saying the same thing but in different ways.
Where it gets confusing is that the higher % you go in frame time, the WORSE the result is. This is different than the normal way of looking at percentile. Having a number LOWER than the 4% above it misrepresents that 4% of data.
Where it gets confusing is that the higher % you go in frame time, the WORSE the result is. This is different than the normal way of looking at percentile. Having a number LOWER than the 4% above it misrepresents that 4% of data.
percentiles aren't about your opinion/definition of the best or worst. they are based on how you set your data up. we could easily take 95%ile of lowest latency - so that we could see the 5% of frames with the lowest rendering time. it's just that tom's website is gathering the "high render time" frames. you can easily make your own chart to show results how you want
Oh come on, take a good hearted jab.
Me, NVIDIA, and most of the gaming community would agree with your sentiment because most people don't even know about it.
Every setup is different though, so if the person already has a substantial amount of input lag due to their monitor and input devices, that 1 frame might be the thing that pushes it over the limits of perception.
ChoklitReign
Member
Will those drivers affect latency for 6000 series cards like my 6850? Skyrim has had really bad latency for me.
Yeah, basically we still need the FPS but accompanied with the frame latency standard deviation.
Now that I understand this whole thing better, when you say 99th percentile, you are not looking at the top 1%. You are looking at the number that separates the top 1% from the bottom 99%. Thus, when you say 95th percentile, you are looking at the number that separates the top 5% from the bottom 99%.percentiles aren't about your opinion/definition of the best or worst. they are based on how you set your data up. we could easily take 95%ile of lowest latency - so that we could see the 5% of frames with the lowest rendering time. it's just that tom's website is gathering the "high render time" frames. you can easily make your own chart to show results how you want
In this light, I think the purpose, my critique of Tom's approach, and TechReport's use is accurate.
It can be used either way. In the context of Tom's graphs, 95%ile clearly refers to all cases outside of that 95%. From your wikipedia link "For example, if a score is in the 86th percentile, it is higher than 85% of the other scores." so 86%ile means 85% of scores below. The grouping above is more or less irrelevant.
Percentiles just split up the data so we can look at it in meaningful ways.
methane47
Member
Question,
is this the reason why there are some games that are 30fps and yet some praise the game for how smooth it is DESPITE being only 30fps?
E.g (Resistance Fall of Man)
Or more specifically, is the smooth(isity) of gameplay dependent more on the standard deviation of Latency rather than the Framerate?
So if we played a game at 20fps but latency std/dev was 0ms
vs a game played at 40fps with latency 10ms
Are you suggesting that this could cause a lower FPS to appear on par?
is this the reason why there are some games that are 30fps and yet some praise the game for how smooth it is DESPITE being only 30fps?
E.g (Resistance Fall of Man)
Or more specifically, is the smooth(isity) of gameplay dependent more on the standard deviation of Latency rather than the Framerate?
So if we played a game at 20fps but latency std/dev was 0ms
vs a game played at 40fps with latency 10ms
Are you suggesting that this could cause a lower FPS to appear on par?
That does seem to follow.
You know, if 30 FPS wasn't complete crap from the beginning.
Genuinely though, yes. A frame latency graph showing a consistent 33.3ms frame time would appear to be more smooth than one which oscillates between 16.7 and 33.3.
Thank you sincerely for helping me with that. GAF is the best.
Similar threads
- Poll