CPU Utilization is Wrong

rgMekanic

[H]ard|News
Joined
May 13, 2013
Messages
5,912
In May of last year, senior performance architect at Netflix, Brendan Gregg posted an interesting article about how the "%CPU" metric is wrong, and is progressively getting worse. Now, Brendan expands on his findings in a 5 minute video from the Southern California Linux Expo. The UpSCALE Lightning Talk from Opensource.com goes over his original idea very well, but also shows an interesting conclusion.

Check out the video

In his Lightning Talk, "CPU Utilization is Wrong," Brendan explains what CPU utilization means—and doesn't mean—about performance and shares the open source tools he uses to identify reasons for bottlenecks and tune Netflix's systems. He also includes a mysterious case study that's relevant to everyone in 2018.
 
  • Like
Reactions: Aenra
like this

cyclone3d

[H]F Junkie
Joined
Aug 16, 2004
Messages
14,851
Sooo.. the whole point of the video was to show that the Meltdown and Spectre patches are causing a 26% slowdown because they flush the TLB cache thus causing a massive number of cache misses.

But instead of just getting to the point, the speaker just said that CPU Utilization is incorrect...... :rolleyes:
 

Shmee

[H]ard|Gawd
Joined
Sep 12, 2014
Messages
1,148
Sooo.. the whole point of the video was to show that the Meltdown and Spectre patches are causing a 26% slowdown because they flush the TLB cache thus causing a massive number of cache misses.

But instead of just getting to the point, the speaker just said that CPU Utilization is incorrect...... :rolleyes:

But it is incorrect.
 

dgingeri

2[H]4U
Joined
Dec 5, 2004
Messages
2,830
Pretty straightforward. It's pretty direct that cache misses cause an increase in what the metric measures as CPU utilization when that is not in fact the case. Good point. The specific case he uses is specific to Intel CPUs and the Meltdown patches, but it is applicable to CPUs in general waiting on main memory reads, which happens all the time.
 

SvenBent

2[H]4U
Joined
Sep 13, 2008
Messages
3,310
its complete correct. You just have to understand what it is you are measuring and stop confussing it with something you believe it is.
 
Joined
Jun 6, 2017
Messages
3
I've been saying this for almost 10 years... Why did I not present on this so many years ago? This isn't smart stuff to me, its duh stuff! This occurs with memory usage too and I should have presented and wrote a white paper on this.
 

bitbum

Gawd
Joined
Mar 10, 2003
Messages
517
If you follow Brendans work you'll know that hes been doing this kind of analysis for years. Usually it has to do with spending inordinate cycles executing a particular code block. The result of the inefficient code produces similar results as an inefficient processor. Not so surprising.
 

TordanGow

[H]ard|Gawd
Joined
May 25, 2015
Messages
1,490
For one thing anyone that is managing large scale server deployments knows about this. Second, if only we had a way to measure external delays like I/O wait... oh what's that, we already do?

I'm a dope and even I know this.
 

ecktt

Limp Gawd
Joined
Oct 22, 2004
Messages
415
Wow. He wrote all those tools and still screwed the pooch on a process waiting on input causing erroneous CPU utilization. Yup, it's a Linux piece. No wonder. Yeah, these guys need to go back and read "Operating Systems" by the legendary Andrew S Tanenbaum. Instead of saying a Context Switches gets more expensive, he makes an absolutely wrong statement about misleading CPU utilization.
 

velusip

[H]ard|Gawd
Joined
Jan 24, 2005
Messages
1,579
Well, the CPU utilization metric is correct if you plan on using it to throttle similarly bottlenecked code.
 

the_real_7

[H]ard|Gawd
Joined
Sep 10, 2007
Messages
1,344
I skipped the meltdown and spectre patches and bios updates .I haven't seen any incident yet on my clients pc are any of mine. These patches do more harm than help, you can see that in benches.
 

naib

[H]ard|Gawd
Joined
Jul 26, 2013
Messages
1,289
4 in 10 Americans think the earth is less than 10,000 years old so when you apply that same level of tech ignorance then it becomes abundantly clear that not everyone knew this.
Well there is that :)
I guess what I meant was every Linux user knows (or should know) this. These are Linux tools being demonstrated and any sysadmin needs to know how to track down io bound tasks


Windows is different as MS doesn't make this obvious.

Fundamentally this isn't CPU loading this is task scheduler loading. A CPU is always working, aspects maybe unlocked to save power BUT it is still used.
An OS scheduler however is different
 

PaulP

Gawd
Joined
Oct 31, 2016
Messages
776
4 in 10 Americans think the earth is less than 10,000 years old so when you apply that same level of tech ignorance then it becomes abundantly clear that not everyone knew this.
Have a source for that stat, or did you just make it up?
 

PaulP

Gawd
Joined
Oct 31, 2016
Messages
776
http://bfy.tw/HwPo

Sadly, it seems to be the case.
All I could find was a bullshit Gallop poll that gave people only three choices: evolution with God's help, evolution without God's help, and creation 10,000 years ago. That leaves out a lot of people that believe in creationism but have other ideas on the timeline, including the idea that the Earth is millions of years (or more) old. These people will not select the first two answers, so get lumped in with the "young earth" creationists. So really all that poll proves is that 40% of the people in this country believe in creationism. I'll bet that percentage is much higher in Muslim countries. Does that make them backwards and stupid too?
 

Ski

[H]ard|Gawd
Joined
Jun 21, 2008
Messages
1,038
Again, this backs up any statement that more or less says: "Most people are idiots."

giphy.gif
 

xorbe

Supreme [H]ardness
Joined
Sep 26, 2008
Messages
6,029
100% utilization doesn't mean 100% max load. It just means the kernel scheduler had something to run other than idle the cpu thread resource. It's always been a pita to quantify cpu and memory usage, everyone wants to know something slightly different.
 

ecuador

Limp Gawd
Joined
Dec 29, 2008
Messages
207
All I could find was a bullshit Gallop poll that gave people only three choices: evolution with God's help, evolution without God's help, and creation 10,000 years ago. That leaves out a lot of people that believe in creationism but have other ideas on the timeline, including the idea that the Earth is millions of years (or more) old. These people will not select the first two answers, so get lumped in with the "young earth" creationists. So really all that poll proves is that 40% of the people in this country believe in creationism. I'll bet that percentage is much higher in Muslim countries. Does that make them backwards and stupid too?

No, you did not have to choose only one of those three, that's why the combined percentage is not 100%. The category you are describing would obviously choose "none of the above", which seems to be a 5%.
 
Top