Software to monitor PCI-E bandwidth usage

Z

ZodaEX

Supreme [H]ardness
Joined
Sep 17, 2004
Messages
4,901
I'm curious how saturated my PCI-E bandwidth is. Is there a program that can show me how much of it is being used at any given time?
 
lopoetve said:
…. Why?
Click to expand...
this sounds interesting. It would be nice to monitor to see if your GPU is PCI Express limited or if other things are saturating it. If your using all your PCI-E lanes and need to run things on a limited speed like GPU at 8x, HBA 4x, M.2 2x, and 10GbE as 2x. You could see if a device is not maxing a link and could lower its total lanes to an even lower number and give the extra lanes to another device.

Using 16x lanes as an example but I could see this being a legitimate thing if you are on an Intel build with 40 lanes or whatever and you need to optimize your use of the lanes.
 
An Aberrant Person said:
this sounds interesting. It would be nice to monitor to see if your GPU is PCI Express limited or if other things are saturating it. If your using all your PCI-E lanes and need to run things on a limited speed like GPU at 8x, HBA 4x, M.2 2x, and 10GbE as 2x. You could see if a device is not maxing a link and could lower its total lanes to an even lower number and give the extra lanes to another device.

Using 16x lanes as an example but I could see this being a legitimate thing if you are on an Intel build with 40 lanes or whatever and you need to optimize your use of the lanes.
Click to expand...
That’s not quite how it works unless you have a PLX chip. Lanes are dedicated- you can’t personally choose to cut the number of lanes assigned to a slot to add more hardware. The motherboard manufacturer makes that choice - and you can’t share, so if they tied an M.2 slot to the same lanes as the SATA ports, for example, you pick which of the two to use. You can’t do both.

Regardless, which ever configuration you have, the lanes don’t share (chipset to CPU aside), so you can effectively maximize usage and never bottleneck anything else
 
  • Like
Reactions: Dan_D
like this
ZodaEX said:
Because i'm on PCI-E 2.0 and I want to know if my PCI-E bandwidth is limiting my performance on a AMD RX 6400 GPU which only has 4X lanes.
Click to expand...

Possibly. I was running a 2070s at PCIE 1.0 speeds because of ignorance and straight up lane shenanigans.

Being honest with myself, I never noticed a difference until I ran the steam survey and it reported the GPU to be running at PCIE 1.
So stop me if you have heard this before, it really depends on your workload.
 
lopoetve said:
That’s not quite how it works unless you have a PLX chip. Lanes are dedicated- you can’t personally choose to cut the number of lanes assigned to a slot to add more hardware. The motherboard manufacturer makes that choice - and you can’t share, so if they tied an M.2 slot to the same lanes as the SATA ports, for example, you pick which of the two to use. You can’t do both.

Regardless, which ever configuration you have, the lanes don’t share (chipset to CPU aside), so you can effectively maximize usage and never bottleneck anything else
Click to expand...
So on ATX or EATX motherboards where you have like 5 or 7 PCI-E slots, you can set in BIOS how many lanes you want to use for whatever slot. So if I have a 1650v3 I have only 40 lanes. In a hypothetical situation with 6 devices, I would want GPU gets 16x, GPU gets 16x, M.2 RAID gets 16x, HBA gets 4x,10GbE gets 4x, M.2 gets 4. But I can't have that because that would be 60 lanes. I only get 40 lanes so I would have to set certain things at reduced lane speeds. If I could see and log what their PCI-E bandwidth was I could tell what is and isn't bandwidth starved. If I had those 6 items, I would realistically have to do something like this, GPU gets 8x, GPU gets 8x, M.2 RAID gets 8x, HBA gets 4x,10GbE gets 4x, M.2 gets 4, which is 36 lanes. But if 10GbE and single M.2 isn't heavily used I could run those at 2x and get the extra 4x I would need to be able to set my m.2 RAID to 16x or a GPU to 16x if those were bandwidth staved.

So yes, there is a use case for this but an extremely niche use case
 
Tactlesss said:
Possibly. I was running a 2070s at PCIE 1.0 speeds because of ignorance and straight up lane shenanigans.

Being honest with myself, I never noticed a difference until I ran the steam survey and it reported the GPU to be running at PCIE 1.
So stop me if you have heard this before, it really depends on your workload.
Click to expand...
ZodaEX said:
Because i'm on PCI-E 2.0 and I want to know if my PCI-E bandwidth is limiting my performance on a AMD RX 6400 GPU which only has 4X lanes.
Click to expand...

Modern GPUs are very much PCI-E bandwidth starves especially some games show over 50% hit in FPS. That AMD 4x card is trash even in PCI-E 4.0 going to 3.0 murders it. It is a chunk card everyone should avoid running in anything other than PCI-E 4.0

~30% hit in FPS in PCIe 3.0 in this game. The average is like 25% so running PCIe 2.0 is even worse. What possed you to buy that card? An nvidia GTX1650 is literally a better card. Even an RX 570 is faster. Hell a used GTX 1060 is faster than all of those.
https://www.techpowerup.com/review/amd-radeon-rx-6400-pci-express-30-scaling/12.html

In this game it looses 50% performance!!! So at 2.0 You might have what 25% performance or some horrible figure!
https://www.techpowerup.com/review/amd-radeon-rx-6400-pci-express-30-scaling/15.html
 
An Aberrant Person said:
So on ATX or EATX motherboards where you have like 5 or 7 PCI-E slots, you can set in BIOS how many lanes you want to use for whatever slot. So if I have a 1650v3 I have only 40 lanes. In a hypothetical situation with 6 devices, I would want GPU gets 16x, GPU gets 16x, M.2 RAID gets 16x, HBA gets 4x,10GbE gets 4x, M.2 gets 4. But I can't have that because that would be 60 lanes. I only get 40 lanes so I would have to set certain things at reduced lane speeds. If I could see and log what their PCI-E bandwidth was I could tell what is and isn't bandwidth starved. If I had those 6 items, I would realistically have to do something like this, GPU gets 8x, GPU gets 8x, M.2 RAID gets 8x, HBA gets 4x,10GbE gets 4x, M.2 gets 4, which is 36 lanes. But if 10GbE and single M.2 isn't heavily used I could run those at 2x and get the extra 4x I would need to be able to set my m.2 RAID to 16x or a GPU to 16x if those were bandwidth staved.

So yes, there is a use case for this but an extremely niche use case
Click to expand...
That’s not how that works. And you can’t do that on any platform today.
 
  • Like
Reactions: Dan_D
like this
lopoetve said:
That’s not how that works. And you can’t do that on any platform today.
Click to expand...
I can do it on my 1650v3 X99 (IIRC) and other motherboards that aren't that old. IB-Skylake. How else would your computer divvy up lanes between 5 or 7 slots? Your CPU never has enough lanes. If you have 4x 16 slots and like a few 8x or 4x slots and your CPU has 40 or 56 lanes you can't run them all at max....how else is that going to work?

Please explain how that works if not like I said. (which is does work as I said.)
 
An Aberrant Person said:
Modern GPUs are very much PCI-E bandwidth starves especially some games show over 50% hit in FPS. That AMD 4x card is trash even in PCI-E 4.0 going to 3.0 murders it. It is a chunk card everyone should avoid running in anything other than PCI-E 4.0

~30% hit in FPS in PCIe 3.0 in this game. The average is like 25% so running PCIe 2.0 is even worse. What possed you to buy that card? An nvidia GTX1650 is literally a better card. Even an RX 570 is faster. Hell a used GTX 1060 is faster than all of those.
https://www.techpowerup.com/review/amd-radeon-rx-6400-pci-express-30-scaling/12.html

In this game it looses 50% performance!!! So at 2.0 You might have what 25% performance or some horrible figure!
https://www.techpowerup.com/review/amd-radeon-rx-6400-pci-express-30-scaling/15.html
Click to expand...

Lately I've been using Linux more than Win7 and the graphics card drivers on the AND side are a lot better than Nvidia's in Linux. Plus with it my PC supports ray tracing now. My Intel HD 2000 graphics handles video encoding just fine for Twitch streaming.
 
An Aberrant Person said:
I can do it on my 1650v3 X99 (IIRC) and other motherboards that aren't that old. IB-Skylake. How else would your computer divvy up lanes between 5 or 7 slots? Your CPU never has enough lanes. If you have 4x 16 slots and like a few 8x or 4x slots and your CPU has 40 or 56 lanes you can't run them all at max....how else is that going to work?

Please explain how that works if not like I said. (which is does work as I said.)
Click to expand...
Yes and no. You can choose to make a slot x16, x8, or x4/4/4/4, and you can lock it to a slower speed, but locking it to a slower speed doesn't free up lanes for other use - you also can't set an M2 drive to x2 or the like. Since each of those connections is 100% dedicated, it doesn't matter if it's fully utilized or not - slowing it down or choosing a smaller allocation won't free up lanes for things that aren't wired. If I set a PCIE slot to x8 instead of x16, I don't free up 2x x4 for M2 drives, for instance - that all depends on how they tied the "optional" lanes from the chipset to things (the CPUs tend to be hard locked to what they work on). And that optional setting tends to be very limited.

in your case, you can't set the M2 drives to x2 to "free up lanes" for the RAID device - those wires don't cross (literally) in this case, and they're tied to different things. Sorry - just doesn't work that way.

Without a PLX chip, that is - which is extremely rare these days.
 
lopoetve said:
That’s not how that works. And you can’t do that on any platform today.
Click to expand...
here is my MB it has 5x 3.0 16x slots and 1x 2.0 16x slot (PCH maybe?) My CPU does not support 80 lanes. It only supports 40 lanes. So I am forced to have to decide on what I want to run at 16x or 8x or 4x or 2x (not sure if 2x is an option but you get my point) So If I have all 5 filled. I have to choose what I want to run at what. So if I have something at 8x and its not maxing out. I can turn it to 4x and now I am using 4 less of my maximum 40 lanes, which means I can use that for something else.* So if these theotereical program existed. I would be able to see what is not being maxed and what is being maxed and than better optimize.

as I said...there is a weird niche use case. I would never need it but the question was "is there a use case for this" and yes there is.

https://www.newegg.com/p/N82E16813157536?Item=N82E16813157536

*how the fuck is this so complicated and I have wasted half a day explaining this to you. fuck

EDIT: the free up lanes dumb dumb is freeing up my 40 lanes limit. DO YOU UNDERSTAND NO NEED for PLX because this has nothing to do with plx o_O. I am done. walks away
 
Last edited:
An Aberrant Person said:
here is my MB it has 5x 3.0 16x slots and 1x 2.0 16x slot (PCH maybe?) My CPU does not support 80 lanes. It only supports 40 lanes. So I am forced to have to decide on what I want to run at 16x or 8x or 4x or 2x (not sure if 2x is an option but you get my point) So If I have all 5 filled. I have to choose what I want to run at what. So if I have something at 8x and its not maxing out. I can turn it to 4x and now I am using 4 less of my maximum 40 lanes, which means I can use that for something else.* So if these theotereical program existed. I would be able to see what is not being maxed and what is being maxed and than better optimize.

as I said...there is a weird niche use case. I would never need it but the question was "is there a use case for this" and yes there is.

https://www.newegg.com/p/N82E16813157536?Item=N82E16813157536

*how the fuck is this so complicated and I have wasted half a day explaining this to you. fuck
Click to expand...
Yeah - it won't work like that. I've got x99 boards. Setting x8 -> x4 doesn't free up 4 lanes for something else. That's the fundamental issue here - they're not a pool, they're literally wired in to certain things, and only certain individual items (generally under the optional list from the chipset) can be allocated around (or switching for consumer platforms x16/x0 to x8/x8. With a 40 lane chip on that board, you get a SET level of lanes to the slots, which you can adjust ~slightly~ (x16->x8/x8 to enable a disabled slot, for instance, or x16->x4/4/4/4 for NVMe bifurcation), but you can't just take those lanes and use them for something else entirely. And you especially can't take a m2 slot and change the allocation there (a couple of weird-as-hell Z690 boards aside). It'd be nice if it worked that way, but it just doesn't.

You can pick and choose SATA or NVMe or the like on the chipset lanes - those are "optional" and the mobo manufacturer can give you control or not as they choose - but the CPU will be locked to certain things permanently. Pull up the manual - it SHOULD list them, but I haven't used ASRock x99 boards, just MSI and ASUS.
 
An Aberrant Person said:
here is my MB it has 5x 3.0 16x slots and 1x 2.0 16x slot (PCH maybe?) My CPU does not support 80 lanes. It only supports 40 lanes. So I am forced to have to decide on what I want to run at 16x or 8x or 4x or 2x (not sure if 2x is an option but you get my point) So If I have all 5 filled. I have to choose what I want to run at what. So if I have something at 8x and its not maxing out. I can turn it to 4x and now I am using 4 less of my maximum 40 lanes, which means I can use that for something else.* So if these theotereical program existed. I would be able to see what is not being maxed and what is being maxed and than better optimize.

as I said...there is a weird niche use case. I would never need it but the question was "is there a use case for this" and yes there is.

https://www.newegg.com/p/N82E16813157536?Item=N82E16813157536

*how the fuck is this so complicated and I have wasted half a day explaining this to you. fuck

EDIT: the free up lanes dumb dumb is freeing up my 40 lanes limit. DO YOU UNDERSTAND NO NEED for PLX because this has nothing to do with plx o_O. I am done. walks away
Click to expand...

keep walkin, 'cause youre wrong
 
You must log in or register to reply here.
Back
Top