How to make NICs/Network use more than 1 cpu core?

W

WestSidaz

Limp Gawd
Joined
Jan 13, 2011
Messages
223
Hello!

I have gotten myself into sitsuation where networking is only using 1 cpu core.. for some reason..
I have dual xeon cpu system and i had this issue with 1gbit network and now i have same issue aswell in 10 gbit network in windows 10 pro

and what makes issue even bigger is that windows uses same core for some reason.. when i abuse nics, windows slows down..

What could i do to make it work better and with more cores? i have plenty of cores to be abused..

What i have already done?

Changed some nic settings like rss to max and jumbo packs to 9k.. also added all cores for windows in msconfig boot file and tryed to give better priority for exploder.exe in windows..


If any1 can point me to right direction, i would appriciate that alot (^^,)
 
Last edited:
Not going to lie, utterly confused by this post. Depending on how good your NIC is most of the work will be offloaded to the NIC itself... Also, your PC slow down when you "abuse" the NIC is actually probably an HDD/SSD Bottleneck.
 
No, I also experienced this when I installed 10Gb network. My server was bottlenecking due to its old/slow CPU (3-core Athlon) at the time. Only "one" core load and it wasn't able to max out the theoretical throughput of the NICs. I upgraded since then and it's Ok.
Not everything could be offloaded I believe and it depends also on the card. Everything in Advanced settings of the cards is set to "hardware offload" or the likes wherever possible.
I believe it's up to the driver and eventually some features of the card, driver and BIOS. One of them is SR-IOV, I haven't tried to enable that (it's disabled both in my BIOS and driver advanced settings) as my CPUs now are not bottlenecking but I think it's more about virtualization (currently I cannot achieve more than 2.1Gbps in a virtual machine but I don't need more anyway).
 
Sounds to me that the application/service that is conducting the network transfer simply isn't multi-threaded. Nothing the NIC nor its driver can do about that.
 
BlueLineSwinger said:
Sounds to me that the application/service that is conducting the network transfer simply isn't multi-threaded. Nothing the NIC nor its driver can do about that.
Click to expand...

Application don't touch the network stack, that's handled at the OS level. I can't believe even an old CPU would be unable to handle 10Gb/s, even with a single core. Is that core spiking to 100% during network load? I've never seen a NIC cpu limited, but ram becomes an issue with very large file transfers.
 
This is most likely a limitation of the PCIe lane being on one CPU core isn't it?
 
Biznatch, for 1Gbps it's no issue and CPU load (even spikes) is negligible. It's almost analogous to torrent programs - for 100Mbps internet every one available will do fine. Once you go above up to 1Gbps internet, most torrent programs fail above 30-40MB/s even with modern CPU.
Once you step 10x times up, to 10Gbps, you instantly start seeing and witnessing things you never suspected you could. Yes, one thread maxed out (3-core CPU at 33%, or 4-core at 25%). Using iperf3, and not some HDD/SSD transfer.
The CPU has to feed the network stack with data anyway (or to supply/move buffers in a direct access), the NIC driver has its own cycles.
 
Receive Side Scaling is how you make the NIC use more CPU cores. Many consumer level chips just have enable/disable. Higher end enterprise level chips can specify number of cores, as well as whether it prefers specific socket or not. receive and transmit buffers are how much memory is used for buffers in transfer. Higher buffers tend to make larger transfers have better bandwidth, but at the cost of latency.
 
Biznatch said:
Application don't touch the network stack, that's handled at the OS level. I can't believe even an old CPU would be unable to handle 10Gb/s, even with a single core. Is that core spiking to 100% during network load? I've never seen a NIC cpu limited, but ram becomes an issue with very large file transfers.
Click to expand...


Of course not. But applications do control the data that is being sent over the network. And if they aren't written to take advantage of multiple threads, then you might see what the OP is.

A common example of this is Samba (smbd/nmbd). While it can spit each individual connection/transfer out to its own process, each of those is only single-threaded and therefore stuck to a single core. If that CPU is older/slower, and/or is concurrently handling multiple other processes that the transfer has to contend with, then it's quite likely the system will not be able to saturate the network link.
 
BlueLineSwinger said:
Of course not. But applications do control the data that is being sent over the network. And if they aren't written to take advantage of multiple threads, then you might see what the OP is.

A common example of this is Samba (smbd/nmbd). While it can spit each individual connection/transfer out to its own process, each of those is only single-threaded and therefore stuck to a single core. If that CPU is older/slower, and/or is concurrently handling multiple other processes that the transfer has to contend with, then it's quite likely the system will not be able to saturate the network link.
Click to expand...


So the application chokes generating packets. That seems like an issue with the protocol/app itself, and has nothing to do with the NIC or network stack.


I guess the OP should clarify the exact operation/application that is choking, but if it's similar to your example there's nothing to be done but upgrade CPU speed.
 
Yes, RSS was the one I forgot about before. Interestingly this has always been On for me but it didn't seem to have an effect.
My speed experiments always included iperf3 between two Win2016 Server. iperf3 was the only method I could see full 9.6Gbps so I guess it's the most trustworthy.
 
You must log in or register to reply here.
Back
Top