pfSense 2.5 New Configurations for igb (Performance)

  • Thread starter Deleted member 214115
  • Start date
D

Deleted member 214115

Guest
I wanted to share some of my findings about pfSense 2.5. The recent changes have made many of the configurations obsolete, but I have also have knowledge of some of the settings in which some users have configured that will cause lesser throughput. This will mainly be for the i350 series and its logics. To note, much of the hw.igb is not relevant anymore and most of the functions are through dev.igb and dev.igb.x.iflib.


My little router is almost an APU model provided by PC Engines. My processor is actually an AMD A4-5000. This is paired with an Intel 350-T2v2 (verified). With each upgrade of FreeBSD, I have gained a bit more and more performance. FreeBSD 13, if the Community Edition gets the upgrade to it in proper time, then 13 will be phenomenal in performance (FreeBSD 13 BETA Benchmarks - Performance Is Much Better - Phoronix). So, never upgraded as of now and this system is long in tooth but excellent performance per watt.



My added configurations that are not placed by default from installation in “loader.conf” are these:

legal.intel_iwi.license_ack=1

legal.intel_ipw.license_ack=1

net.link.ifqmaxlen=2048

net.isr.defaultqlimit=2048

net.inet.tcp.soreceive_stream=1

net.inet.tcp.syncache.hashsize=1024

net.inet.tcp.syncache.bucketlimit=100

net.pf.source_nodes_hashsize=1048576



My “System Tunables” are these:

dev.igb.0.eee_control=0

dev.igb.1.eee_control=0

dev.igb.0.fc=0 [Must be set in Tunablesnot Laoder.conf]

dev.igb.1.fc=0 [Must be set in Tunables,not Loader.conf]

net.inet.ip.redirect=0

net.inet6.ip6.redirect=0

net.route.netisr_maxqlen=2048

net.inet.ip.intr_queue_maxlen=2048

net.inet.udp.maxdgram=16384

net.inet.tcp.syncookies=0

hw.intr_storm_threshold=0

kern.ipc.soacceptqueue=2048



Note, dev.igb.x.tx_int_delay=0 (RX is disabled by default on WAN and LAN interfaces)

[This will change the Enabled/Disable the delay to INTR. When queues and MIS-X are used moderation takes place and supersedes the delay set by default. Adaptive Interrupt Moderation does not have exposure for configuration anymore. When enabled the delay is utilized by default setting or configured setting. Default: dev.igb.x.tx_abs_int_delay: 66 / dev.igb.x.rx_abs_int_delay: 66


The now non-functional configurations for the NIC are:

dev.igb.x.eee_disabled (New setting above)

hw.igb.enable_aim

hw.igb.num_queues

hw.igb.rxd

hw.igb.txd



Some, notes to read:

You want EEE disabled due to fact that it may lower power some and temp a little, a router needs to be constantly on and microseconds of disabling the interface is asinine. On a desktop system it can be a maybe, but I would not advise it either. The LAN interface(s) will most likely be the only negotiated side due to recent switches, and most likely never WAN. However, it should be disabled on all interfaces especially LAN’s to prevents stalls. These will output with: dev.igb.x.iflib.txqx.r_stalls=x

AIM, was Adaptive Interrupt Moderation. Not exposed anymore

Queues are not exposed through hw.igb anymore, and you actually wanted it on the default of 0 which detected each core to bind. These cores were bound by RSS to each core for RX, but also each core for TX. Note, this is the same for Windows. RSS would keep the core’s cache warm for the packet to traverse from the core bound to RXcore/TXcore. Dividing the cores, such like num_queue=2 limited system throughput and adaptive interrupts on a 4 core system. The modern RSS driver handled better once implemented than the user once it became modern in 10’ish and 11’ish version of FreeBSD.

You actually wanted to keep descriptors at default. The amount was for each queue and core. Having 4096 queues actually lowered performance. I never tested with newer version, YMMV. Reference from: FreeBSD forwarding Performance [BSD Router Project] (bsdrp.net), see attachments below.



IFLIB Manpage: https://www.freebsd.org/cgi/man.cgi...=FreeBSD+12.2-stable&arch=default&format=html
 

Attachments

  • igbdescript1.png
    igbdescript1.png
    31.2 KB · Views: 0
  • igbdescript2.png
    igbdescript2.png
    27.6 KB · Views: 0
Last edited by a moderator:
Top