pfSense 2.5.0 upgrade results?

GotNoRice

[H]F Junkie
Joined
Jul 11, 2001
Messages
10,200
I've been running 2.4.5 for a while. I see that starting a few days ago the 2.5.0 upgrade finally became available. This is pretty significant as it includes a newer version of FreeBSD, etc. I'm just curious if others have upgraded already and what their experiences are. I'm torn between wanting to upgrade and not wanting to fix something that isn't broken.

If you did upgrade already, was it a fairly straightforward process? Were your settings retained automatically or did it set everything to defaults? Any regrets?
 

acascianelli

Supreme [H]ardness
Joined
Feb 25, 2004
Messages
6,903
Mine upgraded without any problems. Running on a PCEngines APU2C4.

With the recently announced pfSense CE/Plus plans, I'm putting serious thought into switching to Opnsense though. :-/
 

ComputerBox34

[H]F Junkie
Joined
Nov 12, 2003
Messages
12,560
Upgraded 3 boxes. No issues with config or packages.

There are some issues floating around though. Check /r/PFSENSE
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
20,076
Just upgraded, after first reboot, DHCP leases would not load, but after a 2nd reboot worked fine and all well so far.
 

mvmiller12

[H]ard|Gawd
Joined
Aug 7, 2011
Messages
1,081
The Setup:

Running pfSense on a Dell PowerEdge R620 with the built-in 4-port 1G Broadcom NIC and 3x 2-port 10G Intel NICs added in. The pfSense Firewall has all of that ethernet bridged together to act as a cheap switch since 10G switches were (and still mostly are) Stoooopid Expensive* as well as doing it's duty as the firewall/router for my FiOS gigabit Internet service. Got the Intel NICs because they were super cheap used on eBay and I regularly move large video files from my desktop PC to my file server over the network. Fileserver is on a separate Dell PowerEdge R515 and both that and my desktop also each have one of the 10G Intel dual-port NICs connected to the pfSense machine. There is also a backup Fileserver with a 10G dual port NIC on a Dell PowerEdge R510.

Direct 10G NIC to 10G NIC performance over a separate subnet using SMB gave me about 5 Gigabits/sec when copying large files back and forth. Speed was very good, considering I was going between an NVMe SSD and an 8-drive rust RAID6, but it was absolutely not scalable to include the backup Fileserver or potentially other machines. When I decided to make the pfSense box and also set it up as a 10G switch, LAN performance for the 10G NICs topped out @ ~2 Gigabits/sec running my regular firewall rules on pfSense 2.4.0 and 2.4.5. I could get ~4 Gigabit/sec performance with no rules running on the firewall beyond the bare minimum required to get it to route Internet to my network, but... naw. 2 Gigabits is definitely not stellar performance, but still twice as good as my 1G NICs were giving me. Did I miss 5 Gigabits of performace when copying files? Absolutely, but it was good enough that I was satisfied with it and never gave it any more thought.

Upgrade Results:

The upgrade from 2.4.5 to 2.5.0 went smoothly and all of my rules are still in place and operational (I am defining "smoothly" as: I started the upgrade and 30 minutes later, the system was fully operational with no other interaction required from me). 10G NIC performance, however, has greatly improved. I am now getting the full 5 Gigabits of performance when performing file transfers that used to require a dedicated subnet direct-to-the-fileserver to obtain. On top of that, I really feel like there is more speed there that is not being utilized because my drive speeds are the bottleneck. This is a truly impressive improvement that came "out of the blue" and I am now getting free performance on my network that just wasn't there before.

*Stoooopid Expensive clarification:

MVMiller12, why are you such a cheap bastard? Why didn't you just buy a 10G RJ-45 Switch instead of dicking around with pfSense?

Well, it's like this... The PowerEdge R515 was gifted to me with no drives. It was effectively e-waste that a local business was going to chuck. The IT guy there removed the drives and RAM from it and gave it to me under the proviso I find a use for it. I spent $10 TOTAL to upgrade BOTH CPUs in it to the latest Opterons it would take (courtesy of an e-bay seller), and another $40 SHIPPED for 32G of RAM for it. I got a 256G SSD for the then-decent price of $40 to hold the OS. The Dell R620 was a Craigslist find for ~$150 sans drives, but it had RAM. Another $40 for an SSD for the OS drive. My Cousin owns the Dell R510 (also Craigslist) that he paid ~$200, also with RAM but sans drives. We bought a lot of Dell-branded used 10G dual port NICs from eBay for ~$40 each and split them up amongst the boxes that needed them.

My cousin and I each used Tax Return money to populate the R515 and R510 each with 8x8TB drives from Best Buy - (we shucked Western Digital externals that there on sale for $150 apiece), and this cost us each ~$1300 after taxes. This was the BIG expense on our setup. My cousin wanted to dabble in a small multi-server setup with VMs, and backing up my fileserver was just a good excuse for him to jump in. My R515 and R620 are dedicated boxes - his R510 is a VM host machine, one VM of which acts as the backup fileserver to my R515.

So * MY * cost breakdown is as follows:

R515: Free
R515 CPU and RAM Upgrades: $50
R515 SSD: $40
2x 10G dual port NICs: ~$80 (1 for fileserver, 1 for desktop PC)
8x8TB Drives: ~$1300 (RAID 6 using hardware RAID PERC controller)
------------------------------------------------
My TOTAL Spent: $1470 for my Fileserver duties

R620 for pfSense duty: ~$200 (replacing $250 Asus router)
R620 SSD: $40
3x 10G dual port NICs: ~$120
------------------------------------------------
My TOTAL Spent: ~$360 for my routing/firewall duties


Cost of a 5-port 10G RJ-45 switch at the time I put this together: ~$1100

FUCK.
THAT.

Edit: My cousin, being single with a good job in IT at a government contractor has a LOT more disposable income than I do. He has since purchased another used rackmount server, a rackmount UPS, a rackmount 12-bay drive expansion bay (prepopulated with 12x 2TB drives), and an actual small server rack to hold all of this stuff)
 
Last edited:

longblock454

[H]ard|Gawd
Joined
Nov 28, 2004
Messages
2,034
Mine didn't come back up after reboot, but after a manual hard reboot it came back up fine with all settings retained. I am running the Dev package of pfblockerNG with lots of lists ect, so maybe that was it, I didn't debug just bounced it and it worked.

I've been running pfsense at multiple sites for over a decade and aside from one upgrade issue (some serial port snafu, impacted quite a few users) it's been rock solid.
 

GotNoRice

[H]F Junkie
Joined
Jul 11, 2001
Messages
10,200
Cool. I did the upgrade, took about 5 minutes, went smooth. But I also don't use many advanced features. About the only advanced feature I use is the traffic shaper. I do have a TON of static IPs setup on the DHCP server and numerous mapped ports though, which is what would have been a pain if I had to start from scratch.

Direct 10G NIC to 10G NIC performance over a separate subnet using SMB gave me about 5 Gigabits/sec when copying large files back and forth. Speed was very good, considering I was going between an NVMe SSD and an 8-drive rust RAID6, but it was absolutely not scalable to include the backup Fileserver or potentially other machines. When I decided to make the pfSense box and also set it up as a 10G switch, LAN performance for the 10G NICs topped out @ ~2 Gigabits/sec running my regular firewall rules on pfSense 2.4.0 and 2.4.5. I could get ~4 Gigabit/sec performance with no rules running on the firewall beyond the bare minimum required to get it to route Internet to my network, but... naw. 2 Gigabits is definitely not stellar performance, but still twice as good as my 1G NICs were giving me. Did I miss 5 Gigabits of performace when copying files? Absolutely, but it was good enough that I was satisfied with it and never gave it any more thought.

Upgrade Results:

The upgrade from 2.4.5 to 2.5.0 went smoothly and all of my rules are still in place and operational (I am defining "smoothly" as: I started the upgrade and 30 minutes later, the system was fully operational with no other interaction required from me). 10G NIC performance, however, has greatly improved. I am now getting the full 5 Gigabits of performance when performing file transfers that used to require a dedicated subnet direct-to-the-fileserver to obtain. On top of that, I really feel like there is more speed there that is not being utilized because my drive speeds are the bottleneck. This is a truly impressive improvement that came "out of the blue" and I am now getting free performance on my network that just wasn't there before.

Food for thought: SMB multichannel works very well. 4 Gigabit ports in my main desktop, 4 Gigabit ports in my fileserver, all connected to the same cheap gigabit switch, and I can get very close to 4Gbps transfers. Quad-port gigabit adapters can be bought used for ~$20, and 24-port gigabit switches can be found used for like $50. It works so well, I lost all motivation to deal with expensive 10Gb Ethernet on each desktop (but does still come in handy when connecting separate switches together via 10Gb backbone.)
 

Valnar

2[H]4U
Joined
Apr 3, 2001
Messages
3,352
I did not do the upgrade yet, but wondering if it would be cleaner to install from scratch, add my packages, then restore a backup. It's in my house so I don't have to worry about remote connectivity.

I admit I'm wearing a Windows hat while thinking of this. I did not do an in-place upgrade from Windows 7 to 10 either. I don't like the garbage. I just don't know if FreeBSD and pfSense work the same way. I guess my biggest question is - will anything be lost by starting fresh + restore? Pros and cons? I'm most worried about pfBlockerNG since it's the only elaborate config outside of the base product.
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,815
I did not do the upgrade yet, but wondering if it would be cleaner to install from scratch, add my packages, then restore a backup. It's in my house so I don't have to worry about remote connectivity.

I admit I'm wearing a Windows hat while thinking of this. I did not do an in-place upgrade from Windows 7 to 10 either. I don't like the garbage. I just don't know if FreeBSD and pfSense work the same way. I guess my biggest question is - will anything be lost by starting fresh + restore? Pros and cons? I'm most worried about pfBlockerNG since it's the only elaborate config outside of the base product.

The older releases were much more appliance like and that the upgrade basically just reflashed / loaded the entire OS as a blob. So more like loading firmware on a router or a phone where it was just one big file that was copied over. The newer versions are simply using the package manager to update packages in the back end. So the 2.4.5 -> 2.5.0 is like a Windows update, not a whole new operating system. That said they are good at making it convoluted as possible and in some cases the kernel update is actually jumping BSD releases but generally speaking there is no real reason to be concerned. Updates are well tested and generally fine for like 99% of the install base so there's no reason to do a reinstall for this update.

You should download a backup of the config before doing the upgrade however just in case something does go wrong. Things to note that config file has hashes for all user's passwords who can sign into the system, so it can be a good idea to password protect it and / or store it in a safe location.
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,815
Mine upgraded without any problems. Running on a PCEngines APU2C4.

With the recently announced pfSense CE/Plus plans, I'm putting serious thought into switching to Opnsense though. :-/

Glad you mentioned this as I hadn't read about it. I would say the writing is on the wall at this point they want to lock down the installs to try to steer you towards their appliance / buy a subscription from them. A few versions back when I saw a unique serial number pop into the dashobard for each install it seemed like that was foreshadowing what was coming. Even if Plus is free, I wouldn't be surprised if they will start requiring you to sign up via email for them to send you your "free" code. Seems like Opnsense is going to be a better bet if you don't want to have to register every install you do in the future. (They haven't come out and said any of that, but there's not much of a reason to uniquely identify every install if you weren't planning on doing something with that information)
 
D

Deleted member 214115

Guest
Man, it runs much better with the recent drivers and updates. As in over-all-system drivers; meaning driver updates such as, SMB, PCI, HBA, etc. I posted some changes for the igb/iflib driver used. My pfSense uses an AMD A4-5000 and a i350-T2v2. If anyone is using similar hardware I would advise to update your configurations. NOTE! Any System Tunables you set, you should not apply the setting afterwards! It is better to shutdown and then boot the server for it may hang and be unresponsive to administration afterwards. You can expose some of the settings because most of the igb settings are not there anymore, such as AIM (Adaptive Interrupt Moderation), descriptors, and queues. You can see the changes exposed by sysctl hw. and sysctl dev. in the command line. Do not forget to check your dmesg in command line also, for any changes and proper handling of devices and settings. A little bonus, if you want to rid the mounting wait (most likely unnecessary with most/all builds) and annoying logging in dmesg delete this line in loader.conf: kern.cam.boot_delay=10000

https://hardforum.com/threads/pfsense-2-5-new-configurations-for-igb-performance.2008210/
 
Last edited by a moderator:

munkle

[H]F Junkie
Joined
Jan 16, 2005
Messages
11,800
I upgraded with no issues, but I noticed I'm not getting my full internet speed anymore. I have 600mbps and best I can get wired is 400mbps. I'm using some pretty old intel nics with an intel i5 6500 cpu and 8gb of memory. Might be an isp issue, I'll have to run more speed tests at random times to see if its consistently lower now.
 
D

Deleted member 214115

Guest
I upgraded with no issues, but I noticed I'm not getting my full internet speed anymore. I have 600mbps and best I can get wired is 400mbps. I'm using some pretty old intel nics with an intel i5 6500 cpu and 8gb of memory. Might be an isp issue, I'll have to run more speed tests at random times to see if its consistently lower now.
Which NIC's? Also, do you have any of the offloading enabled and are using a VLAN? I can basically give you a recipe to use, just need to know a couple of things.
 
D

Deleted member 214115

Guest
I will just place the recipe here. According to netmap, these are the drivers:
On FreeBSD: cxgbe(4), em(4), iflib(4) (providing igb, em and lem),
ixgbe(4), ixl(4), re(4), vtnet(4).

You will need to make sure that all NIC offloading functions are disable INCLUDING!!! VLAN encapsulation . These are not compatible with netmap, and netmap IS the packet operator. As noted in netmap manpage:

netmap does not use features such as checksum offloading, TCP
segmentation
offloading, encryption, VLAN encapsulation/decapsulation,
etc. When using netmap to exchange packets with the host stack, make
sure to disable these features.


Some parts have change such as EEE, but also flow control is not in loader.conf. It needs to be a tunable application. Any tunable made should NOT be applied immediately. You save, shutdown (trust me), and then a reboot.

My loader.conf:
legal.intel_iwi.license_ack=1
legal.intel_ipw.license_ack=1
dev.igb.0.iflib.override_nrxds=4096
dev.igb.1.iflib.override_nrxds=4096
dev.igb.0.iflib.override_ntxds=4096
dev.igb.1.iflib.override_ntxds=4096
net.link.ifqmaxlen=8192
net.isr.defaultqlimit=4096
net.inet.tcp.soreceive_stream=1
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.pf.source_nodes_hashsize=1048576
net.inet.tcp.hostcache.cachelimit=0

Note, if you want to increase buffer queue size the loader configuration is X=interface number, such as 0 WAN, 1 LAN, etc:
dev.igb.x.iflib.override_nrxds
dev.igb.x.iflib.override_ntxds


I removed this line to decrease boot delay, and the unnecessary probing to USB ports; it also added dmesg spam that I didn't care for: kern.cam.boot_delay=10000

My Tunables:
dev.igb.0.eee_control=0
dev.igb.1.eee_control=0
dev.igb.0.fc=0
dev.igb.1.fc=0
dev.igb.0.tx_int_delay=0
dev.igb.1.tx_int_delay=0
dev.netmap.buf_num=655360
net.bpf.zerocopy_enable=1
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
net.route.netisr_maxqlen=4096
net.inet.ip.intr_queue_maxlen=4096
net.inet.ip.stealth=1
net.inet6.ip6.stealth=1
net.inet.sctp.blackhole=2

hw.intr_storm_threshold=0
 
Last edited by a moderator:

GotNoRice

[H]F Junkie
Joined
Jul 11, 2001
Messages
10,200
One notable issue that I've been having since upgrading to 2.5.0 is that the Dashboard doesn't seem to want to update automatically anymore. The Traffic Graph still works and updates, but everything else such as the Uptime, Temperature, CPU Usage, Load average, Memory usage, etc, etc no longer update unless I manually refresh the page.
 
D

Deleted member 214115

Guest
Use another browser to see if same symptom(s) persist. If it doesn't then try deleting the local cache of the browser that has the problem-if you continue wish to use it for pfSense administration. You can even try removing the widget that are affected, save, close, load, log-in, and then add the widget.
 

Master_shake_

Fully [H]
Joined
Apr 9, 2012
Messages
17,776
Mine did this cool trick where it tried to fix the file system and created another partition on the drive to boot from.....14 times.

By the end there was 14 different boot partitions on the same drive.
 
D

Deleted member 214115

Guest
At least recovery can be very quick if planned properly prior to the upgrade. Download the recent version, or update you installer before any update. For instance, I have a pouch full of flash drives and OS's. Do not upgrade packages, backup, and install the update. If it goes south. Plug-it-in, reinstall, and restore. Shouldn't take too long.
 

Outlaw85

[H]ard|Gawd
Joined
Feb 7, 2012
Messages
1,173
A little late to the party but..

I've since gone back to 2.4.5p1 for now.
Quick overview (nothing crazy):
-virtual pfsense in ESX
-5 LANs (primarily 2 used)
-5 DHCPs
-Port forwarding
-NAT configs
-few monitoring packages and vmware open tools
-AES-NI on

The actual upgrade was as easy as one would expect. And no noticeable issues.


The issues I am having appears to be related to my ISP (Spectrum) and I'll find out for sure (fingers crossed) tomorrow when the tech takes a look. I reinstalled 2.4.5p1 in my troubleshooting escapade but still seeing tons of drops.
 
Top