PFsense is killing me. SHould I go USG?

Discussion in 'Networking & Security' started by S-F, Dec 19, 2018.

  1. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    I've been running PFsense for about a year now and it's driving me crazy. Every now and then I loose internet connectivity even though PFsense says the connection is up. The only way I can resolve this is to reboot the router. Fk'n lame. I've had to reboot the system three (YES. 3) times today and I've only been home and awake for 2 hours. I'm sure someone with the know how could sort it but I don't have the time, expertise and certainly not the patience to slog through any more PFsense bullshit. Sucks because I spent something like $800 on the system.

    So is the USG Pro a good fit for me? My details:

    VPN. I'm using PIA and would like to continue to keep the entire house behind the VPN.

    100 meg or so connection. So nothing too crazy, and I don't plan on getting any faster speeds unless it's the same price. My issue is with upload speed not down.

    I'd like some ad blocking but this isn't a deal breaker. I could just set up a pi-hole.

    I'm already running Unifi APs so I already have the controller up and running 24/7. If I get the USG I just might also get the Unifi switch to replace my current one to tie everything together as a nerd nicety.

    I'm open to suggestions. I'm in particular open to the suggestion of taking the fucking PFsense machine out back for 12 GA target practice or over to the neighbor's yard to feed it to their pit bull.
     
  2. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,391
    Joined:
    Jun 13, 2003
    There are options other than pfSense- OPNSense for one, but there's also stuff like Untangle and Sophos UTM and XG firewalls.

    Also, the big reason for using a USG is that you want to use a full Unifi stack, and potentially use stuff like their implementation of Suricata IPS. If you don't need that, you might be better served by an Edgerouter X (for far less). Can still do VPN stuff.
     
    FNtastic likes this.
  3. infinity9

    infinity9 Limp Gawd

    Messages:
    202
    Joined:
    Aug 21, 2003
    I've used pfSense quite a bit and not had your experience. Are you sure its your router and not your modem? Its intermittent so its going to be hard to test. USG works fine but personally I think Ubiquiti's best days are behind them.
     
    Ehren8879 and FNtastic like this.
  4. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010

    Tons of people use PFsense with no issue. I'm not one of them.
    If it were the modem then why does rebooting the router fix the issue? Instantly. I'm not ruling out the modem. I just don't understand the logic of the issue being the modem.
     
  5. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010

    I'm not inclined to use an edgerouter. I'm already using Unifi stuff so I might as well go with it. That said, if it sucks or is expensive, as I already have the hardware for the POS PFsense machine, and I can run one of the other solutions you mention and they will just work I'm in. I don't NEED the Unifi integration. I just need to be able to read my emails when I get up in the morning without rebooting the router. And then I need to edit my calendar an hour later. Without rebooting the router.

    I wish I were one of you clever network folks. But I'm not.
     
  6. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,361
    Joined:
    Jul 26, 2007
    I am also using pfSense without issues. I actually had a similar problem but I traced it down to 2 things:
    • My WAP (ASUS RT-66U running DD-WRT in WAP mode) kept freezing (had it for 5 years) - replaced it with a Ubiquiti AC-Lite.
    • My temp directory kept getting full - I was running the Darkstat sniffer package, I thought the logs were circular but it turned out it wasn't. Turning off Darkstat fixed that.

    What box is pfSense installed on (CPU, RAM, NIC, etc.)? What packages are you running?
     
    FNtastic likes this.
  7. Farva

    Farva [H]ard as it Gets

    Messages:
    35,299
    Joined:
    Feb 3, 2004
    What are your parts for you pfsense box? Are you using Intel or Realtek NICs?
     
    FNtastic likes this.
  8. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    Thanks for the input folks. Just got out of the hot tub so this is my last post for the night.

    This is my hardware: https://smile.amazon.com/gp/product/B00G3ED7D4/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1

    I have something like 16 GB of RAM and a 250 GB or so Scamsung SSD.

    I have been using PFblocker to deal with advertisements. Turned it off. Things are still getting worse. Aside from that I'm basically only using it for the VPN and basic routing.

    It's not any of my APs. This is effecting my wired devices as well at the same time.

    When it happens it is network wide. I have full access to the LAN but the WAN goes out of commission. Could be that my switch is taking care of things when the router is down and that's why I still have LAN? I dunno. I just want things to work.
     
  9. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,391
    Joined:
    Jun 13, 2003
    I mention the ER-X specifically because it is cheap (US$50 on sale regularly) and it can do the two things you want: routing/border firewall and OpenVPN endpoint. Use the system you built for something else; I've picked up an appliance to toy with IPS and other types of filtering, using pfSense and others fairly successfully.

    But generally speaking I'd recommend fixing the hardware/software you have, if you can. It's a good setup!
     
    FNtastic likes this.
  10. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,391
    Joined:
    Jun 13, 2003
    Absolutely. Layer two should still work and is preferable, as putting the router in the mix limits speeds somewhat. But that's actually a lot of trouble to pull off with VLANs and the like.

    The basic idea is that the router 'routes', meaning that it routes between two different networks, your home network and the internet in this case. Anything on your home network will only send traffic to the router if it needs to access an internet resource, local resources will go between switchports (and through your AP). Unplug the modem, and you can still print to a network printer and so on.

    This is nice hardware. It's hard to want for anything more, even in an enterprise environment!

    I reiterate that it's worthwhile troubleshooting the issue as is :).
     
    FNtastic likes this.
  11. Farva

    Farva [H]ard as it Gets

    Messages:
    35,299
    Joined:
    Feb 3, 2004
    I have a similar system, but the 5019. What version of pfsense are you running?

    I know this isn't ideal, but have you thought about reinstalling pfsense? It sucks that you won't be able to diagnose it, but it should get you back up and running.
     
  12. FNtastic

    FNtastic [H]ard|Gawd

    Messages:
    1,269
    Joined:
    Jul 6, 2013
    It's likely not pfsense that's the issue. Unless you accidentally configured something incorrectly, or out of order. I've done it. It's also possible it's gotten into a weird state due to order. For example, some plugins won't change a configuration file upon "save", but only with an "update". Pfblockerng is a good example of this. And, that can sometimes be a factor in creating a weird state for your firewall.

    What is the behavior you're experiencing when the "internet goes down"? Can't reach websites? Can you ping 8.8.8.8? Ping 1.1.1.1? Ping Google.com? Check your nameservers. Are you using the ISP nameservers? Show us screenshots of your current config in pfsense. When it happens, do you still see a WAN IP in pfsense admin interface (web ui)? What's the lease time on that IP address? Has pfsense tried to renew it, and it isn't getting a response? Or, something else?

    Don't go buying anything without doing the above first. I suggest if the basic troubleshooting doesn't uncover something obvious, do a fresh install and see if you get the same behavior.
     
  13. Machupo

    Machupo Gravity Tester

    Messages:
    4,769
    Joined:
    Nov 14, 2004
    I guess I will echo what a lot of others are saying -- you are running some seriously good router hardware. Plenty to crush a 100meg connection.

    If your setup isn't that complex, you could always zeroize the PFSense install and start from scratch with a known-good install guide. The first time I messed with PFBlockerNG I used a guide to get a known-working setup in commission, took an image, and then starting messing with it.
     
  14. Farva

    Farva [H]ard as it Gets

    Messages:
    35,299
    Joined:
    Feb 3, 2004
    I've had a couple of weird issues where I've had to reinstall pfsense, not just wipe the config.
     
    FNtastic likes this.
  15. Biznatch

    Biznatch 2[H]4U

    Messages:
    2,151
    Joined:
    Nov 16, 2009
    What do the logs say? Pfsense has some decent logging, and it should at least point you in the right direction.

    A couple things to check next time you lose internet.
    1- Is dns working? Can you resolve a host name? ping a couple sites and see if it returns an IP
    2- Try pining an IP directly like the previous poster suggested?
    3- During the ping test, filter the logs for packets with that destination IP and see what it's doing
    4- Instead of rebooting, go to the Interfaces tab and release/renew the DHCP lease on the WAN. Does that fix it?
    5- Before rebooting PFsense, unplug the WAN from your modem and plug directly to a laptop. Do you have internet now?
    6- Try just rebooting the modem instead of pfsense and repeat step 5 if it failed
    7- Try clearing your states table instead of rebooting pfsense
     
    IdiotInCharge likes this.
  16. Orddie

    Orddie 2[H]4U

    Messages:
    2,356
    Joined:
    Dec 20, 2010
    I have had no issues with my PfSence server.

    I have a Gbit connection and get those speeds without issue.

    I run it in VMware
     
  17. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,391
    Joined:
    Jun 13, 2003
    Funny you mention that- I got around to putting ESXi on my appliance (J3160 with four Intel NICs), and loaded pfSense and Ubuntu Server on it. Think I'll play with setting up pfSense on two ports as a passthrough like I was doing with Untangle.
     
  18. Orddie

    Orddie 2[H]4U

    Messages:
    2,356
    Joined:
    Dec 20, 2010
    That’s how I do it. I pass through one physical NIC to the Van as an outside interface - directly connected to the cable
    Modem. The other interface is part of a vswitch which goes to 10Gbe interface to the rest of the network
     
  19. Aluminum

    Aluminum Gawd

    Messages:
    572
    Joined:
    Sep 18, 2015
    Been running pfsense or its replacement (IMO) opnsense on a Dell T20 or T30 (deals) on gigabit connections for a few years now. Always added a dual or quad intel NIC from fleabay (not putting the vpro port on the internet) and never had problems.

    Most are on VPNs, but not using PIA. Are the isp and vpn providers actually stable?

    There is nothing inherently unstable about the freebsd the *senses are built on, if anything its the other way around - with a known good configuration they are rock solid (updates can be scary at times though).
     
  20. NoOther

    NoOther [H]ardness Supreme

    Messages:
    6,356
    Joined:
    May 14, 2008
    First before doing USG, I would probably recommend the Edgerouter X as mentioned earlier. There are also other cheap options out there as mentioned. But if you are having issues with PFSense, I would think it is the deployment or configuration you are using that is not quite correct. Edgerouter X might be a more hassle-free path for you.

    The issue you are having seems to coincide with a few things I have experienced in the past with routers/firewalls:

    1) Memory issue. The device is not properly clearing the memory, especially virtual memory. This ends up causing the network stack to crash.

    2) Behavior Blocking. If you have your router setup to block based on certain behaviors, it could be mis-classifying certain traffic as bad and shutting it down.

    3) Unintentional DOS. Sometimes depending on how you have your router setup, it can DOS itself trying to re-establish connections, especially when trying to re-establish a VPN. Check the logs to see if there is an issue with the VPN preceding it shutting down the network.
     
  21. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    OK, OK. You've all convinced me to stay with pfsense for the time being. I never thought to look at the logs. Foolish of me I guess but every time I've looked at them I got immediately overwhelmed. I actually tried sending the logs to the syslog server in my home automation software so I could trigger things based on the router logs but it quickly got way too crazy and it was all Greek to me.

    Thanks for all the input folks. I'll report back when it goes cattywompus again. It might not since I have disabled pfblockerng. Too bad as the ad blocking is pretty important to me.
     
  22. thrash408

    thrash408 Limp Gawd

    Messages:
    322
    Joined:
    Jan 22, 2010
    I love my usg for home use, I'm full unifi and have never had issues.
     
  23. infinity9

    infinity9 Limp Gawd

    Messages:
    202
    Joined:
    Aug 21, 2003
    Cable modems, especially ones you cant buy, are guilty of using non-standard DHCP.
     
  24. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    OK, it happened again. I could ping 1.1.1.1 which I have pfsense set to use. I couldn't ping www.google.com. The dashboard said that the WAN was online. Resource usage was effectively nothing. Something like 2% CPU, 20% memory (using pfblockerng) 0% swap. 1% disk usage. The log for the firewall was FULL of block messages from about every other second.

    Example:

    Dec 23 16:23:55 WAN [fe80::2cc:fcff:fe61:822] [ff02::1] ICMPv6

    Almost all of the entries are from fe80::2cc:fcff:fe61:822.

    A reboot made things right.
    Immediately when I check the logs after a reboot I see tons of the same messages. Is this suspect?
     
  25. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,361
    Joined:
    Jul 26, 2007
    Did you trying pinging that IP address to see if it is internal or external?

    I'm wondering if that's the IP of your cable modem and your ISP is doing something.
     
  26. FNtastic

    FNtastic [H]ard|Gawd

    Messages:
    1,269
    Joined:
    Jul 6, 2013
    That's an IPV6 link-local address (private IP). It's not publicly routable. If you aren't using IPV6, turn it off on your PCs

    http://packetlife.net/blog/2011/apr/28/ipv6-link-local-addresses/

    It's likely your firewall rule for IPV6 doesn't exist, hence blocked by default, which is why you're seeing a bunch of them

    What happens is that your PC prefers IPV6 over IPv4 and it can cause issues like what you're experiencing. For some reason, it tries to resolve domains using IPV6 when IPv4 exists and is working perfectly fine..
     
    Machupo likes this.
  27. rekd0514

    rekd0514 Gawd

    Messages:
    730
    Joined:
    Nov 24, 2007
    Just putting my 2 cents in. Pfsense is very stable. I could go years without rebooting if not for updates.

    I have IPv6 turned off or set to none on WAN and LAN. No support from my ISP yet anyway.
     
  28. EniGmA1987

    EniGmA1987 Limp Gawd

    Messages:
    152
    Joined:
    May 2, 2017
    1.1.1.1 is Cloudflare DNS. It is pretty popular and one of the best ones out there.




    @OP - turn off ipv6 and see if your connection stops having issues. Likely you are not configured right to use ipv6 properly so it is causing issues.
    If you can ping 1.1.1.1 but not www.google.com then it is definitely an issue with the DNS not resolving. But WHY the DNS is not resolving is another matter. It very well could be that the DNS is responding sometimes on ipv4 and then it decides to respond over ipv6 when somethings triggers an ipv6 connection and from then on gives you an issue.
     
  29. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010

    All ipv6 stuff is turned off on the router. Has been for about a year. I forget why. It was causing some issues that escape me at the moment.
     
  30. NoOther

    NoOther [H]ardness Supreme

    Messages:
    6,356
    Joined:
    May 14, 2008
    What about IPv6 on your devices?
     
  31. acascianelli

    acascianelli [H]ardness Supreme

    Messages:
    6,773
    Joined:
    Feb 25, 2004
    Mentioned a few times above already, but I would start with re-installing pfSense and starting a new config from scratch.

    I just tracked down a problem with pfSense I was experiencing. I was having a problem where my firewall was booting up faster than my cable modem and wouldn't retry obtaining an address from the modem. Somewhere along the lines it looks like pfSense introduced changes into the interface definitions or the DHCP client and my 'legacy' config left all of the DHCP options blank. I reset it back to the pfSense defaults and seems to have corrected my problem.

    Second option, also mentioned above, would be to try an alternative like Opnsense.

    I have coworkers trying to convince me to leave pfSense and run OpenBSD.

    Just thinking of options to keep using the hardware you've already invested in.
     
  32. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    Honestly turning ipv6 off on all devices on the network isn't really an option. Imagine me giving access to the Wi-Fry network to guests and first having to modify their phones? I don't want to sound like a fool but my previous DD-WRT router didn't have such hiccups which is why I'm thinking that pfsense is above my pay grade. I simply don't know enough to manage it, even with the help of talented folks such as yourselves who have chimed in here. I am EXTREMELY grateful for every word people have contributed to this topic for the record.

    I am hesitant to reinstall pfsense. Out the gate it didn't work for me. I had to have the assistance of a pseudo friend to simply get it running. That was when ipv6 was turned off. I'm worried that a clean install will reintroduce all of those problems I was previously experiencing.

    Is Opensense a little more network retard friendly?

    For the record I'm not wedded to using this hardware for a firewall. I have been toying with the idea of turning my current file server into a ZFS machine to be a data store for a Windows server running on the SM hardware that's currently the router. I really need my file server to run Windows, else I'd keep everything on one machine.
     
  33. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010

    I had this post of yours in mind when the router flaked out the other day but....... Internet access was, well, I didn't have any, so I couldn't check these items. I have copied the list to a word file and will consult it the next time this happens. Thanks!
     
  34. acascianelli

    acascianelli [H]ardness Supreme

    Messages:
    6,773
    Joined:
    Feb 25, 2004
    I’ve disabled IPv6 on my home network. Comcast supports it on the edge, but I have no need for it inside my LAN. I don’t think you’re doing yourself any favors running dual stack IPv4/IPv6.
     
    Farva and FNtastic like this.
  35. Machupo

    Machupo Gravity Tester

    Messages:
    4,769
    Joined:
    Nov 14, 2004
    Not to derail, but this got me curious: what service are you looking for that requires your file server to run windows?
     
  36. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010
    My home automation software and video surveillance software need Windows. Unfortunately.
     
  37. FNtastic

    FNtastic [H]ard|Gawd

    Messages:
    1,269
    Joined:
    Jul 6, 2013
    Quoted for "wi-fry". For some reason, that was hilarious to me
     
  38. EniGmA1987

    EniGmA1987 Limp Gawd

    Messages:
    152
    Joined:
    May 2, 2017
    There is no need to have ipv6 in any way on your home network and every single device will work perfectly fine. Nothing on the planet is an ipv6 only device right now, and many are ipv4 only.



    To be honest, if you didnt have pfsense working out of the box then that means you dont know how your internet service is even set, which means you really dont know enough about your network that you should be running pfsense. So I vote that you should move to something else. EdgeRouter is also off the table as it requires the same basic knowledge that pfsense does for initial setup.
     
  39. S-F

    S-F Gawd

    Messages:
    646
    Joined:
    Aug 5, 2010

    Yeah, that's what I'm suspecting. This whole pfsensse endeavor has been more involved than I had anticipated, unfortunately. I'm going to keep plugging away with it and if these issues continue to stump me I think I'll just get a USG. If it's anything like the rest of their products it should be pretty point and shoot. Would the USG Pro be able to handle my VPN usage and does it have some ad blocking features? Google is giving me spotty answers to these questions.
     
  40. acascianelli

    acascianelli [H]ardness Supreme

    Messages:
    6,773
    Joined:
    Feb 25, 2004
    VPN on pfSense can be a bit difficult for someone who isn't familiar with all the nitty gritty details. Somehow through the grace of Krom I was able to get IPSEC working with Apple iOS clients, but now I'm working on switching over to OpenVPN.

    Getting pfSense up and running just acting as a basic firewall and gateway should take less than 5 minutes out of the box.