Post Your pfSense Setup/pfSense Sugestions

Discussion in 'Networking & Security' started by Killerxp100, Nov 15, 2018.

  1. Killerxp100

    Killerxp100 Gawd

    Messages:
    748
    Joined:
    May 20, 2011
    I've been running pfSense for years now, the last few I've been running it on a Dell T20 that I had collecting dust. It's been super reliable, went almost a year without any down time. I would like to repurpose the Dell as a blueiris server, so I'm looking for ideas on what to replace it with. Something small, low power, but still powerful enough to run my 1000/100 connection. I do run openvpn and would like close to equivalent speeds that my current setup provides. I also run squid, snort, and a few other packages.

    Originally I was going to go with pc engines apu2c4 but after some research I don't think it can keep up with my connection, and definitely not my vpn. Now I'm looking at some of the minisys & qotom devices that seem popular and quite capable. Namely the i3 & i5 versions.

    Any insight is greatly appreciated!
     
  2. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,109
    Joined:
    Jun 13, 2003
    This i5 Qotom is more or less what I've been looking at. I have a J3160-based version now, which is about twice as fast as the AMD mess in the APU-series boxes, and I wonder if it would be able to handle gigabit downloads myself.

    There is also ProtectLI that sell rebadged Qotoms (etc.) and have i5-based six-port version for the top-end, but they seem to be really proud of them.
     
  3. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,109
    Joined:
    Jun 13, 2003
    Ah, here's a link to a similar Qotom listing to that ProtectLI i5. Looks like a significant markup if you don't need the RJ-45 console port (which I find useful).

    Also note that anything 6000-series and newer is going to need DDR4, while the 4000-series Core builds are DDR3.
     
  4. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,325
    Joined:
    Jul 26, 2007
    Do you happen to have an ESXi server or a Hyper-V box available?

    pfSense works quite well for me as a VM in ESXi 6.5, especially as I use Intel NICs that have paravirtualized drivers - they run pretty close to bare-metal speeds so no need to do passthrough.

    That way you don't have to buy another box, at most you have to buy a NIC.
     
    bbenz33 likes this.
  5. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,109
    Joined:
    Jun 13, 2003
    I thought about this- prototyped pfSense in Hyper-V first- and wound up doing Untangle (free) for the moment on the appliance.

    One consideration: server maintenance takes out your LAN :D.

    That's why I have a separate box. Need my tinkering buffer ;).

    [Note- I wound up with Untangle because I couldn't get Sophos XG Home running on the appliance, and I couldn't get pfSense (or OPNSense) to easily run in passive mode, which Untangle did effortlessly- I'm keeping my Edgerouter4 as router/edge firewall/DHCP server for now]
     
  6. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,325
    Joined:
    Jul 26, 2007
    I know what you mean, fortunately for me I use Bell Fibe and a HomeHub 3000. The 3000 doesn't go into bridge mode, instead you designate a MAC address as the DMZ and that NIC gets an external instead of an internal IP. That means the 3000 can provide Internet access simultaneously and independently of pfSense should I screw up and need to look up what I did wrong this time and how to fix it. I have the 3000's WiFi running so I don't even have to move cables around when pfSense/ESXi is down.
     
    IdiotInCharge likes this.
  7. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,109
    Joined:
    Jun 13, 2003
    That's pretty nice-

    On another note, found ProtectLI's source on AliExpress, not much of a discount after all: 7200U Appliance

    This is basically my 'perfect'. ECC support would be nice, but overall, that's as small as you're going to get with dual drives for a ZFS mirror for pfSense in a 'modern' platform with real cores. I'd run ESXi on it myself, and perhaps more than one router OS instance alongside my other stuff (Ubiquiti apps, pihole...).
     
  8. Killerxp100

    Killerxp100 Gawd

    Messages:
    748
    Joined:
    May 20, 2011
    I've thought about this as well. I do have an exsi box and have messed around with a pfSense vm. I'd like to do it, but I know I usually have to shut down my host at least once a month. Not sure I could get away with the down time.

    Question, why don't you use passthrough for your nics? Is there any benifit other than not needing so many physical nics?


    These qotom boxes look perfect for my needs. I'd Iove to get one of the six nic 7th gen one's but I can't justify spending so much on something that'll most likely be underutilized. $300ish is about what I have to spend. I don't think I have any need to run esxi on top, as my other host does pretty much everything I need.
     
  9. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,109
    Joined:
    Jun 13, 2003
    The DDR3-based ones would probably work just as well. I mentioned the newest ones because they're likely as fast as you'd want for any of the available routing OS distros for basic services and IPS etc. packages on top.
     
  10. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,325
    Joined:
    Jul 26, 2007
    It is one less variable to deal with when you can avoid passthrough.

    I also have NICs teamed in ESXi so I not only have more throughput (when multiple clients connect) I also have redundancy if a NIC or switch port fails. Way easier to do in ESXi than on a managed switch.
     
  11. Machupo

    Machupo Gravity Tester

    Messages:
    5,345
    Joined:
    Nov 14, 2004
    Those qotom boxes look pretty snazzy. I am running a 1u netgate octocore box I picked up secondhand. Throwing every package i can use at it including ssl proxy and a massive inspection database and can't make it sweat. Total overkill for my 100/5 connection.
     
  12. Killerxp100

    Killerxp100 Gawd

    Messages:
    748
    Joined:
    May 20, 2011
    I ended up getting a Qotom Q355G4. It's a four nic i5 5200u model. Should be perfect for my needs for the foreseeable future. $219 shipped from China, supposed to be here early next week. Add in a $25 8gb ddr3l stick and a 64gb msata ssd I have laying around, should be a kick-ass low power pfSence setup.

    I'm still interested in virtualizing at some point. I did test it out over the weekend getting virtualized pfSence set up on my esxi box. Worked perfectly actually still have it going running the home network. I did end up using passthrough for the pcie nic I have. After more research I've seen several people set up a hardware failover pfSense box for when they need to take their host down. Some have even set it up to where the failover automatically boots and takes over when the virtualized router goes down. Seems pretty cool, maybe someday I'll dive more into it. For now a simple and in my case a more reliable hardware solution will work. I tend to tinker and over think these things way more than I should. At least I learn a lot while doing so.
     
    Last edited: Nov 19, 2018
  13. Biznatch

    Biznatch [H]ard|Gawd

    Messages:
    2,042
    Joined:
    Nov 16, 2009

    I've been running pfsense virtualized in a bunch of production environments for the past 6+ years. It's completely stable/supported and I've never had any issues. Yes when you do maintenance on the host itself you have downtime, but that is a pretty rare event.
     
  14. EniGmA1987

    EniGmA1987 n00bie

    Messages:
    58
    Joined:
    May 2, 2017
  15. JeffBlair

    JeffBlair Limp Gawd

    Messages:
    330
    Joined:
    Jul 13, 2009
    Right now I've got mine in a ESXi VM. But, the issue I'm having is every time I reboot pfSense, the whole box locks up, and I have to power cycle it. Don't know what's causing it, but it's a pain. Might get that mini-PC that someone listed. Had it running on another one, but kept dropping the network because I was having to use a USB-to-LAN adapter that kept dropping out.
     
  16. Biznatch

    Biznatch [H]ard|Gawd

    Messages:
    2,042
    Joined:
    Nov 16, 2009
    I don't use ESXi but are there any logs you can view on what failed prior to the crash? Are you running the latest software updates/drivers/firmware?
     
  17. JeffBlair

    JeffBlair Limp Gawd

    Messages:
    330
    Joined:
    Jul 13, 2009
    No, that was the first thing I looked at. It just totally locks everything up. I did just upgrade ESXi to the latest version, and it seams like it reboots now without an issue. But, I should be getting my standalone tomorrow. So, I won't have to worry about that any more.
     
  18. Red Squirrel

    Red Squirrel [H]ardForum Junkie

    Messages:
    9,378
    Joined:
    Nov 29, 2009
    Curious how this is done from a security standpoint.

    Wouldn't that involve hooking up the VM server straight to the internet? I know you would assign the outside NIC only to the pfsense VM so the other VMs arn't actually facing the internet, but I still would feel a bit uncomfortable with that, I like to have a physical barrier between internet and LAN in the form of a physical box with two NICs.

    Anyone familiar with the Netgate boxes?

    https://www.amazon.ca/SG-1000-micro...id=1544568636&sr=8-3&keywords=pfsense+netgate

    I'm kinda thinking of buying one, as it would be way lower power usage than a full blown computer/server like I have right now.
     
  19. Red Squirrel

    Red Squirrel [H]ardForum Junkie

    Messages:
    9,378
    Joined:
    Nov 29, 2009

    This thread might help you: https://www.digitalhome.ca/forum/19...4496-some-tidbits-fibreop-infrastructure.html

    I have the same service and I too found it annoying being stuck with a double NAT but I was able to get rid of the Actiontec and replace it with an Asus router and custom firmware that emulates the actiontec but allows for pass through. Might be a lot to read through though, I don't recall which specific parts of the thread that helped me but pretty sure that's the one that should have all the info.

    It's even simpler if you don't have TV service and only have internet, as you just need to plug straight into the ONT and assign vlan 35 to the WAN port of pfsense. (been a while since I played with it so don't recall specifics) I ALMOST had TV service working too with just a switch, but it would cut out all the time. There's some type of layer 3 QoS you have to setup and I could not quite get it to work.
     
  20. Biznatch

    Biznatch [H]ard|Gawd

    Messages:
    2,042
    Joined:
    Nov 16, 2009

    You put the WAN on a dedicated NIC, that is only connected to the pfsense box. I haven't heard of any vulnerability on any main hyper-visor that allows bypassing that virtual boundary. Nothing but the pfsense VM is attached, so no other resource has an IP or is listening on that interface. I have a bunch of prod deployments virtualizing pfsense in this way with zero issues.
     
  21. EniGmA1987

    EniGmA1987 n00bie

    Messages:
    58
    Joined:
    May 2, 2017

    Spectre? Meltdown? Thats why they were so bad for enterprise. They allow escaping the virtual machine boundaries and access full system. This wont be the last time such vulnerabilities are found either.
     
  22. Biznatch

    Biznatch [H]ard|Gawd

    Messages:
    2,042
    Joined:
    Nov 16, 2009
    That's a completely different case than a virtual nic, which like I said, I have not seen any vulnerabilities for. Spectre/meltdown gave access to protected RAM due to shitty intel architecture to fluff the up their speeds, requiring a software patch to fix that reduced performance. To my knowledge, it still has not been fixed on the latest gen of intel processors either..... It did not however give access to the full system, just the potential to leak secrets.
     
  23. EniGmA1987

    EniGmA1987 n00bie

    Messages:
    58
    Joined:
    May 2, 2017
    The point was they are a vulnerability that allows escaping the virtual boundary of a hypervisor, and they are a very specific threat in the case of a router being run on a server through a hypervisor. It is the exact case being talked about as a possible risk of the WAN port being in hardware on the system. But if you specifically want only a vulnerability of a NIC itself in a hypervisor, here are two that affect the newer generation 3 virtual NICs of VMWare:
    https://www.vmware.com/security/advisories/VMSA-2018-0027.html
     
  24. Red Squirrel

    Red Squirrel [H]ardForum Junkie

    Messages:
    9,378
    Joined:
    Nov 29, 2009
    Yeah kinda my train of thought, say you have a risky VM that has a port forwarded to it, it's on the appropriate vlan etc, but that VM gets hacked (which on it's own might be fine or even expected), from there they could in theory hop on the firewall and then start changing stuff. Then again you probably want a separate VM server altogether for any VMs that may be a risk. (honey pots, servers etc)
     
  25. tangoseal

    tangoseal [H]ardness Supreme

    Messages:
    6,450
    Joined:
    Dec 18, 2010
    Probably has a Chinese spy chip in it like all of super micro does. Google for proof. Was released info by US government.
     
  26. EniGmA1987

    EniGmA1987 n00bie

    Messages:
    58
    Joined:
    May 2, 2017

    There is no SuperMicro spy chip, that has been proven false. Even without having testing proving the story to be false, it didnt make sense in the first place. The chip described in the story was of godlike capability. If anyone had the ability to create such a device in the size of a grain of rice, our CPUs would be leaps and bounds beyond what we have now. I mean really, has multi-gigabit Ethernet hardware (already not possible in the size of a grain of rice), has enough memory not only for its own bios but to store multiple OS files that have been corrupted so that it can replace the real ones from an OS, has a small logic core so execute code that exploits OS vulnerabilities to gain root level OS access to replace system files, and processes the keylogger files it stores and sends them back to China. All in the size of a grain of rice. Ya how about no.
     
  27. Red Squirrel

    Red Squirrel [H]ardForum Junkie

    Messages:
    9,378
    Joined:
    Nov 29, 2009
    I'd be more worried about the Intel ME backdoor (AMD has something like that too I think) than Supermicro as it affects practically every modern PC. Sadly there does not seem to be much known about that backdoor and anyone that talks about how to disable it gets a C&D from Intel. I've briefly tried to see if I can even connect to the port and I can't, so I don't know how it's triggered.

    But yeah the Supermicro one has not really been proven 100%. Well I guess the Intel one too, but that one is more certain I think.
     
  28. acascianelli

    acascianelli [H]ardness Supreme

    Messages:
    6,743
    Joined:
    Feb 25, 2004
    I'm running pfSense on an PCEngines APU2

    https://pcengines.ch/apu2.htm

    IIRC, these are good to about 700-800MB/s throughput so if you're running Gigabit service probably not a good option. I only have a 150MBit connection. Last I checked it runs about 8-10W. The whole setup (board, SSD, case, AC adapter) only cost me like $175.
     
  29. klank

    klank Killer of Killer NIC Threadz

    Messages:
    2,158
    Joined:
    Aug 22, 2011

    You will not get over 400-500Mbit of service through an APU2. I know because I have one. Basic firewall with no other filtering it would not break 500Mbits/s.