PF Sense test box build

shaggy77

Gawd
Joined
Jul 2, 2005
Messages
803
Hey everyone,

Since we are pretty much dead at work but I can still go in, my boss wanted me to focus on getting a test box built for PF sense. The budget is -$0.00. Effectively, I am looking around at 10 year old computers at this point to perform a proof of concept sand box. Now I understand computers and hardware somewhat but I am mainly a technician or what is called a field engineer at the company. Here is what I hobbled together so far.

Asus P5QPL-AM mATX board
Intel E 8200 Core2Duo It was that or a E5600 or something like that.
2 gigs of DDR2-8500
Toshiba SSD128GB
Realtek add on Ethernet 1Gb card
PFSense 2.4.4

We have static IP addresses from our ISP. Connected directly to the cable modem in the test configuration.

I figured the hardware would be good enough for a test run. So far I have been running into a wall with getting the software and possibly hardware to work. Atheros 1GB NIC is on board. 1GB Realtec expansion card. My boss and I can get the software running but we cannot make it work to the outside (internet). It doesn't ping out. From my readings we might be going the wrong direction with the test build. This also supported my theory that maybe the NICs are the problem. I kind of felt like the test hardware might not be the best. However if proof of concept works, we have an X99 box that would run the software for a full time basis. From what I read, it's better to run the network through dual or quad Intel based NIC. I got no budget even without the Covid-19. I want to make sure if we invest in something like a 4 port card, that it would work. The boss generally handles most of this stuff but it got put into my lap as a challenge.

Let me know what you think.
 
Does PFSense see both NICs?

You have to specify which is LAN and which is WAN during setup. You then have to assign the static IP address, default route, and a static DNS server on the WAN interface.
 
Yes PFSense sees both Nics and during one our builds, we were in the web portal. IPs were assigned on both sides WAN with the ISP and the LAN have a internal test format. We still were not able to get out of the box.
 
During install you should use the automatic method of finding which NIC port is used for what. This way you can verify during the install that the port is seen and gets a connection fine.

If you did that and are having WAN problems, make sure your configurations is set right. Right static IP you need the WAN address, subnet mask, and gateway address. If you dont really have a static IP, then DHCP on the WAN is probably the correct way to go.

You can verify WAN connectivity from the web portal using the ping feature
 
pfSense has known issues with Realtek NICs, in addition to the fact that Realtek NICs are terrible anyway. They straight-up recommend Intel NICs.

You can get dual-port PCIe Intel NICs for $15 or less on eBay, something like a dual port Pro/1000 PT adapter or even something slightly newer like a dual port I340-T2 adapter. Surely you can afford $15?

Examples:
https://www.ebay.com/itm/184191623363
https://www.ebay.com/itm/274077899887

I see that the motherboard has regular PCI slots also, so you could get a dual port Pro/1000 MT adapter for even cheaper. Though PCIe would be better as long as you don't need to use the 16x slot for anything else.

Example:
https://www.ebay.com/itm/333542492594
 
As an eBay Associate, HardForum may earn from qualifying purchases.
If you're using a static IP and a cable modem from say Comcast, sometimes that connection can be sticky in their system. Restarting the cable modem can some times fix that issue. If it's an SMC gateway, those things love to be a hassle to actually get it to assign the IP to the system.

I assume you've had this static IP? There's no other routers or the gateway itself that's trying to hoard it? On the PFSense side it's literally go through the wizard, assign one interface to WAN, one to LAN. If you plug into one side and can get into the 192.x address space, then you have the right one. If not the the other interface is the LAN and you should reverse the connections.

I don't necessarily buy that the NIC doesn't work at all. It might not work well, but I can't say as I've ever come across a hard wired NIC that shows up in the system but just doesn't work. But either way you could easily just reset the system from the command line and swap which NIC is the WAN.

My guess is that your issue is less PFSense and more modem related if it's not just a surfboard and the modem has it's own ability to hand out leases. If you normally get a private IP like 10.x from the modem you're using, then PFSense blocks those networks by default so it's not going to pick up a lease from that on the WAN port. You may need to fiddle around with getting the modem to bridge to actually get your static IP working correctly. I've seen where you can use a static + have the modem also get a real world IP and do NAT, but it requires some modem configuration to make it happy.

What I'd probably do first if you're unsure about the static IP is first set that on a Windows box, and get that figured out and working with the modem. Once that's sorted unplug the modem, unplug the windows box, plug in PFSense WAN, wait 30 seconds, then power the modem back on. That way you should clear any sticky settings and hopefully it might work.

Another point, everyone thinks they need a monster machine to do general firewall / routing. If you're not turning on IDS / IPS (Snort) or doing a ton of VPN traffic, then even the E8200 will basically blow away most appliances you'd normally be buying. That thing is more than enough to handle medium sized businesses with like 1,000+ pcs and gig interface traffic. It might seem small on the desktop computing end, but on the networking side it's a lot more powerful than an ARM or atom based appliance.
 
Last edited:
Hi Everyone,

Just to clarify I am a field tech for large printing equipment not a computer tech persue. I can perform basic diagnostics on systems to determine if they are causing a problem with the equipment. In some cases, I build systems to run the printing equipment when needed. Most of the time we are sourcing a system that will meet our requirements and also alleviates us having to give support. As customers see it, you sold them the system they want to maintain it as well. However we tell them we source from Microcenter so they can have them troubleshoot the system instead of us. I digress.

During install you should use the automatic method of finding which NIC port is used for what. This way you can verify during the install that the port is seen and gets a connection fine.

If you did that and are having WAN problems, make sure your configurations is set right. Right static IP you need the WAN address, subnet mask, and gateway address. If you dont really have a static IP, then DHCP on the WAN is probably the correct way to go.

You can verify WAN connectivity from the web portal using the ping feature

Yes I installed the software with standard options. It does seem to show the Atheros on board NIC and Realtek Expansion NIC. Both my boss and i reviewed the WAN settings. They seem to be OK with our static IP addresses we have with our ISP. As for rebooting the modem, I will let the boss handle it. I don't like messing with that stuff plus I rather him do it so I don't take the blame for it :)



pfSense has known issues with Realtek NICs, in addition to the fact that Realtek NICs are terrible anyway. They straight-up recommend Intel NICs.

You can get dual-port PCIe Intel NICs for $15 or less on eBay, something like a dual port Pro/1000 PT adapter or even something slightly newer like a dual port I340-T2 adapter. Surely you can afford $15?

Examples:
https://www.ebay.com/itm/184191623363
https://www.ebay.com/itm/274077899887

I see that the motherboard has regular PCI slots also, so you could get a dual port Pro/1000 MT adapter for even cheaper. Though PCIe would be better as long as you don't need to use the 16x slot for anything else.

Example:
https://www.ebay.com/itm/333542492594

Afford $15? me? yes. If this was my personal PFsense box, a card would be ordered. Since this is for work, see next statement. Management? IDK. The boss said do not spend any money on this project. Source everything from inside the building. I was trying to see if a few friends in the IT business have any dual or quad Intel based NICs around but I drew blanks. One of my friends deals with enterprise level hardware based stuff. I told that to my boss, he nearly ran me out of the office. LoL Honestly, with Covid-19 our business has been down like many other people. SO I don't think management wants to spend money to make sure they can keep us on the payroll and hopefully through it all.

As for the NIC brand/based types, I have come across this in several web page searches. I watched a few videos on it, it was like the first thing they went to in the review/ build. I showed it to the boss and it was like you can pay for it if you think it's going to work.

If you're using a static IP and a cable modem from say Comcast, sometimes that connection can be sticky in their system. Restarting the cable modem can some times fix that issue. If it's an SMC gateway, those things love to be a hassle to actually get it to assign the IP to the system.

I assume you've had this static IP? There's no other routers or the gateway itself that's trying to hoard it? On the PFSense side it's literally go through the wizard, assign one interface to WAN, one to LAN. If you plug into one side and can get into the 192.x address space, then you have the right one. If not the the other interface is the LAN and you should reverse the connections.

I don't necessarily buy that the NIC doesn't work at all. It might not work well, but I can't say as I've ever come across a hard wired NIC that shows up in the system but just doesn't work. But either way you could easily just reset the system from the command line and swap which NIC is the WAN.

My guess is that your issue is less PFSense and more modem related if it's not just a surfboard and the modem has it's own ability to hand out leases. If you normally get a private IP like 10.x from the modem you're using, then PFSense blocks those networks by default so it's not going to pick up a lease from that on the WAN port. You may need to fiddle around with getting the modem to bridge to actually get your static IP working correctly. I've seen where you can use a static + have the modem also get a real world IP and do NAT, but it requires some modem configuration to make it happy.


I know my boss wants to run something different LAN IP scheme, it was like 172.18.10.xxx with a /24 subnet. After reading what you have to say, it might be an issue with the modem being pissy with PFsense box. I know we set up the ips on the WAN side correct to everything else we have in the office. (I know we have a server but i really know nothing about it.) All I know it's a box behind my boss's desk. I will have to see if I can work on monday.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Update!!!

We got it working this afternoon. The boss gave it another shot this morning. It seemed like we were doing everything right to get it working. He gave in and said if I could source the part, buy it to verify proof of concept. At work, one of our vendors is an electronics supply house. I called them only to find out he had a couple of NICs in stock. When I got there, it was a dual NIC Intel Pro based. He said it was a good thing he asked what we were building. When he went in the back he said he saw the Dual NIC and was like perfect. So I got 2 of them for 30 bucks. Got back to the office. The boss downloaded PF Sense build 2.4.5. The NICs worked instantly. The system loaded up. Within about an hour of reloading the software using yet our original tower, turning off On board NIC, we were in business. A quick bit of additional set up, we were operational for sand box testing. Right now the phone app is being a pain but they set up guide we were working with for PF Sense. Most likely it's a simple setting that needs to be configured but once done, the boss wants to move the company over to the once things are finalized and tested but he can see it happening sooner rather than later. The boss seemed to be happy being able to move around in the web client and locating and understanding the menu structure. He was pretty much we should have been doing this a long time ago.

Once the phone app is verified, there will be a push to get the desktop phones removed from the network. We will not be at the mercy of Cisco and its messed up protocol with PoE phones if the switch dies. The only thing we would have update are the computers with blue tooth adapters and headsets for the all the phones. However there are 2 locations in the office that will require a phone. I think we can press a couple of older brick powered IP phones in their places. We had 2 of them there before prior to the Cisco phones in service.

So I am happy that I pushed for the dual NIC this morning when tried again.
 
If you are doing testing and such you should try out some other firewall OSes like Untangle or OPNsense. IPFire is cool too but a bit more cumbersome and less hardware support.
Untangle costs money, but is the best one IMO. PFSene's antivirus is just CLamAV which is kinda sucky. Untangle paid version gives you a multi-threaded virus scanner powered by BitDefender. You also get things like SSL inspection, spam blocking, and threat protector that scans encrypted traffic too.
 
I guess it will be up to whatever my boss decides. 3 coworkers and I were laid off this morning due to low call volume and no incoming sales, no business in general. Management is hoping they are going to be able to call us back in in a few weeks or less once things settle. Simply put, I removed my employers tooling from my one rolling tool box. Secured my items. Said a few good byes. Left at around 1 PM. There was no reason to hang around more than I needed. Currently my boss and his dad are finalizing the remaining inbound shipments, verifying some stuff and seeing what happens in the next week or two.
 
Untangle is great, ease of use in terms of web filtering and doing all of that, This is the reason they charge you money for it. While you can do all of this in pfsense, it is more complex and tedious. All depends on how much time do you want to spend baby sitting your firewall to get it configured how you want. I love Pfsense, use it at home, but if you wanted complex web filter rules and groups and such, Untangle is worth it. As for OPNSense, i trust them as far as I can throw them since they blatenly stole pfsense code and didn't even bother to remove commenting...
 
As for OPNSense, i trust them as far as I can throw them since they blatenly stole pfsense code and didn't even bother to remove commenting...
Thats because it is a fork of PFsense from some of the old developers back when PFSense went corporate and then started removing drivers and kernel modules for things the new owner company didnt like.
PFSense is the one lying about supporting all the same things as FreeBSD, when they have a good bit of hardware support removed.
 
Last edited:
Just install two nics in your contemporary office workstation and use a VM to test out pfsense. A damn calculator cpu can run pfsense basically cool thing about vm is that you can do all the changes you want and see how it performs. Take a snapshot, fuck it up badly, load snapshot back up and running in 1 min.

If you have a layer 3 switch you can use just one nic

You can use an old laptop with a single nic but you need a l3 switch as well.or use buikt in nic and toss a usb nic on there for testing.
 
Last edited:
Thats because it is a fork of PFsense from some of the old developers when PFSense went corporate and started removing drivers and kernel modules for things the owner company didnt like.
PFSense is the one lying about supporting all the same things as FreeBSD, when they have a good bit of hardware support removed.

Im just glad aesni is not required like they originally planned. No reason you need hardware encryption unless youre VPNing. Not everyone needs vpn usage.
 
Well I got a small update. The system is pretty much on line. A few settings being figured out still but overall on line. My boss is no longer at the mercy if the POE or even the old firewall. I gave him a few minutes of help every week or so testing outbound and bound communications while furloughed. Today was the first test with a live customer. It was new experience for the company and the customer. We would not have capacity it was not for the slow down to catch our breath on this project. So my boss is finalizing his tweaks and he is likes it. It works and well.
 
I am necro threading this...

So far everything has been going smoothly. My boss has played around with configurations from time to time. He set up a PF sense box at his residence. I am thinking about going PF Sense at my house. Today while working with a client we had a need for hardware. My boss and I found a computer from a trade in equipment we wound up scraping to make room in the warehouse. After getting the necessary files and hardware key from the PC, he basically said the computer is useless to us. It's not strong enough to run the current generation of printing equipment the company distributes. It was really meant for that generation of printing tech. Those printers are now too slow and the output for the head tech is ancient. Basically, scrap value they are not worth the time to refurb. So I got to thinking about using it for a PF Sense box. Here is the specs.

Dell Optiplex 990 SSF case (half size)
Intel i7-2600
16GB DDR3 1333mhz Ram
320 GB 7,200 RPM HDD
Windows 10 Pro

The boss said if I want it for the build, I can pretty much have it. The other thing is I can also have the Dual NIC Intel based card I picked up last year for the test bench build. He has PF Sense working on some newer hardware that is supporting the needs of the business. He also said he would give me a hand getting the box set up and running if I need assistance with configuration. I'm thinking about removing the HDD and installing a spare SSD I have sitting on my work bench. 400 GB no problem. Windows 10 Pro is installed. The only thing that makes it seem slow is the spinner. I betcha if there was an SSD, it would feel pretty smooth overall. You sometimes forget how slow spinners are after not using them.

My question is it overkill? ( Ithink I know what the answer is :) )
or
is it too old hardware spec? From what I can tell, even though the CPU is 10 years old, it has all the modern instruction sets (AES-NI) needed for the latest builds for PF Sense.

Use
Currently 300 MB service at home.
Future
Fiber is finally being deployed to the neighborhood. So I would be getting on to that once lit. The wife's company requires VPN which they provide and she does connect with it. She has used it over the years (original WFH pre pandemic). I was talking with others and VPN seems to be a thing to do to home networks now or at least I am now noticing it. So I would want hardware that is going to play nice with any additional demands made by any services added.

Finally.... In case you are enticed by hardware and why I talk some computer. I work as a Wide Format Service Engineer. I basically work on large printers. Yes I deal with some of the desktop equipment but mostly with 54 inch to 126 inch width equipment. The 54 and 64 inch units are roll to roll. Other equipment is known as flatbeds. The units units are either 4 by 8 foot or 6.5 by 10 foot units. These printers use pretty powerful hardware to run the controller software as well as the RIP. This is to carry enough data processing for the printer to operate efficiently. The computer above ran what we call 512 /1024 fixed dot heads on a more primitive RIP program. This was all running through a USB 2.0 connection. Carriage speeds were slow. However print head firing data was handled by a fiber optic cable from the USB board (DAC to another DAC on the carriage board) It was not taxing at all. These units were slow in the range of about 6-7 boards (4' by 8' sheets) an hour depending on profile color configuration and RIP settings. Date range was around 2011 through 2014.

In 2015, new head tech was introduced. Out the fix dot heads. In with the variable dot 1024i technology. USB 3.0 connectivity. Controller used to run the printer. Only used the RIP for data to print. Controller handled everything. This printer needed high end hardware to run it. The first build was an i7-6700 PC with 16GB of DDR4 ram. Turns out slow ram actually effected speed of the printer. If the ram was set at 1833, it would actually stutter. At the time 2133 and 2400 was more expensive but available. Since the company was about to lose the sale, we tossed in some faster RAM and quickly found out that fixed the issue. We also invested in a SSD hard drive to give the printer as much output from the PC as possible. Unfortunately, the national distributor of the equipment turned out to be of questionable character. They decided to abandon the channels and sell direct ( another story in itself) Turns out there were a number of unhappy people. It was also found out the dealership channel or customers were not getting the right information for the product. The example above about the RAM. The manufacturer's own engineers said to get the best components possible since the equipment such a huge leap. The speeds went from around 6-10 boards an hour to around 20 depending on the configuration. The printer offered dual row 1024 configurations from the factory as well as the single row configuration.

In 2018, the company I work for formed a new company as the manufacture came to them. My boss and I sat down. We started remembering all the issues we had with the equipment due to hardware. A plan was formed. We would supply the hardware to run the printer. Everything was at least an i7 and 16GB of RAM. NVME drives. Systems built from 2018 thru mid 2020 were at least an i7 with 16 GB of DDR4 3200. Dual NVME drives of 500 GB each. One drive is for the OS The second handles the controller and RIP data.

2020 to current. The printers have gotten faster through the redesign process. Late last year, my boss and I installed (2) 3 row CMYK. I know this means nothing to most of you. To put it in perspective. The first generation a board would be down about 12 minutes on average quality. The next generation the same board and same DPI was about 8 minutes. Add in a second row of heads. You are down to around 4 minutes or so. Add in the 3rd row of heads, you are down to around the 1:40 mark. In certain environments speed is what matters. The next generation of printing tech that is coming in next month. New head tech requires new drive tech. The heads fire approximately 30% faster than the 1024i model. Due to this speed, there is no more drive belt to move the carriage back and forth. It is actually magnet driven. Less moving parts and wear items!

Back to hardware. We have gone to AMD hardware. At first it was R7 3000 series. Then to R5 when we realized it can handle the output. Still on 16GB of DDR4 3200 or greater. Same specs on the NVME drives. However, we have increased the control drive to 1TB and next year it will be 2TB due to cost and field feedback from what we have seen. We had a few 500 GB drives get so full that the operator could not run the printer. After a remote log in, we found the drives were getting too full with data and operators were not clearing work form the cache on the controller. So 1TB and 2TB drives are going to be installed. The last 4 builds we ran AMD 5600G processors without a problem on B550 boards. Very impressed with the hardware. Also next year's builds are going to be increased to 32GB of RAM due to RIP requirements. Some of the latest RIP software such as Onyx is going RAM hungry and likes 32GB over 16GB for data processing. So our new builds are going to be overbuilt for clients. We might go up to 5700G APU. We are also looking at Intel's 12th gen CPUs. DDR5 like everything else right now is the lynch pin on this matter. However we have no problem using an i5 or i7 11th gen since they will work without a problem. Our biggest issues now are not hardware related but operators installing other programs to the computer that is really meant to run a RIP program and the controller. Nothing else.

I know it was a bit off topic but it also gives some perspective on the system above and how we acquired it. Thanks for the assistance on this project.
 
  • Like
Reactions: xx0xx
like this
If you are running the basics, that setup is plenty.
I run 400Mb service through my pfSense as a VM
-4cores of L5638 (~1yr older than yours)
-8GB DDR3
-100GB allocated HDD

I bumped this down from 6 CPUs and 12GB DDR3 because it wasn't using it.

Mostly idle it sits at 3-5% CPU and 11% MEM. Running speed test, it maxed at 41% CPU @ 468Mb. I only run a few services though: ntopng, Open-VM-Tools, openvpn-client-export and I have AES-NI enabled. I specifically like the ability to run two ISPs through it with an auto-failover to keep the house internets running. The wife and I both work from home and use VPNs that don't have any issues passing through.
 
On my pfSense box, I've run my 250Mb service through an i5 4590 and now an i3 4370 because the i5 was overkill and better used elsewhere. Full speed transfer takes ~1 core. 8 GB of cheap ram was plenty.

The SSD isn't necessary. I keep my install on an old 120GB laptop drive and it only takes a couple minutes to go from cold to running, and it only does that after the power's been out, or the rare update release.

But, do keep this in mind if you do go SSD: Temp files. Tons of temp files. You can wear out that SSD faster than you realize (can't find the article that tested it, but it was quite the daily writes). There is a setting to put /tmp and /var as ramdisks (I use 500MB each, my /var is the hog and currently sits at 300MB) and it drops the drive writes to almost nothing.
 
I think I will pull out the HDD on this unit in case the customer who provided the computer wants it back for some reason. It looks like it has some generic files on it from production. Nothing important looking. When the equipment was being decommed, I asked the owner if he wanted the computer and the files and he said no. There was nothing important on there. Toss in a spare hard drive I have laying around here and give it a whirl.

One other thing I should plan for is the FTTP build out of my neighborhood. It would be a good idea to figure out what the best path is. Right now I am on 300 megabit service. FTTP is going to offer up to 1 Gigabit service to start. If anything I may take advantage of a 500/500 service level. This would fit well within the needs of the household. Costs on replacing equipment will be minimized. No need to replace the current router (which would be turned into AP) The current router is AC based. The extender is also AC based.

My other problem which I just noticed while figuring out if the current hardware will work is placement. Right now, my modem is a totally different section of the house for a reason. First is the one of the closest points to the demarc of the premise. It also has a maybe a 10 foot run to the outlet and another 5 to the modem. The other consideration I have too is keeping the modem in a place in which all family members can see it. My family can see the modem. If there is any question of the internet working, they know exactly where to find it with minimal of searching. My wife knows what to do for basic diagnostics before calling me too. Check the lights. Knows how to power cycle. However, this section of the house doesn't have space for this size box. The place I would put this box in would be the basement. This would be away from the CM, WR, and 8 port switch. All on a shelf with stuff surrounding it. The place I would have to install this box, would be in the basement near my computer station. I have some space but not the runs of Cat 5e or higher back to the networking equipment. I have 1 of them for the computer and that's it. I had 2 but it got damaged. So it was removed. So to move the networking equipment is one thing. Running some new cabling is another. I am starting to think more about this project. But at least I am planning a bit here. I would have hated to do all this set up work to find out I got another issue in the network. hummm.. Might have to look into this further.
 
I don't run a general purpose PC for my pfSense firewall because of the noise and power usage. Although I certainly understand how a free computer can make up for the cost of a dedicated appliance. You don't need much unless you run a ton of packages or have 1Gb Internet at home. Any small, modern ATOM type box would work as long as it sports Intel NIC's. There are several other options too.
 
Back
Top