pfSense help for a n00b

S

Synthetickiller

Limp Gawd
Joined
Apr 5, 2009
Messages
285
I recently built a pfsense box & I've found that the dedicated pfsense forum isn't fond of fielding questions that are more basic. Maybe you guys can help me out.

I have a few questions.

1. Does anyone have a cheat sheet or solid walkthrough for certain ssh commands? Here's an example of what I've having problems with:

Login with SSH and open the shell.
Run /usr/local/sbin/ufslabels.sh
Add the line ahci_load="YES" to /boot/loader.conf.local
reboot the machine
Login with SSH and open the shell
touch /root/TRIM_set; /etc/rc.reboot
Click to expand...

If I type in "/boot/loader.conf.local", I get permission denied & I can't find out why as it's too basic of a question. I've tried other walkthroughs & the commands either do not work or I don't have permission. I can't log in under su.

2. This follows questions one, but how do you increase mbuf max? It's 25600 by default. I read that intel nics require a higher mbuf max & I don't want kernel panics.

3. Has anyone set up a dual SSD pfsense box?

I have a spare samsung 840 250gb drive. The OS right now resides on a 30gb msata toshiba. I was thinking of using the 840 as a cache drive. Good idea, bad idea?



Thanks in advance to anyone fielding these n00b questions. At least installing 2.1.4 w/ intel i210 nics went almost as smooth as butter. :D
 
Wasting a 250GB SSD on a 1GB install that will never see 1.5GB on it makes me think you're REALLY high, made of tons of money, or confusing FreeNAS with pfsense.

If you're running into mbuf problems on a home router, you're doing something seriously wrong.


That said, SSH is SSH. make sure it's turned on and make sure you have the password.
 
Last edited:
mwarps said:
Wasting a 250GB SSD on a 1GB install that will never see 1.5GB on it makes me think you're REALLY high, made of tons of money, or confusing FreeNAS with pfsense.

If you're running into mbuf problems on a home router, you're doing something seriously wrong.


That said, SSH is SSH. make sure it's turned on and make sure you have the password.
Click to expand...

Where do I begin?

If you actually go on pfsense forums, you'll see people using far more than 1gb. It depends on your needs & packages installed. Large caches are extremely useful for updating multiple windows machines. Obviously, there are other uses as well.

Having a single spare samsung 840 makes me rich? :rolleyes:

As for the intel nic problem & my "doing something seriously wrong," please actually read the basics... at least I did my homework:

https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

Certain intel igb cards, especially multi-port cards, can very easily/quickly exhaust mbufs and cause panics, especially on amd64. The following tweaks should help:

In /boot/loader.conf.local - Add the following (or create the file if it does not exist):

kern.ipc.nmbclusters="131072"
hw.igb.num_queues=1
That will increase the amount of network memory buffers, and make the card use one queue instead of multiple queues, to reduce the strain on the system.

The same settings can also apply to em(4) cards, just use "em" in place of "igb" in the setting(s) above.
Click to expand...

I can log into SSH, but can't get commands working as I'm being denied. I can't find any literature on why. I'm not famliar with freebsd, so I figured I'd ask someone who could actually contribute some knowledge. :rolleyes:


Thanks for all the "help." I'm really that much further along along now...
 
Sorry but that last response makes you sound like an asshole, and makes it really hard to want to help you at all....

I can't help with SSH, not sure why you need to use that at all, I never did. As far as I know everything can be done through the GUI, but I haven't tried changing the settings you are looking at.

As for an SSD, I would avoid using that with pfsense as it doesn't seem to handle disks with limited writes well. I had a setup using a Flash card that ran fine for about a year until it wore the card out. Then the box would boot, and within a couple minutes would stop handing out IPs and start throwing errors until it just kind of stalled. It turned out pfsense wore out the card, so I replaced it with a regular 2.5" HDD. I really can't see why you would waste an SSD on that box anyway as there is very little benefit to doing so.
 
Biznatch said:
Sorry but that last response makes you sound like an asshole, and makes it really hard to want to help you at all....

I can't help with SSH, not sure why you need to use that at all, I never did. As far as I know everything can be done through the GUI, but I haven't tried changing the settings you are looking at.

As for an SSD, I would avoid using that with pfsense as it doesn't seem to handle disks with limited writes well. I had a setup using a Flash card that ran fine for about a year until it wore the card out. Then the box would boot, and within a couple minutes would stop handing out IPs and start throwing errors until it just kind of stalled. It turned out pfsense wore out the card, so I replaced it with a regular 2.5" HDD. I really can't see why you would waste an SSD on that box anyway as there is very little benefit to doing so.
Click to expand...

Eh, I ask for help stating I don't know enough & get an ignorant response as if the guy knows better. Considering his pov on mbuf exhaustion, I doubt I'd want to take his advice anyways. It's also not wrong to ask how to change the setting since I keep running into what seems to be a problem with the user mode. I can't find any info on how to use superuser mode or single user mode or to change whatever is denying me permission to change settings. That would be helpful & I see no literature on the topic, anywhere. If I wanted to be told SSH is SSH, I wouldn't have bothered. His post contributed nothing to answering any of my questions.

I want to point out, many things cannot be done in the gui. The gui is great & offers a lot of features, but it's not an option for certain things like enabling trim, increasing mbuf count & a range of others. You wouldn't see so many command line based topics in the pfsense forums if this were the case. Maybe in future iterations of the OS, we'll see gui options, but as for now, no, it's not.

On the SSD topic, I see your point, as that's originally what I thought. From what I've read, enabling trim has fixed a lot of wear out issues; at least that's what they claim on the pfsense forums. That's why I'm trying to enable trim before wearing either my msata or another ssd out.

That compact flash card you had obviously wouldn't benefit from trim, so I can see why your's died.

The SSD is being "wasted" sitting in a box as well. I'd rather use it than not & I don't have a single PC that it would benefit from being installed in.
 
Last edited:
I think you'd be much better off posting in a forum that is more geared towards the OS that is under pfsense. Its running FreeBSD. It seems what you're asking isnt really anything specific to pfsense. In your op you mentioned access denied on that conf file. Lookup chmod.

As much as I absolutely love pfsense and would have its babies if I could, i rarely ever go into the shell or command line. When I do its for some pretty basic stuff like pings, and brining interfaces up/down or look at logs. All of which can be done inside the web interface anyway.
 
jadams said:
I think you'd be much better off posting in a forum that is more geared towards the OS that is under pfsense. Its running FreeBSD. It seems what you're asking isnt really anything specific to pfsense. In your op you mentioned access denied on that conf file. Lookup chmod.

As much as I absolutely love pfsense and would have its babies if I could, i rarely ever go into the shell or command line. When I do its for some pretty basic stuff like pings, and brining interfaces up/down or look at logs. All of which can be done inside the web interface anyway.
Click to expand...

Thanks. That gives me a direction to look into.

Much appreciated.
 
If you're running it on an SSD make sure you're running a nanobsd image else it'll kill the drive within 6 months.

mbuf max I think it's in advanced option else it can be done with sysctl. If you are running a nanobsd build then the root will be mounted read only hence why you are getting access denied, you'll need to remount the partition read-write then remound read only when you're done.

And using an ssd for pfsense is overkill full stop, won't hurt anything but a 4gb CF card will do just as well.
 
As a FreeBSD user I can help you with a few things...

/boot/loader.conf.local should be /boot/loader.conf and the AHCI driver should be loaded by default, are your devices called ada*?

mbufs are most likely properly configured by default, run with the defaults and if you encounter issues try to change it if issues occurs.

Using 840 drives for pfsense itself is a waste of money as a few else have indicated. You can do mirroring but I have no idea if pfsense supports gmirror which is only supported using the MBR partitioning scheme and doing 4k alignment is a bit of a hassle.

That said, I've been running FreeBSD without any major tweaks off a USB-stick for months with logging and that box is doing fine despite the wear.
//Danne
 
I've killed a Crucial C300 (admittedly low end drive) with pfsense in about 3 months, it's the logging apparently.
 
Synthetickiller said:
If I type in "/boot/loader.conf.local", I get permission denied & I can't find out why as it's too basic of a question.
Click to expand...

It's because you're trying to execute (run) a config file.

Try vi /boot/loader.conf.local

Then change the line likes is says.

Bring up a vi cheatsheet in another window before preceding.
 
bmh.01 said:
I've killed a Crucial C300 (admittedly low end drive) with pfsense in about 3 months, it's the logging apparently.
Click to expand...

All this pfsense kills SSDs seems strange. Does it write TBs of logs daily?
 
It's not odd, it makes absolutely no sense - at least if you can do basic math and have any understanding whatsoever about the data volumes involved.
 
diizzy said:
As a FreeBSD user I can help you with a few things...

/boot/loader.conf.local should be /boot/loader.conf and the AHCI driver should be loaded by default, are your devices called ada*?

mbufs are most likely properly configured by default, run with the defaults and if you encounter issues try to change it if issues occurs.

Using 840 drives for pfsense itself is a waste of money as a few else have indicated. You can do mirroring but I have no idea if pfsense supports gmirror which is only supported using the MBR partitioning scheme and doing 4k alignment is a bit of a hassle.

That said, I've been running FreeBSD without any major tweaks off a USB-stick for months with logging and that box is doing fine despite the wear.
//Danne
Click to expand...

I was able to put in a command simliar to /boot/loader.conf (I honestly can't recall) & it brought up what drives were what (I believe it said ADA), but now it states something about how a change was made & it doesn't display the drives. I'll have to back track & see what happened.



squishy said:
It's because you're trying to execute (run) a config file.

Try vi /boot/loader.conf.local

Then change the line likes is says.

Bring up a vi cheatsheet in another window before preceding.
Click to expand...


These are things never mentioned. For people using something based on freebsd for the first time, there's just no way to innately know where to source info.

You've saved me days if not weeks of finding the info I need. Thank you! :D

TCM2 said:
It's not odd, it makes absolutely no sense - at least if you can do basic math and have any understanding whatsoever about the data volumes involved.
Click to expand...

This is what I run into...

Unless you've installed a package & you're pushing that much data across your network that would drive such a high read/write environment, it won't happen. I've seen reports of people running SD cards for 2 years w/o a failure (died after 2, but that's neither here nor there. :D) You can imagine the limited number of read/writes on those vs any newer SSD.

There's so many packages, unless

I'm not going to get into if using an SSD is a waste. Just reference seek time for SSDs vs HDDs & that's one of the major reasons I see people using to justify ssds in these routers. If my spare was a 64gb, no one would bat an eye, but since I say it's 250gb, everyone's loses their minds. ;):p

bmh.01 said:
If you're running it on an SSD make sure you're running a nanobsd image else it'll kill the drive within 6 months.

mbuf max I think it's in advanced option else it can be done with sysctl. If you are running a nanobsd build then the root will be mounted read only hence why you are getting access denied, you'll need to remount the partition read-write then remound read only when you're done.

And using an ssd for pfsense is overkill full stop, won't hurt anything but a 4gb CF card will do just as well.
Click to expand...

I've searched. I can't find mbuf as an option.

I was able to get sysctl to change it though. One more thing done. Thanks.


I'm using an atom based itx board. I can either install a USB drive, msata (what I'm using now) or hdd/ssd sata drive. CF/SD would not be ideal.

bmh.01 said:
I've killed a Crucial C300 (admittedly low end drive) with pfsense in about 3 months, it's the logging apparently.
Click to expand...

What packages were you running? Did you change any of the logging settings?
 
Nope left it as before as I had done on hdd installs without issue no squid caching or anything like that, lasted around 3 months before I started getting errors where it couldn't read/write from sectors on the drive. Will eventually cause a kernel panic usually, it's exactly the same behaviour you see from a dying sd/cf card.

Old thread below has some info/data in here, a search on the pfsense forum should bring up plenty of topics. Switched to running the nanobsd image which runs mostly on ramdisk with seldom writes to the drive when needed and haven't had a problem in over two years. That said I think being a cheap, small drive didn't help. Whilst not using the whole drive I have no idea how effective the wear levelling on the C300 was.

https://forum.pfsense.org/index.php?topic=34381.60

As for running CF/SD I just use a sata<->sd/cf bridge from ebay and it works as well as anything has the benefit of easy backups by removing the card and reading it with dd on something *nix based. Not that you really need to do that very often, i'd go usb if you don't want to use a bridge card less power than running a fully blown ssd.
 
There is nothing there that would indicate problems with SSDs. Your drive probably just failed in no relation to pfsense.
 
The whole idea behind pfsense is the intuitive GUI, there is very little need to do stuff in the command line unless you're doing some really advanced stuff.
 
I'm fairly tired of the SSD topic. If I want to put in an SSD & use it as a cache due to it's extremely low latency, that's not an odd or useless endeavor.

As for SSDs wearing out, everyone who says they will are 100% full of it & everyone who's had an SSD die had a lemon.

Read this thread from XS concerning SSD write endurance: http://www.xtremesystems.org/forums/showthread.php?271063-SSD-Write-Endurance-25nm-Vs-34nm

Also read this:
http://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte

We're not writing even CLOSE to 1PB of data to these things under pfsense. SSDs are not comparable to write performance of USB flash drives or CF/SD cards. I hope we can all put that discussion to rest.

I've had two SSDs die in 5 years of owning SSDs as a whole. Both were OCZ agility 120gb & 30gb SSDs; they both died within 18 months of receiving them. We all know OCZ's track record between 2009 and 2011. All other drives, Samsung, Kingston, Crucial & Intel, still work & perform within an acceptable range of read/write speeds.

Red Squirrel said:
The whole idea behind pfsense is the intuitive GUI, there is very little need to do stuff in the command line unless you're doing some really advanced stuff.
Click to expand...

Enabling trim is really advanced then, lol.
 
Last edited:
Actually SSDs *DO* wear out, it's just the way the tech works. You can only write so many times to the NAND chips before they die. It's kinda like CD-RWs you could only write so many times to them till they'd start to fail.

That said SSD for an OS drive is usually fine as it wont do that much writing, you just want to move the logs to a spindle drive or syslog server.
 
That said, you _wont_ wear out the flash memory within 3 months using pfsense. You'd just got a bad drive and that's it.
//Danne
 
Did you get this sorted out?

You actually don't really even need to use SSH to type those commands in. If you have a monitor attached to the computer you're installing on (Most likely do right now since you are installing it) once the box boots up you can type 8 to drop into a shell window. You can type the commands from there.

As squishy said vi is the text editing tool of choice (why I don't know lol) as a default on all *nix distros. I can't think of one that didn't have that installed on it by default so if all else fails you can usually fall back onto that. It is worth learning the very basics as you will probably be forced to use it again at some point.
 
Red Squirrel said:
Actually SSDs *DO* wear out, it's just the way the tech works. You can only write so many times to the NAND chips before they die. It's kinda like CD-RWs you could only write so many times to them till they'd start to fail.

That said SSD for an OS drive is usually fine as it wont do that much writing, you just want to move the logs to a spindle drive or syslog server.
Click to expand...

I won't even go into how hilarious it is to compare CD-RWs to NAND flash rewrites. It's been shown that these drives can write an insane amount of data before meeting it's maker.

How many drives fail due to NAND wear out vs controller failure? I'm not asking this rhetorically. It's something I haven't seen, but i can't find a lot of literature proving that SSDs are dying all the time because of worn out NAND.

I have a hard time beliving text logs take up that much space & pfSense writes 20gb+ per day.

diizzy said:
That said, you _wont_ wear out the flash memory within 3 months using pfsense. You'd just got a bad drive and that's it.
//Danne
Click to expand...

Btw, those 2 drives that died had controllers fail. The nand didn't wear out.

bman212121 said:
Did you get this sorted out?

You actually don't really even need to use SSH to type those commands in. If you have a monitor attached to the computer you're installing on (Most likely do right now since you are installing it) once the box boots up you can type 8 to drop into a shell window. You can type the commands from there.

As squishy said vi is the text editing tool of choice (why I don't know lol) as a default on all *nix distros. I can't think of one that didn't have that installed on it by default so if all else fails you can usually fall back onto that. It is worth learning the very basics as you will probably be forced to use it again at some point.
Click to expand...

I haven't had time... car problems & the like. Everything else is set up.

I happened to find a spare Sata II Intel 320 160gb SSD that I'll never use. That's going to go in the rig over the 840.

SSH isn't the problem, it was just getting the basics down. It's a big secret if you didn't start out with coding / command line & were mostly a hardware guy / OC guy like myself. I agree, it's always worth learning the basics.

I'm probably doing a reinstall later tonight when no one's using the internet. I'll try to get everything, including trim enabled.






Btw guys, after reading this thread on pfSense, no one seems sure of SSD failure rates, but I'm a betting man my SSD won't die quickly. ;)

https://forum.pfsense.org/index.php?topic=34381.60

These guys tested the 40gb version of the Intel 320 drive. Since I happen to have the 160gb version, which basically sat in a drawer since I bought Lenovo w520 at the beginning of 2013, I'll be more than happy to test it out for you guys. :D

Here's the "review" of the drive going to "bad health"

http://diit.cz/clanek/ssd-deep-in-hell-5-1

What I'd like to know are the actual details of this death study. I've seen that health program before (I can't place it's name) & it's said a LOT of my drives were "bad" & they've been working fine w/o a performance hit for 3+ years. Other people have seen it's results & find it to not function properly. At least that's how it was a year ago.
 
I have killed flash drives due to NAND failure, though those NANDs tend to have less write capacity.

I think the issue with something like pfsense is it's lot of tiny writes. Then again I imagine the same can be said for a standard Linux system. My file server that I built last year has a SSD for the OS drive and the wear level count is only 1 according to the smart data... not sure how accurate that is though. My workstation is 6 but that gets heavier use as I download stuff to the desktop etc. Should probably try to make a habit of saving stuff to the network.
 
You must log in or register to reply here.
Back
Top