Need help with Server/NAS build

I used WD's diagnostic tool to zero out the misbehaving "1TB" drive and it reports as 2TB again. This stopped the need for "-f" to work on the zfs create command. Got everything set up enough to allow access and am currently doing test copying to the share.
Yay:
Code: 
brisbane-1# zpool status
  pool: raid5-1
 state: ONLINE
 scrub: none requested
config:

        NAME              STATE     READ WRITE CKSUM
        raid5-1           ONLINE       0     0     0
          raidz1          ONLINE       0     0     0
            label/wd2tb1  ONLINE       0     0     0
            label/wd2tb2  ONLINE       0     0     0
            label/wd2tb3  ONLINE       0     0     0
            label/wd2tb4  ONLINE       0     0     0

errors: No known data errors
brisbane-1# zpool list
NAME      SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
raid5-1  7.25T  3.81G  7.25T     0%  ONLINE  -
brisbane-1# zfs list
NAME      USED  AVAIL  REFER  MOUNTPOINT
raid5-1  2.85G  5.34T  2.85G  /media/raid5-1
brisbane-1#
How do you set ZFS to email alerts? That said, back in linux I did add my email to mdadm.conf but it never emailed me when it kept kicking off drives (before I disabled IntelliPark). *SHRUG*

One thing that bugs me right now is that I can't issue a reboot or shutdown -r now command and expect it to come back. Nothing comes up on my monitor after it does the shutdown part of the reboot, not even POST. This probably has nothing to do with FreeBSD.. but then again this didn't happen with linux. Do you guys have any clue why it won't boot after a reboot?

EDIT:
With mdadm, when a RAID5 array is created it would spend hours rebuilding onto the spare HDD, this same thing should be happening with ZFS, but where can I check on its progress?
 
Last edited:
Ever since using freebsd first in 2004 I've used
"shutdown -p now", actually
"sync && sync && sleep 5 && shutdown -p now" because
that long CLI I read is best for preventing inadvertant data
loss (another ttyN may be open etc...). So about the
reboot command I cannot answer...
(btw the history is saved across reboots so I actually
just type ...
"!sy[tab]" and the shell expands it to the long CLI.
Then I just press [cr] (the enter key)...
although
on some machines I put it in a .sh or something
so ...
...
cd /
. ./shutdown.sh (or "source ./shutdown.sh")
...
typically using a shell I've not set up as extensively.
 
Last edited:
palmboy5: how are your labels setup? did you use the raw device names?

glabel status

gives you output of all your labels.

To reboot the system, simply type:
reboot

Email alerts would be done with a crontab. That's universal among virtually all unices.
 
That said, back in linux I did add my email to mdadm.conf but it never emailed me when it kept kicking off drives (before I disabled IntelliPark). *SHRUG*
Click to expand...

When you were using linux, did you have your email manager configured to actually send it somewhere externally? Otherwise, the mdadm alerts would not go out even if you entered an email address in mdadm.conf.
 
jb_hafo, would it be possible to tie such a script to the pressing of the power button?

sub.mesa, reboot was my original command. I only tried shutdown -r now after reboot didn't work. Both don't work though, in the same way :[.
Code: 
brisbane-1# glabel status
        Name  Status  Components
label/wd2tb1     N/A  ad4
label/wd2tb2     N/A  ad6
label/wd2tb3     N/A  ad8
label/wd2tb4     N/A  ad10
brisbane-1#

Yeah I haven't done any email setup outside of mdadm.conf, oops!
 
There are scripts which can be run at shutdown. I've never
tried to (just) press the power button to shutdown except when
testing, or not even then... except by accident once or twice...
If I recall that is an "unclean shutdown" which negates the
shutdown scripts. (Others may know more who run with the
OS loaded "acpi enabled".)
 
"acpi enabled" isn't default? Would enabling it allow for correct behavior from the reboot command?

Hopefully there is some sort of fan control software out for FreeBSD, but from my searching it doesn't sound promising :[
 
williamlee said:
Also, if you're going to run any kind of RAID6, I wouldn't recommend it with a low number of drives (like 3).
Click to expand...
Huh? RAID 6 already requires all of 2 disks for any size of the array, if all your disks are of the same capacity. So with 3 disks only, you have only 1 available. Besides, RAID 6 requires a minimum of 4 disks (with 50% of all raw space taken by data parity). A RAID 6 array should realistically contain at least 8 disks, so data parity takes only 75% of all space.
I have a SATA II RAID 6 array with 8 x 1.5TB disks on an Adaptec controller.

The best configuration, which offers 88% of usable space, requires 16 disks, but such enclosures are at least 4 times more expensive than for 8 disks.
Each of us can now own what was considered as big datacenters a few years ago!

http://www.adaptec.com/en-US/_commo...mon/cs_2820sa/_learn/RAID_level_compar_wp.htm
http://www.pc-pitstop.com/sata_enclosures/
 
Acpi enabled is default. I never run that way with
day-to-day usage. (I think there are sysctl's which
can do fan-control, but I never pay attention and
read such threads too quickly to remember.)
 
Chimel said:
Huh? RAID 6 already requires all of 2 disks for any size of the array, if all your disks are of the same capacity. So with 3 disks only, you have only 1 available. Besides, RAID 6 requires a minimum of 4 disks (with 50% of all raw space taken by data parity). A RAID 6 array should realistically contain at least 8 disks, so data parity takes only 75% of all space.
I have a SATA II RAID 6 array with 8 x 1.5TB disks on an Adaptec controller.

The best configuration, which offers 88% of usable space, requires 16 disks, but such enclosures are at least 4 times more expensive than for 8 disks.
Each of us can now own what was considered as big datacenters a few years ago!

http://www.adaptec.com/en-US/_commo...mon/cs_2820sa/_learn/RAID_level_compar_wp.htm
http://www.pc-pitstop.com/sata_enclosures/
Click to expand...
I'm not sure if you misread wouldn't as would, but my post was arguing against the use of RAID6 with a low number of drives, not for it. The OP had asked about RAID6 originally.
 
Sorry, @williamlee, I totally misread you indeed. Oh well, hope my answer serves the OP.
 
Technically the data parity requirement is twice the size of the smallest drive in the array, so it may mean that you can "cheat" on the 50% usable space for 4 drives by using a 256GB drive in the array to boost the usable space to 92%. Not sure how well the RAID controllers support this, and it might be a hassle to have 2 different types of drives in the array, if you want to keep spares. But it could be an option if you don't need more than 5.75 usable terabytes (3x2TB + 1x256GB) and a good way to recycle some of your lower capacity drives.

Has anybody tried RAID 6 with one drive smaller than the other drives in the same array?
 
Where do you get fmadm? I ran the find command on /usr/ports and it didn't find anything called "fmadm" (or "*fmadm*").
 
Where is fmadm??

I transplanted the server into a more permanent case and the OS drive ended up as slave rather than master on the IDE. This made the HDD appear to be not the same HDD and FreeBSD was unable to boot. I can only begin to fathom the stupidity in their design here but regardless.. Are geom labels supported under /etc/fstab?
 
How would I go about reestablishing the zpool and everything after reinstalling FreeBSD?
 
Chimel said:
Technically the data parity requirement is twice the size of the smallest drive in the array, so it may mean that you can "cheat" on the 50% usable space for 4 drives by using a 256GB drive in the array to boost the usable space to 92%. Not sure how well the RAID controllers support this, and it might be a hassle to have 2 different types of drives in the array, if you want to keep spares. But it could be an option if you don't need more than 5.75 usable terabytes (3x2TB + 1x256GB) and a good way to recycle some of your lower capacity drives.

Has anybody tried RAID 6 with one drive smaller than the other drives in the same array?
Click to expand...

Is this true? The logic doesn't quite make sense to me, I would think then that the size of the array would be limited to the size of the smallest disk * the number of disks (less parity of course)

so you would end up with (n -2)(size of smallest disk) being your total array size for RAID 6
 
That's what wikipedia says for RAID 5 and 6, but I haven't raid, sorry, read it elsewhere, that's why I was asking if others could confirm that fact.
 
the usable capacity of a RAID 6 array is (N-2)*Smin, where N is the total number of drives in the array and Smin is the capacity of the smallest drive in the array.
Click to expand...

pulled this from that wiki
 
Chimel said:
That's what wikipedia says for RAID 5 and 6, but I haven't raid, sorry, read it elsewhere, that's why I was asking if others could confirm that fact.
Click to expand...

for any parity system, if can cover x number of independent this up to the size of the parity device. ie, you parity device(s) needs to be collectively as large as the largest thing being protected by the parity. if you have 3 drives of 250GB, 500GB, and 1TB then your parity drive needs to be 1 TB or you have to resort to asymmetric mirroring.
 
Yeah, sorry, I read it wrong, it's the usable storage that's based on the smallest drive, not the parity data, so no cheating there... ^-^
So that would make the total size of the parity data at Smin for RAID 5, twice that for RAID 6, or am I wrong again?

@aaronspin, your 2.75TB 4-disk array would have 250/500GB of RAID 5/6 parity data, not 1TB: The "largest thing" in the array is meaningless since RAID limits its usable+parity size to the size of the smallest disk in the array. Each disk will have 125GB of usable space, 125GB of RAID 6 parity data, and the rest of the space for the 3 bigger disks will be unallocated, right?

Anyway, discussing arrays where all drives don't have the same capacity is pointless, I swear I won't do it again!
 
Chimel said:
@aaronspin, your 2.75TB 4-disk array would have 250/500GB of RAID 5/6 parity data, not 1TB: The "largest thing" in the array is meaningless since RAID limits its usable+parity size to the size of the smallest disk in the array. Each disk will have 125GB of usable space, 125GB of RAID 6 parity data, and the rest of the space for the 3 bigger disks will be unallocated, right?
I was assuming you would want parity/mirror coverage for every drive.

Anyway, discussing arrays where all drives don't have the same capacity is pointless, I swear I won't do it again!
Click to expand...

Generally, yes, though there are situations where it makes sense for people. The Drobo device dynamically switches between mirror/parity across different stripes to support different disk sizes. This allows someone to start with say 2 500GB disks, later on buy a 1 or 2 TB disk and then a final 1 or 2 TB disk. Eventually the they can replace the 500GB disks one at a time. So from a piece meal consumer standpoint, it makes sense.
Click to expand...
Click to expand...
 
Soooo... how would you go about reestablishing the zpool and everything after reinstalling FreeBSD?
How do you set up email alerts in FreeBSD?
 
To not continue hijacking that other thread, I'm posting my reply to the following in my thread.

sub.mesa said:
You would have to tell me more about your ZFS server. What OS? Is it 64-bit? How much memory? How much kernel memory that ZFS can use?

In some cases as i've noticed, systems with 4GB would only assign 1GB or so for ZFS by default, which is too little and gets memory starved. But i've yet to do more thorough research on ZFS performance in general. For now i'm focusing on issues which persist with 4K sector drives.

So the fact that you have low performance may not only have to do with 4K sectors; first assure that ZFS has enough memory. 3GB dedicated to ZFS is a reasonable start; but that won't happen with default 4GiB RAM on FreeBSD; you would have to tune the loader.conf to do so, by adding:
vm.kmem_size="3g"
vm.kmem_size_max="3g"
vfs.zfs.arc_max="2g"

Also see: http://wiki.freebsd.org/ZFSTuningGuide
Click to expand...
<EDIT>
I have FreeBSD x64 8.1-RELEASE installed on the following hardware:
PC Power and Cooling PPCS370X 370W
AMD Athlon 64 X2 4000+
Gigabyte GA-MA69GM-S2H w/ BIOS version F6
Corsair XMS2 2x1GB DDR2-800
Western Digital AV-type 80GB IDE OS drive
2x Western Digital WD20EADS 2TB
2x Western Digital WD20EARS 2TB
</EDIT>

I temporarily upgraded the server to 6GB RAM and set loader.conf to have

ahci_load="YES"
vm.kmem_size="5g"
vm.kmem_size_max="5g"
vfs.zfs.arc_max="4g"

Rebooted and tried my test:
Play a 1080p mkv (stored on server)
Boot into a VMWare Windows XP VM (stored on server)
Observe if the 1080p video stutters

It did.

This test scenario was not uncommon from back with my ext4 & Ubuntu based server with which I never experienced this lack of performance.

I can see what you mean about ZFS being memory intensive, 4GB RAM got used up during the test.

As the server is now, I have to hold off doing multiple things at once and that's just unacceptable. :( Please help!
 
Last edited:
You must log in or register to reply here.
Back
Top