Inexpensive ZFS or BTRFS build

Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #0 r294663: Sun Jan 24 12:00:39 EST 2016
ryan@nas:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.7.1 (tags/RELEASE_371/final 255217) 20151225
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): resolution 640x480
CPU: Intel(R) Pentium(R) CPU G3220 @ 3.00GHz (2993.13-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,C MOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4ddaebbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,C X16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,XSAVE,OSXSAVE,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x2603<FSGSBASE,TSCADJ,ERMS,INVPCID,NFPUSG>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 12884901888 (12288 MB)
avail memory = 12330233856 (11759 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL CBX3 >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 2
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff80ee0e70, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <DELL CBX3 > on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
Event timer "HPET5" frequency 14318180 Hz quality 440
Event timer "HPET6" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xf7800000-0xf7bfffff,0 xe0000000-0xefffffff irq 16 at device 2.0 on pci0
agp0: <Haswell desktop GT1> on vgapci0
agp0: aperture size is 256M, detected 32764k stolen memory
vgapci0: Boot video device
xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xf7d20000-0xf7d2ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0: waiting for BIOS to give up control
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.4.2> port 0xf080-0xf09f mem 0xf7d00 000-0xf7d1ffff,0xf7d39000-0xf7d39fff irq 20 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: Ethernet address: 64:00:6a:50:f7:f9
em0: netmap queues/slots: TX 1/1024, RX 1/1024
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7d38000-0xf7d383ff irq 16 at device 26.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
hdac0: <Intel Lynx Point HDA Controller> mem 0xf7d30000-0xf7d33fff irq 22 at dev ice 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci4: <ACPI PCI bus> on pcib4
ahci0: <ASMedia ASM1061 AHCI SATA controller> port 0xe050-0xe057,0xe040-0xe043,0 xe030-0xe037,0xe020-0xe023,0xe000-0xe01f mem 0xf7c10000-0xf7c101ff irq 18 at dev ice 0.0 on pci4
ahci0: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7d37000-0xf7d373ff irq 23 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci1: <Intel ICH8 AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0 -0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xf7d36000-0xf7d367ff irq 19 at device 3 1.2 on pci0
ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich3: <AHCI channel> at channel 1 on ahci1
ahcich4: <AHCI channel> at channel 2 on ahci1
ahcich5: <AHCI channel> at channel 3 on ahci1
ahciem0: <AHCI enclosure management bridge> on ahci1
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROMs> at iomem 0xd9000-0xe0fff,0xe1000-0xe1fff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
usbus0: 5.0Gbps Super Speed USB v3.0
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
hdacc0: <Realtek (0x0280) HDA CODEC> at cad 0 on hdac0
hdaa0: <Realtek (0x0280) Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek (0x0280) (Analog 2.0+HP/2.0)> at nid 20,21 and 26 on hdaa0
pcm1: <Realtek (0x0280) (Rear Analog)> at nid 27 and 24 on hdaa0
usbus1: 480Mbps High Speed USB v2.0
usbus2: 480Mbps High Speed USB v2.0
ugen0.1: <0x8086> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <SAMSUNG HD204UI 1AQ10001> ATA8-ACS SATA 2.x device
ada0: Serial Number S2H7JD5ZC02580
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors)
ada0: quirks=0x1<4K>
ada1 at ahcich2 bus 0 scbus2 target 0 lun 0
ada1: <WDC WD7500BPVT-22HXZT1 01.01A01> ATA8-ACS SATA 2.x device
ada1: Serial Number WD-WXL1E11XHT03
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 715404MB (1465149168 512 byte sectors)
ada1: quirks=0x1<4K>
ada2 at ahcich3 bus 0 scbus3 target 0 lun 0
ada2: <SAMSUNG HD204UI 1AQ10001> ATA8-ACS SATA 2.x device
ada2: Serial Number S2H7JD5ZC02572
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1907729MB (3907029168 512 byte sectors)
ada2: quirks=0x1<4K>
ada3 at ahcich4 bus 0 scbus4 target 0 lun 0
ada3: <SAMSUNG HD204UI 1AQ10001> ATA8-ACS SATA 2.x device
ada3: Serial Number S2H7JD5ZC02582
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1907729MB (3907029168 512 byte sectors)
ada3: quirks=0x1<4K>
ada4 at ahcich5 bus 0 scbus5 target 0 lun 0
ada4: <SAMSUNG HD204UI 1AQ10001> ATA8-ACS SATA 2.x device
ada4: Serial Number S2H7JD5ZC02565
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 1907729MB (3907029168 512 byte sectors)
ada4: quirks=0x1<4K>
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1496564754 Hz quality 1000
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/ada1p2 [rw]...
uhub1: 2 ports with 2 removable, self powered
uhub0: 21 ports with 21 removable, self powered
uhub2: 2 ports with 2 removable, self powered
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.04, addr 2> on usbus 1
ugen2.2: <vendor 0x8087> at usbus2
uhub4: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.04, addr 2> on usbus 2
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
uhub3: 6 ports with 6 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ugen1.3: <Dell> at usbus1
ukbd0: <EP1 Interrupt> on usbus1
kbd2 at ukbd0
ugen1.4: <vendor 0x046d> at usbus1
ums0: <vendor 0x046d product 0xc051, class 0/0, rev 2.00/30.00, addr 4> on usbus 1
ums0: 8 buttons and [XYZ] coordinates ID=0
em0: link state changed to UP
 
@ carobee
Marvell controllers are pretty crappy which I quite specificly told facesnorth that ASMedia is the way to go if you want/need cheap SATA controllers that just works (tm). :)
Where in EU are you?

Romania.
 
@ facesnorth

Add this to /boot/device.hints
Code:
hint.scbus.0.at="ahcich2"
hint.ada.0.at="scbus2"
hint.scbus.1.at="ahcich3"
hint.ada.1.at="scbus3"
hint.scbus.2.at="ahcich4"
hint.ada.2.at="scbus4"
hint.scbus.3.at="ahcich5"
hint.ada.3.at="scbus5"
hint.scbus.4.at="ahcich0"
hint.ada.4.at="scbus0"
hint.scbus.5.at="ahcich1"
hint.ada.5.at="scbus1"

Also make sure to make changes in /etc/fstab accordingly (change ada1 to ada0).

@ carobee
Hmm...
http://www.amazon.de/PCI-Express-Co...e=UTF8&qid=1454232654&sr=8-1&keywords=ASM1061
http://www.amazon.de/Syba-PCI-Expre...e=UTF8&qid=1454232654&sr=8-4&keywords=ASM1061

Dunno of any Romanian e-tailers :-(
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
Dunno of any Romanian e-tailers :-(

Amazon.de is perfectly fine (at least stuff sold/shipped directly by Amazon), delivery is not prohibitive as it was for us a couple of years ago. Thanks!
The Marvell is currently at home to play with.
 
Pretty sure I typed this in exactly, but I'm getting errors now booting up.
Mounting from ufs:/dev/ada0p2 failed with error 19.
mountroot>

If I hit enter, I get:
panic: mountroot: unable to (re-)mount root.
KDB: enter: panic
Stopped at kdb_enter+0x3b: movq $0,kdb+why
db>

FYI, in /etc/fstab, there were 2 lines one was something like ada1p1 and the other ada1p2 relating to / and swap. I just changed the 1 to 0 on both lines and saved. The other file, I simply typed in what you wrote at the bottom of the file, and double checked for accuracy before saving. Not sure how to copy/paste yet from windows into my putty terminal. Then I did a shutdown -r now.
 
I'm retarded sorry....

Boot the USB-flash or CD with FreeBSD installer
Select Live CD
Login as root (no passwd required)
mount /dev/ada1p2 /mnt
edit /mnt/boot/devices.hint

Change the lines to the following....

Code:
hint.ada.0.at="scbus2"
hint.ada.1.at="scbus3"
hint.ada.2.at="scbus4"
hint.ada.3.at="scbus5"
hint.ada.4.at="scbus0"
hint.ada.5.at="scbus1"

Save

umount /mnt

type shutdown -r now

All fixed
 
Last edited:
The file is empty when I try to edit it, and when I try to save the file, it says "unable to create file "/dev/boot/devices.hint"

I'm retarded sorry....

Boot the USB-flash or CD with FreeBSD installer
Select Live CD
Login as root (no passwd required)
mount /dev/ada1p2 /mnt
edit /dev/boot/devices.hint

Change the lines to the following....

Code:
hint.ada.0.at="scbus2"
hint.ada.1.at="scbus3"
hint.ada.2.at="scbus4"
hint.ada.3.at="scbus5"
hint.ada.4.at="scbus0"
hint.ada.5.at="scbus1"

Save

umount /mnt

type shutdown -r now

All fixed
 
I f*cked up the path, this is not my day....
edit /mnt/boot/devices.hint

Actually I'm also getting an error on mount /dev/ada1p2 /mnt also

R/W Mount of /mnt denied. Filesystem is not clean - r . Forced Mount will invalidate journal contents: Operation not permitted

Full disclosure: I came down this morning, and was already logged into root from the USB login, and had already performed that mount operation (I thought successfully) yesterday. So I simply typed in edit /mnt/boot/devices.hint - it was also empty, so I typed in the contents above and saved it. But at this point, I forgot to umount /mnt. And I also at this point pulled the USB key out before doing shutdown -r now - and it brought me immediately to the db> prompt where I typed reboot. Of course, I still couldn't login without the USB key. So this time when I logged in with the USB key as root, I got the error above when trying to mount /dev/ada1p2 /mnt
 
Last edited:
Thanks. Couple of highlights FYI:

** SU*J Recovering /dev/ada1p2
** Reading 33554432 byte journal from inode 4.

** 3 journal records in 1024 bytes for 9.38% utilization
** Freed 1 inode (0 dirs) 0 blocks, and 1 frags.

***** FILE SYSTEM MARKED CLEAN *****
 
I performed the steps listed. But still getting

mountroot: waiting for device /dev/ada0p2
Mounting from ufs:/dev/ada0p2 failed with error 19.

And sent to mountroot>
 
Last edited:
Uhm... That's very odd, I even confirmed this myself locally just to be sure.
Can you hop on irc and poke me there, it's much faster (and less tedious)?

efnet network --> #freebsdhelp --> diizzy
http://chat.efnet.org/ (if you cannot enter #freebsdhelp just type something random and type /msg diizzy hello

Are the drives listed in different order?
 
Got it fixed. Thanks again for your help!

One question I had is regarding the fsck results. Was there any actual damage done? Or are those types of errors fully correctable? I guess what I'm asking is if there's any reason I should start over from scratch either with the BSD installation or perhaps using a different boot drive? Or is there no reason whatsoever to worry about that.
 
Next question, briefly.

I'd really like to start with just 1 vdev in my zpool if possible, and I'd like a minimum of 2 parity disks. If I stick with just my 4 2TB drives, what's the reason why I wouldn't just make a raidz2 with 4TB useable space and 2 parity drives?

Second, can I have multiple drives of different sizes in a single vdev? Or must they all be the same size? (and still get full use of their native size....ie. not default to the smallest size of any drive within the vdev). That's the only reason I can think of for making a 2nd vdev in my zpool right off the bat (if I bought some larger drives that is....)
 
It's just telling you that one "data block" was marked being used while it wasn't. While UFS(2), journaling and soft updates aren't as robust as ZFS it works very well in general. When a filesystem is marked dirty it means that it wasn't cleanly disconnected (unmounted) for some reason. Most likely because you forgot to unmount it the first time before rebooting.

All is fine ;-)

As for your 4 HDDs you can use RAIDZ2 but according to this post you're getting much worse performance compared to a striped mirror.

I'm not entirely sure how to efficiently align you drives as they are apparently 4k drives but report themselves as 512.

First you need to add ZFS support by adding this line (I usually put it under the ifconfig lines):

edit /etc/rc.conf
Code:
zfs_enable="YES"
Save and reboot

The commands that'll create a striped mirror array (as root):
Code:
sysctl vfs.zfs.min_auto_ashift=12
gpart create -a4k -t freebsd-zfs -l a00d01 ada1
gpart create -a4k -t freebsd-zfs -l a00d02 ada2
gpart create -a4k -t freebsd-zfs -l a00d03 ada3
gpart create -a4k -t freebsd-zfs -l a00d04 ada4
create array0 mirror /dev/gpt/a00d01 /dev/gpt/a00d02 mirror /dev/gpt/a00d03 /dev/gpt/a00d04

Just make sure that ada1-4 are your Samsung HDDs before running the above.
Code:
dmesg |grep SAMSUNG

You should also have a look at this:
http://knowledge.seagate.com/articles/en_US/FAQ/223571en
https://www.smartmontools.org/wiki/SamsungF4EGBadBlocks

You can have multiple different drives but it has some drawbacks, that smallest drive will be the lowest common denominator meaning that if you have (500Gb and 750Gb) your array will only use 500 on each drive and so on. Another downside by doing this is that performance will most likely be affected in a negative way as the drives aren't equal in terms of speed, the drives always needs to be in sync or at least very close to it so if one drive has longer seek time the other will have to "idle" until the other drive catches up.
 
Last edited:
OK, for sure then I need to keep my drives equal within each vdev. Do you feel there is an optimal # of drives in a vdev? Or an optimal way to configure a vdev? I'd consider simply adding a couple more 2TB drives in that case, of comparable specs/speed to the ones I already have. I want to start it off right.

RAID 10 I see has the drawback that you can only lose 1 drive in each stripe, but not 2 in a single stripe. Whereas raidz2 allows any 2 drives to be lost. However, I understand the emphasis on backups. I guess then I don't understand the point of having parity disks at all - if backups are so imperative. What are the best options for regular backups? Most economical, easiest methods that I can perform at least once every 2-4 weeks, if not having it set up to run automatically on a daily or even hourly basis.
 
Hmmm, just noticed the articles on the Samsung Spinpoint F4's. I see a DOS patch, but I have them installed in a BSD system now. They were purchased on 12/26/2010 for $80 each, and haven't been used once.

http://www.newegg.com/Product/Product.aspx?Item=N82E16822152245

Busy at the moment so didn't fully read the articles. Does the patch fix the issue, or are they useless as NAS disks? I have no other use for them.... Can ZFS with ECC RAM and a mirror help mitigate the issue? Or will bad data get written to my mirror and to my backups..
 
I'd say that optimal drives per vdev highly depends on what kind of RAID type you're using and how much "risk" you're willing to take in terms of data loss. Keeping that in mind and remembering that this is home usage I'd say something along the following...

RAID-Z(1) - 4-5 HDDs (5 behing a bit more risky but more space efficient)
RAID-Z2 - 6 or 10HDDs (if you have space efficiency in mind)

Think of RAID as a way to keep you data available "at all costs", it does save you from data loss if a HDD/HDDs dies (depends on storage etc and how many that dies but you get the point) however it doesn't save your if something happends physically to your box or if something malfuctions in a really bad way like the PSU frying everything. RAID also doesn't save you from user errors like someone wiping a directory per se however you have snapshots in ZFS this doesn't apply like on your "standard" array. This is why backups and by that you're referring to external backups are suggested depending on how valuable your data is.

As a home user usually not all data you have on your NAS is as important, personal files like photos, work related stuff etc is usually more highly regarded than lets say the last season of Lost or whatever. Instead of backing up everything you can usually fit everything into lets say 1-2TB. You can do backups using ZFS itself and snapshots but you'll need an x86 box to do this reliably and usually you don't want to spend that kind of money (again, this depends highly from case to case) and parents, friends or whatever doesn't want to house a full sized server. What you can do however is to get a somewhat decent router running OpenWRT for instance and hook up an external HDD which does rsync over VPN or SSH of your imporant files. That sets you back ~150$ incl storage and works good enough as long as you have a decent connection. You can of course get a cheap NAS box but it might be a bit troublesome to get it to do VPN of some kind. On the plus side however you can get RAID1 (at least) and it might even do snapshots of the backuped files.

As for the patching, you need to go into BIOS and change the AHCI setting to compatible and flash the drives using a bootable USB floppy, turn off the computer. Turn it on, go into BIOS again and change it back to AHCI and you're all set. The issue only occurs (from what I understand) when you run SMART tests and write data at the same time, if that ever would occur ZFS will tell you about it too.
 
OK, so here's what I'm thinking. I'll order 2 more 2TB drives so I have 6. The Spinpoint F4 ecogreen doesn't seem to be a good drive. Can I mix in 2 of these http://www.amazon.com/Red-Desktop-Hard-Disk-Drive/dp/B008JJLZ7G with my 4 Samsung drives?

Is there a similarly inexpensive 3 SATA port PCIe card that I can replace the other one with? Or get another one of those same cards for another 2 ports? Is there a way I can simply hack/mode my case or drive holder to be able to hang the 7th hard drive in that case? At that point, I think I'd set this up as a raidz2 and call this box finalized. (undecided if I should do 2 striped mirrors tho instead.)

I do want to keep my music backed up, and I actually have about 7TB of music files. Photos are probably under 1TB, and personal files are probably under a couple hundred GB. Not sure how to backup the music files, but I guess just 1 or 2 large external hard drives?

So my intention is to build a separate 10 or 12TB NAS and give this current one to my parents. I'd ideally like to have each of our networks VPN'd to each other and incrementally backup to one another's NAS. But space requirements would be too big in that case, so I'll just do that with the photos and personal documents.

I have an Asus RT-AC68U and my parents have a Netgear Nighthawk R7000. Both of these routers have USB ports and also offer an OpenVPN service. Should I have an always on VPN tunnel established between these 2 routers or would that slow our network activity down too much or eat up too many resources on the routers? Or would it be better to set up OpenVPN connections directly through these FreeBSD boxes?

For my future personal NAS, I don't want any performance issues, yet I don't want to overspend on the hardware for very minor increases. But I don't want to regret not spending a little more because of sluggish performance.

My uses will primarily be storing photos and music. Music will be in a variety of compressed and uncompressed formats. I will likely be accessing the music from a Sonos player or a phone, or a receiver or a Windows laptop. Photos I'd just periodically want to set up a slideshow. Those are the only 2 things were I suppose performance would matter. Occasionally I think I'll play stored movies, but largely I either just stream those or play blu-rays now. I never really bought into the ripping or downloading movies concept.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
You can use the WD Red but I have personal experience with those HDDs (I'd personally go for Toshiba and HGST over WD). I use these in a number of boxes just fine and they're a bit cheaper too (appears to have 2y warranty in US).
http://www.amazon.com/Toshiba-3-5-I...F8&qid=1454616666&sr=8-2&keywords=toshiba+2tb
I also have very good experience with the 3TB drives from the same series and SMART actually works so they're great consumer drives IMHO.

Channels comes in pairs of 2 or 4 usually unless it's a "HBA" classified card. Unfortunately there aren't any decent 4 channel cards around that are decently priced compared to 8-port ones.
The only card I've seen that might be a option if performance isn't top priority this seems quite interesting.
http://www.addonics.com/products/ad10sa6gpx2.php
Basically it's two ASM1061/62 (don't remember) chipsets each connected to a JMicron portmultiplier, it would be interesting to test such a card. I did have a short conversation with a FreeBSD AHCI driver developer about support for this kind of setup and he mentioned that PMs add complexity that might be hard to predict and may behave bad/odd when errors occurs.
Probably not you want to put in production but might be fine at home. Anyhow, you're better off getting an additional ASMedia 2ch-card for now.

I'm a bit worried about how happy the included PSU would be firing up 7 HDDs at boot but there's one simple way to try that ;-)
It's a 280W with split 12V rails according to the tech specs on it hence my concern.
You can probably use something like this http://www.aliexpress.com/item/Chea...48&btsid=67a9eeef-6427-4ee9-8529-e9e0d1c27677 and some stripes to secure the HDDs.
I would however highly recommend you to get one of these in case you need to replace the PSU or in the future as the current one uses proprietary connections.
http://de.aliexpress.com/item/ATX-2...oard-8Pin-Male-Adapter-Power/32346420024.html

Ideal would be something like this:
http://www.newegg.com/Product/Product.aspx?Item=N82E16816111036
That one is unfortunately out of stock and finding eSATA enclosures without PMs is very hard and usually quite expensive. :(

While using VPN between the boxes themselves would be faster it's a hassle if you want to upgrade the OS or even upgrade OpenVPN itself so I would highly recommend you to run that on the routers.
According to this review the Asus seems to be the slower one and according to this it does about 3-4mbyte/s which is probably more than your upload speed is.

Given the amount of data you might pull it off using a 2TB HDD for now but I'm not sure if either of these routers support rsync (have it installed). You might be able to get on the Asus using Merlin's modded firmware but I'm not sure it includes rsync/rsyncd.

I'd say that you'd be fine using a T20 yourself and an external box for HDDs along with the LSI card. I'm pushing about ~115mbyte/s which is what gigabit ethernet does roughly (Samba) and is most likely the limiting factor off a RAID-Z (4 HDD array).
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
I have no problem using the Toshiba instead, I just thought the Red might be good because it seems to be designed for servers and NAS's. I'm not familiar yet with SMART so I'll have to look into it. I assume my Spinpoint's are not SMART? Or once I apply that patch everything is OK and SMART should work? Do I have to do anything to enable SMART if I install the Toshiba 2TB's? SMART is smart enough to work on mismatched drives with different levels of compatibility?

That card does look interesting. I'm not sure I would say performance isn't top priority, it depends how much performance I would be losing. Looks about $79 + shipping for that card, so not sure I wanna spend that much on something if it isn't a great solution. You're right, for this box, another ASMedia 2ch card is prob the best bet. I do recall earlier someone mentioned the possibility of getting an LSI card for around $65. Perhaps that would have been good instead of the 2 PCIe cards if no further expenses were required in addition. Not sure if that was used pricing or if a decent product though.

OK I'll def get the 8-pin adapter. I may even have a power supply laying around that I could use. Maybe I'll grab one of those hard drive enclosures and see if I can make it work. I'm not sure what you mean by "stripes" though.

As for the eSata enclosure, I kind of like the idea of having everything secured inside the box.

I'll explore having the VPN set up in the router and if rsync is available.

For a 10 or 12TB setup, maybe I'll mimic your setup. The speeds sound great. Possible with 2 NIC's to exceed gigabit limitation?

If I'm going to use 6 drives in RaidZ2, it might make more sense though to go 16TB, as I'm not sure the savings of 3TB over 4TB is enough to justify. I'd be tempted though to use those $99 western digital enterprise drives if they become available again on newegg, unless there's a better option for not too much more. I'd also consider 7 drives in RaidZ3.
 
I have 4 x SAMSUNG EcoGreen F4 ST2000DL004 2TB 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive Bare Drives that I ordered back in December of 2010 and never used. I know I'm an idiot for waiting this long to build my storage solution which is why I'm looking to just go ahead and do it inexpensively. I realize these drives have very well lost most of their useful life even just sitting there unused. To make matters worse they were all bought on the same day from the same location, so more likely to fail together. I just don't want that $320 to go to waste.
I've had 8 of these spinning 24/7 for the past 5+ years in RAID-6. Never so much as a hiccup. I wouldn't worry too much about them. Samsung made very good hard drives.

It is a little ironic to me that you're in the process of using them for the first time as I'm in the process of taking mine out of service and replacing them with something larger though. :p
 
Some quick searching came up with this:

RAIDZ1 vdevs should have 3, 5, or 9 drives in each vdev (IDEALLY 5)
RAIDZ2 vdevs should have 4, 6, or 10 drives in each vdev
RAIDZ3 vdevs should have 5, 7, or 11 drives in each vdev
Z2 doesn't quite match that; using 4K drives, it likes 6, 9 or 13 disks for max capacity. See here, keeping in mind that it counts the data disks. IOW, you'll need to add the RAIDZ level to get the total # of HDs. Figure 4's probably the easiest to read.

I'm also not sure on the pros/cons of mixing drive sizes.
Mixed in 1 vdev or pooling vdevs of different sizes?
 
@ facesnorth
https://en.wikipedia.org/wiki/S.M.A.R.T.
The WDs are probably pretty good but the Toshiba HDDs are usually cheaper and they're known to work just fine.

I think the Addonics is an interesting solution but I have no idea how well it works in general. In theory it should be a pretty cost effective way for SMR ("Archive") drives but I haven't heard of any running ZFS and/or FreeBSD on such a card yet. I've seen people express interest in it however on russian forums and it's also widely available in Japan.

Regarding LSI card, you'll need an external case if you're going to fully utilize such a card. Is it's about 80-100$ + cabling if you go for an internal version.

As far as "stripes" goes I forgot that they're usually called zip/cable-ties everywhere else in the world ;-)

The eSATA enclosure was just a suggestion to offload the PSU and seat the HDDs properly.

You can use 2 NICs but you'll need a switch that supports LACP and it wont increase speed to a single device as Samba doesn't as far as I know do mutiple streams at least as of writing. But having multiple devices that maxes out 1Gbit it would help indeed.

Regarding 3TB vs 4TB it's ~90 (Newegg) vs 125 ( Provantage and B&H Photo) looking at the Toshiba HDDs. What WD HDD are you talking about?
 
Hey guys, sorry to fall off the map like that! But had to take some time away to save up for taxes and put time into some other things. Anyway I'm back to wanting to finish this project so I can get my storage server up and running finally!

At this point I see 2 options I need to decide between.

1) Buy another one of these

http://www.amazon.com/IO-Crest-Port...5423&sr=1-1&keywords=ASM1061&tag=hardfocom-20

and 2 more hard drives. I'm fine going with the Toshiba's if that's a good choice. I'm not sure if I should get 2TB since that's all that will be usable anyway. Or if I should just get 2 4TB drives, so that as the Samsung drives start going bad, and I slowly replace them with 4TB's also, eventually I will have 6 4TB drives and my storage capacity will instantly double once they've all been replaced correct?

I'd be willing to try booting all 6 + 1 ssd on my T20's native PSU. Although I did buy that proprietary cable to have on hand in case I need to replace the PSU. What I want to implement to make it work is staggered spin-up. Anybody doing that? Would it work with my system and drives I have?

For this setup I'd prob just buy that ********** BRP525-2S internal storage rack and use a few zip ties to suspend the 7th hard drive in the T20 case.

2) nd option centers around purchasing the M1015 controller. I guess this involves crossflashing it to an LSI921-8i in IT mode? Are there any benefits to buying this over the 2nd ASM1061 card? Reliability or speed benefits besides the ability to connect more drives? What is the best deal right now on a M1050 controller and what other accessories would I need?

If I go this route, I'd consider buying a bigger case which will hold everything. I'd also consider a bigger PSU if it has any reliability benefits.

The final question is RAIDZ2 or striped mirrors? Are we talking just personal preference here? Or does one have a striking advantage over the other? I suppose striped mirrors have performance advantages, but RAIDZ2/3 offers the advantage of allowing any of the disks in the entire zpool to go bad, which makes me less nervous. How significant is the performance difference? What types of activity will it affect in real life usage?

If I got the M1015 controller and a larger case, I'd probably consider just starting with 9-11 drives in my zpool as RAIDZ2 or RAIDZ3 or striped mirrors. Whatever is best. I like the idea of having a minimum of 2, perhaps even 3 parity disks.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
Welcome back, =)

Go with the ASM1061 controller, it's the most cost effective solution until you get a new case. As for the HDDs it's up to you, I'd just keep the Samsungs for now. The Intel controller doesn't do staggered spin-up, the LSI controller does however.
As far as reliability goes they're equal, the ASM1061-controller is slower but as long as you have mechanical driver it wont really be a bottleneck. You can always expand using a proper HBA later on, the ASM-controller card wont interfere.

As this is home setup, go with RAIDZ2. There are performance penalties but you'll still max out Gigabit speeds without any issue.

Also, you also probably want to update the OS due to recent vulns found in OpenSSL just like you did before. Don't overwrite files you've edited including the user/password-files during the mergemaster step ;-)
 
OK thanks! I may need some step by step help on the update from you in a bit.

Alright I guess the downside to the ASM1061 is mostly that I can't do staggered spin-up. If I picked up an LSI controller, I could do that.

Oh I would definitely be keeping the Samsung drives until they die. Was just trying to decide if I wanted to go ahead and start off with more drives in my zpool. At a minimum I'm going to order 2 more drives and do a 6 drive RAIDZ2 zpool. That would be simplest.

Is it true if I replace all the drives with 4TB drives, the extra capacity automatically becomes available? In that case maybe just buying 2 more 4TB drives makes sense? So as the Samsung's die I'd eventually replace them and have increased capacity once they are all replaced. I don't see the Toshiba available in 4TB though, is there a similar good deal on 4TB drives for this use?
 
Can anyone suggest a 4TB drive that will work well for my application? I am leaning towards adding 2 4TB drives, maybe 3 4TB drives if I go RAIDZ3, even though I will only be able to use 2TB until I eventually replace all the Samsung 2TB drives. I wouldn't do this any time soon, just as they die, with the eventuality that I will have 16TB usable space instead of 8TB usable space. Since adding a 2nd ASM1061 allows me to have 8 drives total, I may simply do a RAIDZ3 with 7 drives + my SSD for OS. In this case, I would need to mount 2 of the drives with those zip ties on those enclosures. Unless anyone knows of an external product that holds 2 drives which would work? In this case maybe I would need a different ASM1061 card? One with external ports?
 
I don't see those **********'s available anymore.

Best suggestion on a case to hold either 6+1ssd or 7+1ssd?
 
Ahhh just realized the FRYS deal is in-store pickup only. Unfortunately we don't have FRYS stores out here. Hmmm, I may actually just get the 2TB Toshiba's for $70 each then. Seems the per TB price on HD's is still higher on the 4TB's. I like the R4 case, though. I think I'll pick that up. Should I still just go with a 2nd ASM card? Or should I just go ahead and pickup an HBA instead?
 
The case supports (without doing some hacking) 8 x 3.5" drives, 2 x 5.25" which can be converterd to 8 x 2.5" slots or 3 x 3.5".
The R5 also adds additionally 2 x 2.5" on the back of the mounting plate for the motherboard.

So it all comes down to how much you want to expand it in the end. If you want more than 8 drives in total you need to go the LSI route.

dell_t20_back.jpg


That should give you an idea... ;-)
 
Last edited:
Back
Top