I currently have a FreeNAS box with 160 TB of raw space as follows:
vdev1 10x2TB drives in raidz2 16TB net
vdev2 10x2TB drives in raidz2 16TB net
vdev3 10x2TB drives in raidz2 16TB net
vdev4 10x4TB drives in raidz2 32TB net
vdev5 10x6TB drives in raidz2 48TB net
I was at 80% capacity and since this is ZFS, it was time to expand. So I started replacing the 2TB drives in vdev5 one at a time with 6TB drives until I was done. This is after running badblocks on the all, which took about 100 hours per drive. Yes I did some in parallel, but still a huge time sink. It was a huge pain, but I preferred this method to adding vdev6 and then having to deal with 60 spinners consuming power at all times.
So I was wondering what other options are out there today for home media storage good for 150 - 250TB of storage?
Is ReFS finally getting to the point of being half way decent? I know that Storage Spaces is limited to 64TB per volume, but I can learn to live with that I suppose.
The FreeNAS server is rock stable and I run a handful of plugins on it, have a 10GigE connection to my core switch, but I don't like always having to expand when I reach 80% capacity. I'd also much prefer to run Windows since that is what all my other machines on the network are running. Large file copies over CIFS don't even come close to saturating my 10GigE network (running 10Gig to primary workstation as well).
Friend of mine has a similar setup, only he is hardware raid6 based with a pair of servers, each spotting Areca 1882 controllers with 24 x 4Tb drives in a single raid 6 volume. They mirror each other, so if he looses 3 drives at once in one of them, he can replicate from the other one. Besides, I act as his offsite backup and visa versa. He's running Windows server 2008 on those, and is able to saturate his 10G network copying files back and fourth between the servers or to a workstation. And he doesn't have to worry about staying at 80% capacity.
So should I stay with ZFS and FreeNAS, or are there other options out there for "home storage" in the 100TB + range? I have tried FlexRaid (tRAID), drivepool/snapraid, and hardware raid6 using Areca controllers, but I don't really want to go there again. FlexRaid and drivepool were very slow on writes and couldn't even saturate 1GigE, let alone 10Gig. Raid6 on the Areca controllers was ok, but very picky about drives timing out, etc. I thought about Raid60, but all my drives would need to be the same size. I could set up multiple raid6 arrays and then pool them, but I suspect I would have issues with drives wanting to drop out and then loosing the whole pool. Like I said, FreeNAS has been rock stable, but I don't like to 80% limit, nor the fact that I can't run Windows natively. My main plugin is Emby, but the FreeNAS version is always behind the windows one, and with 4 or more clients online, my whole FreeNAS server bogs down. Didn't have this issue when I was running Emby natively under windows.
My current server hardware is as follows:
SuperMicro X10SRL-F mobo
Xeon E5-2683 14 Core 28 Threads
4x Samsung 16GB DDR4 ECC 2133
1x Intel X520 SFP+ NIC
2x LSI SAS9200-e8
1x LSI SAS9211-i8
3x Supermicro 846 chassis with a total of 72 hot swap bays
So is ReFS as serious option at this time with Server 2016, or should I stick it out with FreeNAS/ZFS for another year or two? Anything else to consider?
vdev1 10x2TB drives in raidz2 16TB net
vdev2 10x2TB drives in raidz2 16TB net
vdev3 10x2TB drives in raidz2 16TB net
vdev4 10x4TB drives in raidz2 32TB net
vdev5 10x6TB drives in raidz2 48TB net
I was at 80% capacity and since this is ZFS, it was time to expand. So I started replacing the 2TB drives in vdev5 one at a time with 6TB drives until I was done. This is after running badblocks on the all, which took about 100 hours per drive. Yes I did some in parallel, but still a huge time sink. It was a huge pain, but I preferred this method to adding vdev6 and then having to deal with 60 spinners consuming power at all times.
So I was wondering what other options are out there today for home media storage good for 150 - 250TB of storage?
Is ReFS finally getting to the point of being half way decent? I know that Storage Spaces is limited to 64TB per volume, but I can learn to live with that I suppose.
The FreeNAS server is rock stable and I run a handful of plugins on it, have a 10GigE connection to my core switch, but I don't like always having to expand when I reach 80% capacity. I'd also much prefer to run Windows since that is what all my other machines on the network are running. Large file copies over CIFS don't even come close to saturating my 10GigE network (running 10Gig to primary workstation as well).
Friend of mine has a similar setup, only he is hardware raid6 based with a pair of servers, each spotting Areca 1882 controllers with 24 x 4Tb drives in a single raid 6 volume. They mirror each other, so if he looses 3 drives at once in one of them, he can replicate from the other one. Besides, I act as his offsite backup and visa versa. He's running Windows server 2008 on those, and is able to saturate his 10G network copying files back and fourth between the servers or to a workstation. And he doesn't have to worry about staying at 80% capacity.
So should I stay with ZFS and FreeNAS, or are there other options out there for "home storage" in the 100TB + range? I have tried FlexRaid (tRAID), drivepool/snapraid, and hardware raid6 using Areca controllers, but I don't really want to go there again. FlexRaid and drivepool were very slow on writes and couldn't even saturate 1GigE, let alone 10Gig. Raid6 on the Areca controllers was ok, but very picky about drives timing out, etc. I thought about Raid60, but all my drives would need to be the same size. I could set up multiple raid6 arrays and then pool them, but I suspect I would have issues with drives wanting to drop out and then loosing the whole pool. Like I said, FreeNAS has been rock stable, but I don't like to 80% limit, nor the fact that I can't run Windows natively. My main plugin is Emby, but the FreeNAS version is always behind the windows one, and with 4 or more clients online, my whole FreeNAS server bogs down. Didn't have this issue when I was running Emby natively under windows.
My current server hardware is as follows:
SuperMicro X10SRL-F mobo
Xeon E5-2683 14 Core 28 Threads
4x Samsung 16GB DDR4 ECC 2133
1x Intel X520 SFP+ NIC
2x LSI SAS9200-e8
1x LSI SAS9211-i8
3x Supermicro 846 chassis with a total of 72 hot swap bays
So is ReFS as serious option at this time with Server 2016, or should I stick it out with FreeNAS/ZFS for another year or two? Anything else to consider?
Last edited: