- I also posted this on serverfault, but it is offtopic because of home use -
Hello,
I am currently running my home server using Debian Squeeze, on a Intel Pentium E6700 based PC (Intel DQ45CB Board), and 6GB of regular (non-ECC) RAM.
There is a total of 15 drives connected, mostly green.
I decided to dedicate one a relatively small drive to rsnapshot; before this action, when rsnapshot was running, I was experiencing lots of frame drops and interruptions on my HTPC, which obviously accesses the NAS files through samba. No further issues after this action.
The backup of important data is mostly addressed and automatic, and it is not part of the question.
The other drives host media files, and of course the workload is mostly sequential. Currently, I have no raid volumes on this system. I use aufs to join the filesystem and have a samba share that show files contained on all the individual drives.
I own another system, an old AMD64 x2 3800+, socket 939, Asus A8N32-Sli board, 2 GIGs of RAM, some cheap controllers to allow more drives.
This system runs OpenMediaVault (debian based), and hosts a 4x2TB software RAID5.
I used xfs, tuning the su and sw parameters accordingly with my configuration. The performance is excellent for my workload.
This system has other drives, used as single disks. I use aufs to join them and show all files in a single smb share. It works just fine. Obviously the performance is lower when I access files on the single drives.
I am trying to modify the main system (the Intel based one) to use raid 5.
I know about baarf, read a lot of scary stories about very long recovery times of raid 5/6, but I really cannot spend an equivalent time trying to recover data from a failed hard drive when this happens. This has happened twice, but I have been lucky enough to be able to recover 99% of the data. Only, it took a lot of time.
My idea is to create multiple raid-5 arrays (4 drives) on the Intel based system, and to keep up using aufs to join those filesystems. I have no doubt it works as I essentially already used this approach on the other system. I also would like to have enough free space on all the array, at least to be able to tolerate a degraded array. I mean: when a disk failure happens, and an array changes its state to degraded, instead of immediately replacing it with a new drive and start the (dangerous) recovery, I would start to copy (or move) files to the other available arrays. At the end, I would simply replace the failed drive and rebuild a new array with the good drives and a new one, and eventually (if needed) re-balance.
I understand that leaving the array in a degraded state is equivalent to having a raid-0 array, and that this solution is obviously not enterprise proof, but take into account that we are talking about a home server, and that the costs must stay low (along with the noise). Also, the data on the raid volumes is not that important, some data loss is tolerable (of course I would like to avoid or minimize the chance of data loss).
I usually checksum new files before putting them on the server. I like ZFS features, but I would prefer to stick to linux. I'm satisfied with the checksum system (wrote some handy scripts), and had no issues since I started with it; also, I would prefer to avoid server-grade components, because of the higher cost.
Now, the real question. As I am open to alternatives, I am asking you to propose alternatives. Thank you, and sorry for the long post.
Hello,
I am currently running my home server using Debian Squeeze, on a Intel Pentium E6700 based PC (Intel DQ45CB Board), and 6GB of regular (non-ECC) RAM.
There is a total of 15 drives connected, mostly green.
I decided to dedicate one a relatively small drive to rsnapshot; before this action, when rsnapshot was running, I was experiencing lots of frame drops and interruptions on my HTPC, which obviously accesses the NAS files through samba. No further issues after this action.
The backup of important data is mostly addressed and automatic, and it is not part of the question.
The other drives host media files, and of course the workload is mostly sequential. Currently, I have no raid volumes on this system. I use aufs to join the filesystem and have a samba share that show files contained on all the individual drives.
I own another system, an old AMD64 x2 3800+, socket 939, Asus A8N32-Sli board, 2 GIGs of RAM, some cheap controllers to allow more drives.
This system runs OpenMediaVault (debian based), and hosts a 4x2TB software RAID5.
I used xfs, tuning the su and sw parameters accordingly with my configuration. The performance is excellent for my workload.
This system has other drives, used as single disks. I use aufs to join them and show all files in a single smb share. It works just fine. Obviously the performance is lower when I access files on the single drives.
I am trying to modify the main system (the Intel based one) to use raid 5.
I know about baarf, read a lot of scary stories about very long recovery times of raid 5/6, but I really cannot spend an equivalent time trying to recover data from a failed hard drive when this happens. This has happened twice, but I have been lucky enough to be able to recover 99% of the data. Only, it took a lot of time.
My idea is to create multiple raid-5 arrays (4 drives) on the Intel based system, and to keep up using aufs to join those filesystems. I have no doubt it works as I essentially already used this approach on the other system. I also would like to have enough free space on all the array, at least to be able to tolerate a degraded array. I mean: when a disk failure happens, and an array changes its state to degraded, instead of immediately replacing it with a new drive and start the (dangerous) recovery, I would start to copy (or move) files to the other available arrays. At the end, I would simply replace the failed drive and rebuild a new array with the good drives and a new one, and eventually (if needed) re-balance.
I understand that leaving the array in a degraded state is equivalent to having a raid-0 array, and that this solution is obviously not enterprise proof, but take into account that we are talking about a home server, and that the costs must stay low (along with the noise). Also, the data on the raid volumes is not that important, some data loss is tolerable (of course I would like to avoid or minimize the chance of data loss).
I usually checksum new files before putting them on the server. I like ZFS features, but I would prefer to stick to linux. I'm satisfied with the checksum system (wrote some handy scripts), and had no issues since I started with it; also, I would prefer to avoid server-grade components, because of the higher cost.
Now, the real question. As I am open to alternatives, I am asking you to propose alternatives. Thank you, and sorry for the long post.