linux software raid 5, what filesystem?

R

Rakinos

[H]ard|Gawd
Joined
Jul 31, 2001
Messages
1,520
Right now, my 1.5 tb raid 5 array is building in ubuntu using mdadm. I was wondering what your guy's recommendations would be for the file system. This is going to be holding mp3's dvd rips, and other larger files. A lot of people seem to recommend ext3, but I was wondering if there was something a bit more exotic that would work better in this situation. Also, would someone explain to me how lvm fits into the picture?
 
Rakinos said:
Right now, my 1.5 tb raid 5 array is building in ubuntu using mdadm. I was wondering what your guy's recommendations would be for the file system. This is going to be holding mp3's dvd rips, and other larger files. A lot of people seem to recommend ext3, but I was wondering if there was something a bit more exotic that would work better in this situation. Also, would someone explain to me how lvm fits into the picture?
Click to expand...


Think of LVM2 as fdisk for the array. It allows you to slice up the large /dev/md0 (or whatever) into multiple 'partitions'.

I used this guide: http://www.tldp.org/HOWTO/LVM-HOWTO/

This is what I have currently in my setup.

5x400g Raid5 (1 as spare) = md0
8x320g Raid5 (1 as spare) = md1

I then used LVM to slice them up.
Code: 
hoth ~ # vgs <<<<<<<<< Volume Groups
  VG   #PV #LV #SN Attr   VSize VFree
  vg00   1   8   0 wz--n- 1.09T 2.83G
  vg01   1   1   0 wz--n- 1.75T    0
Code: 
hoth ~ # pvs  <<<<<<< Physical Volumes 
  PV         VG   Fmt  Attr PSize PFree
  /dev/md0   vg00 lvm2 a-   1.09T 2.83G
  /dev/md1   vg01 lvm2 a-   1.75T    0
Code: 
hoth ~ # lvs  <<<<<<<< Logical Volumes (will be mounts)
  LV                   VG   Attr   LSize   Origin Snap%  Move Log Copy% 
  vg00-lv00-home       vg00 -wi-ao 150.00G                              
  vg00-lv01-bacula     vg00 -wi-ao 150.00G                              
  vg00-lv02-vmware     vg00 -wi-ao 150.00G                              
  vg00-lv03-files      vg00 -wi-ao 150.00G                              
  vg00-lv04-disk2      vg00 -wi-ao 150.00G                              
  vg00-lv05-portage    vg00 -wi-ao  15.00G                              
  vg00-lv06-disk1      vg00 -wi-ao 200.00G                              
  vg00-lv07-disk3      vg00 -wi-ao 150.00G                              
  vg01-lv02-media      vg01 -wi-ao   1.75T

I then have my fstab pointing to the logical volumes.
Code: 
/dev/vg00/vg00-lv00-home        /home   reiserfs        defaults,noatime        0 0

As you can see I chose to use reiserfs. I didn't know much about other fs like xfs, etc. reiserfs has been great as I've been able to expand/shrink LVM stuff and reiserfs on the fly. I don't think you can do that with the ext3 stuff.

Once you get your head around LVM its really easy. :)
Create a Volume Group. Assign Physical Drives to the VG. Slice up the VG into Logical Volumes for the mount points.

I named my VG and LV to make sence for me, but you can name them anythign you want.

Yes.. I made the entire md1 into 1 huge VG/LV. But if I ever wanted to expand that, I would just add more drives to teh VG and expand as needed. That is why I made it use LVM2 rather than just mount the /dev/md1.


Hope that helps.
 
XFS is rather nice. If you use it, though, you need an UPS with OS support - powering down uncleanly can destroy data. This isn't just with XFS, by the way - even with journaled filesystems some don't aim to save data in the event of unclean shutdown.
 
XFS works very well in larger arrays and when dealing with large files. As unhappy_mage stated though, it can be very unreliable. I was using XFS on puffy for a long while until I had what could be described as a catastrophic filesystem failure that was unrecoverable. After doing some research it seems that these sort of things do happen from time to time with XFS, and it isn't necessarily an actual XFS problem (that is, the actual algorithms and specifications) but bugs in the implementation. The Linux kernel changes a lot so it isn't easy to place blame either. I reformatted and now use ext2 /boot, reiserfs for a very tiny /usr/portage, and the rest is ext3. Ext3 has been around for a long while and there are lots of tools to work with it, being that it is very well understood. If you are feeling exotic and don't mind the off-chance of having something go wrong, definitely go XFS - you won't feel bad. If you regard your storage with data-center like reliability then you will probably want ext3. ReiserFS does not deal well with large storage arrays and should not really be considered.
 
You should never run a filesystem that isn't what most of the developers of that OS are using. That's the key.

No matter how good, changes in the OS, namely in the VFS interface, cause "friction" that can lead to data corruption when changes are not properly applied to all filesystems.

Use whatever filesystem the guy who changed the VFS layer last is using.
 
You must log in or register to reply here.
Back
Top