• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Linux and RAID

M

M4573R

Gawd
Joined
Jul 30, 2006
Messages
522
I've been researching RAID on Linux because of the future box I'm going to put together. I've never used Linux before and have only set up RAID0 on my windows machine. From what I've read, you cannot install Unbuntu or Fedora onto a raid. So would I have to get an extra drive? Or just not use all my drive space for the raid. (would I be left with 3 small partitions if I went with raid 5)? What about expandability in the future? The guide I was looking through was this: http://www.optimiz3.com/installing-fedora-11-and-setting-up-a-raid-0-1-5-6-or-10-array/
Thanks for any help :p
 
You normally make a 256MB or smaller raid 1 on the first partition that contains the boot and then you can raid0 using the second partition of each disk. linux software raid allows you to create raid arrays using partitions. If the graphical installer does not have an option for raid you can set this up via a live cd before the install.

Here is an example:

Code: 
datastore2 ~ # fdisk -l /dev/sda

Disk /dev/sda: 750.2 GB, 750156374016 bytes
255 heads, 63 sectors/track, 91201 cylinders, total 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7525c771

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63      514079      257008+  fd  Linux raid autodetect
/dev/sda2          514080     1992059      738990   82  Linux swap / Solaris
/dev/sda3         1992060    25446959    11727450   fd  Linux raid autodetect
/dev/sda4        25446960  1465144064   719848552+   5  Extended
/dev/sda5        25447023    48901859    11727418+  fd  Linux raid autodetect
/dev/sda6        48901923  1465144064   708121071   fd  Linux raid autodetect

datastore2 ~ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4] [multipath]
md2 : inactive sda5[0](S) sde5[5](S) sdd5[4](S) sdc5[2](S) sdb5[1](S)
      58636480 blocks

md0 : active raid1 sdg1[5] sdf1[6](S) sde1[3] sdd1[4] sdb1[1] sdc1[2] sda1[0]
      256896 blocks [6/6] [UUUUUU]

md3 : active raid5 sdh6[5] sdg6[7](S) sdf6[6] sde6[3] sdd6[4] sdb6[1] sdc6[2] sda6[0]
      4248725760 blocks level 5, 64k chunk, algorithm 2 [7/7] [UUUUUUU]
      [==================>..]  check = 90.8% (643645524/708120960) finish=50.0min speed=21448K/sec

md1 : active raid5 sdh3[5] sdg3[6](S) sdf3[7](S) sde3[3] sdd3[4] sdb3[1] sdc3[2] sda3[0]
      58636800 blocks level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]

unused devices: <none>

datastore2 ~ # mount | grep md
/dev/md1 on / type xfs (rw,noatime,nobarrier,sunit=128,swidth=512,noquota)
/dev/md0 on /boot type ext2 (rw,noatime)

/dev/md2 was a raid0 that I had mounted to /var/tmp and /tmp but that got lost on a bad disk removal and I did not recreate it so it is listed here as inactive. /dev/md3 is where our images go. We are a medical imaging department.

BTW, the /boot raid 1 is a raid 1 of more than 2 disks (with all disks being mirrors of each other). I install and setup grub on as many drives as it lets me (there seems to an 8 drive limit on grub) so that if any drive dies the system will pick the next drive and the boot will still work. Obviously if you are going for / on raid 0 if any drive dies you will not have that ability.
 
Last edited:
That makes a lot of sense, I didn't think about raid 1 for boot across all drives. But I'm still a little confusing on installing boot to a raid since the guide I read made it seems not possible.
 
I can not help with fedora. I have not used that in 5+ years. I do remember doing this in the past however.
 
@OP

The reason you cannot boot from a striped RAID is that the kernel image has to be whole on a given disk, hence why Linux can boot from RAID 1 but not RAID 0. Another thing to note is that while you can RAID with partitionless disks (i.e. /dev/sda, /dev/sdb as opposed to /dev/sda1, /dev/sdb1) you don't get the autodetect feature, and you cannot partition the resulting /dev/mdx RAID device like you would with a hardware RAID or fakeRAID card.

The best thing to do is to get hold of a live CD and three hard disks (so you can at least try it with RAID 5, or a degraded RAID 10), and have a play about.
 
M4573R said:
I've been researching RAID on Linux because of the future box I'm going to put together. I've never used Linux before and have only set up RAID0 on my windows machine. From what I've read, you cannot install Unbuntu or Fedora onto a raid. So would I have to get an extra drive? Or just not use all my drive space for the raid. (would I be left with 3 small partitions if I went with raid 5)? What about expandability in the future? The guide I was looking through was this: http://www.optimiz3.com/installing-fedora-11-and-setting-up-a-raid-0-1-5-6-or-10-array/
Thanks for any help :p
Click to expand...

I'm running Ubuntu 10.10 x86 on a software RAID-0 array right now.

You need to use the alternate install disc and install equal root and SWAP partitions across all disks, but make a separate boot partition on only one drive, then set the BIOS to boot to that drive first (you don't have to but it boots faster).

Not sure about Fedora, but in theory it should work the same way.
 
parityboy said:
@OP

The reason you cannot boot from a striped RAID is that the kernel image has to be whole on a given disk, hence why Linux can boot from RAID 1 but not RAID 0. Another thing to note is that while you can RAID with partitionless disks (i.e. /dev/sda, /dev/sdb as opposed to /dev/sda1, /dev/sdb1) you don't get the autodetect feature, and you cannot partition the resulting /dev/mdx RAID device like you would with a hardware RAID or fakeRAID card.

The best thing to do is to get hold of a live CD and three hard disks (so you can at least try it with RAID 5, or a degraded RAID 10), and have a play about.
Click to expand...

Yes, linux CAN boot from RAID0/1/5/6 etc, it just has to be setup correctly.

The system I'm typing this from is running the Ubuntu distro from 4 Samsung 80GB drives in softRAID-0 right now. ;)
 
john4200 said:
Please post the results of "cat /proc/mdstat" and also "mount"
Click to expand...

Is this what you wanted?

mdstat:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid0 sdd5[3] sde5[2] sdc5[0] sda5[1]
7810816 blocks 64k chunks

md0 : active raid0 sdd1[3] sde1[2] sdc1[0] sda1[1]
273436416 blocks 64k chunks

unused devices: <none>
Click to expand...

Do remember that the /boot partition can only be on one HDD as GRUB does not understand how to "see" a RAID array as a single device.
Also, SWAP partitions must be made across all HDDs for optimal performance.
 
Last edited:
:~$ mount
/dev/md0 on / type ext4 (rw,errors=remount-ro,commit=0)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
/dev/sdc6 on /boot type ext4 (rw,commit=0)
nfsd on /proc/fs/nfsd type nfsd (rw)
Click to expand...

There you go!

If you need any help doing this, just ask. ;)
 
Red Falcon said:
Is this what you wanted?

mdstat:


Do remember that the /boot partition can only be on one HDD as GRUB does not understand how to "see" a RAID array as a single device.
Also, SWAP partitions must be made across all HDDs for optimal performance.
Click to expand...

I have /boot on a RAID5 array, using software raid
 
Do you have /boot partitions on every disk? RAID 0 was picky for me and only liked it on one disk.
 
Did you use the liveCD or the alternate disc for your install? I didn't know it was possible to not have a separate /boot partition as GRUB cannot physically read a RAID array, unless there is something special about RAID5 that I'm unaware of.
 
i used the alternate disc, just created the raid5 using the installer and didnt create seperate boot partition, i believe the latest version of GRUB does read the raid, if you need to to get any details from the system let me know.

i do remember that as the disk are GPT, i had to create a partition on each disk with the type bios_grub, so maybe this is how it gets round reading the RAID

here is a print from parted


Model: ATA SAMSUNG HD204UI (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 1049kB 99.6MB 98.6MB bios_grub
2 99.6MB 2000GB 2000GB raid
 
That's awesome, I never knew it was capable of that, I'll have to check it out, thanks.
 
mikebuzz said:
i used the alternate disc, just created the raid5 using the installer and didnt create seperate boot partition, i believe the latest version of GRUB does read the raid, if you need to to get any details from the system let me know.
Click to expand...

I assume that is with GRUB2 and not the GRUB legacy.

Edit: I found some info on that:

http://grub.enbug.org/LVMandRAID

I am using GRUB legacy version 0.97.X
 
I learn from someone. Not specific, but generic info. It gives an operational method suitable for many situations

1. Prepare 1-PC with up-to-date Fedora 14 installed. Install latest gdisk. (system AHCI-mode)
2. one long SATA data cable, connect to internal SATA port and lead the other end through expansion slot cover (open 1 slot cover)
3. One Molex-to-SATA-power-cable, double purposes of extending length and provide SATA-power. Same method of leading one end out through expansion slot cover (0pen 1 slot cover)
Note : 2 and 3 are for people who need to do this frequently with minimal hassle. Also If you are willing to open your computer side panel, operation can be much simpler.

Example
4. Buy 3-disks.
5. Boot PC in point 1. Connect new-disk-0 externally. Use gdisk for GPT setup (latest gdisk automatically align)
5a. Partition 1 : 512MB Linux RAID
5b. Partition 2 : 1GB Swap
5c. Partition 3 : 48 GB Linux RAID
5d. Partition 4 : 400GB Linux RAID (example size)
save and quit.
note: unplug new-disk-0, put in new-disk-1. Repeat procedure. Same thing for new-disk-2
6. When everything is done. shutdown PC, collect all the new disks.

7. Goto new PC. install all new disks.
8. Boot with Fedora 14 or newer LiveCD or LiveUSB.
9. Start Installation install-to-disk
10. Manual Disk Layout
12. Follow the example order in Anaconda installer process.
11. Build /dev/md0 RAID-1 with 1 spare. /dev/sda1 /dev/sdb1 /dev/sdc1
12. Setup /dev/md0 for /boot
13. Build /dev/md1 RAID-5 /dev/sda3 /dev/sdb3 /dev/sdc3
14. Setup /dev/md1 for /
15. Build /dev/md2 RAID-0 /dev/sda4 /dev/sdb4 /dev/sdc4
16. Setup /dev/md2 for /vmstore (example say VM testing area with RAID-0 fast I/O, no regard for reliability)
17. Complete the installation. Then Setup GRUB on both disks.

18. Steps 1 to 6 basically allow you a generic way to setup hard disks. You can use the resulting disks for non-Fedora Linux distro as well. You can even use the procedure to partition disk in an exact way for later use on Windows, obviously you need to change partition-type where appropriate)

19. You can do everything inside Fedora 14 LiveUSB/CD without steps 1 to 6 using built-in fdisk.

20 If you check carefully the sector starting point for multiple partitions on one hard disk, fdisk and gdisk differ slightly. 1st partition both start sector 2048, the rest may differ depending on how you use gdisk. (I am talking latest fdisk/gdisk with alignment awareness)

Edit : to practice from step 7 until completion, test using VM environment with multiple virtual-disks
 
Last edited:
mikebuzz said:
Yep, not sure if GRUB legacy supports GPT

Am using 1.98
Click to expand...

Is there a command to download the new version of GRUB?

How did you obtain this and how do you tell your current version?

This would really help, thanks!
 
Red Falcon said:
Is there a command to download the new version of GRUB?

How did you obtain this and how do you tell your current version?

This would really help, thanks!
Click to expand...

grub-install -v will show current version

what distribution are you using?
 
Red Falcon said:
That's awesome, I never knew it was capable of that, I'll have to check it out, thanks.
Click to expand...

No I didn't know that either. :) Obviously things have moved on. Normally I would create a RAID 1 partition on each disk in the RAID for /boot, create the swap partitions on each drive in the RAID, and then pick which ever RAID level for the / partition.
 
While it may be possible to boot from a RAID, I would not recommend that. Kernel-based RAID autodetection only works with old style superblocks (0.90), which can result a huge mess sometimes.

I usually boot from a small SATA flash stick which contains the /boot partition. Those cost almost nothing. This also allows the use of GPT on all harddrives.
 
omniscence said:
While it may be possible to boot from a RAID, I would not recommend that. Kernel-based RAID autodetection only works with old style superblocks (0.90), which can result a huge mess sometimes.

I usually boot from a small SATA flash stick which contains the /boot partition. Those cost almost nothing. This also allows the use of GPT on all harddrives.
Click to expand...

That's a really good idea, I never thought about that. They do make internal ones that go on the mobo USB ports.
 
As others have said you can easily boot linux off software raid. All that is required is that the kernel is loaded off a non-raided partition. In reality you can do this with a partition that is <10 MB a lot of times or even use a thumb drive or something to boot.

Once the kernel is loaded then the raid driver (mdadm) is initialized and thus it can access your array for actually booting the OS. I would say 100-200MB is a good partition size as it allows for disk space for a lot of kernels. I also used this method to boot off a system where the BIOS could not see the controller (couldn't access the raid BIOS utility or see the volumes as boot drives). I simply installed the kernel and booted it from an external HD even through the operating system was on the raid array which was not supported by the motherboard.

A normal kernel is 3-5 megabytes in size and is loaded in 2-5 seconds depending on the speed of the machine.
 
1. Can boot from a mdadm RAID-1 /boot partition.
2. Need to setup Grub on both hard disks housing the RAID-1 /boot partitions.
 
You must log in or register to reply here.
Back
Top