How To - Create a RAID array in Linux (Ubuntu Edgy - 6.10)

C

caffiendo

n00b
Joined
Jan 10, 2007
Messages
53
I decided to create a 5 disk RAID 0 array in Linux. I'm running Ubuntu 6.10

Here is the procedure I used. I think it is the correct way, but I'm having problems getting the array to mount.
Code: 
sudo apt-get install mdadm
sudo fdisk /dev/hda1 
                    /dev/hde1 
                    /dev/hdf1 
                    /dev/hdg1 
                    /dev/hdh1
(Created a primary partition with the full size of the HD, then used "t" and "83" for partition type)
Code: 
[COLOR="Blue"]cd /dev/
sudo  MAKEDEV md
sudo modprobe md
sudo mdadm --create --verbose [COLOR="Red"]/dev/.static/dev/md0[/COLOR] --level=0 --raid-devices=5 /dev/hda1 /dev/hde1 /dev/hdf1 /dev/hdg1 /dev/hdh1[/COLOR] You can change the --level=0 to the appropriate level for the array you are configuring
***mdadm: layout defaults to left-symmetric
***mdadm: chunk size defaults to 64K
***mdadm: /dev/hde1 appears to be part of a raid array:
***    level=raid5 devices=5 ctime=Wed Feb 21 03:27:41 2007
***mdadm: /dev/hdf1 appears to be part of a raid array:
***    level=raid5 devices=5 ctime=Wed Feb 21 03:27:41 2007
***mdadm: /dev/hdg1 appears to be part of a raid array:
***    level=raid5 devices=5 ctime=Wed Feb 21 03:27:41 2007
***mdadm: /dev/hdh1 appears to be part of a raid array:
***    level=raid5 devices=5 ctime=Wed Feb 21 03:27:41 2007
***mdadm: size set to 156288256K
***Continue creating array? y
***mdadm: array /dev/.static/dev/md0 started.
***denotes operation output
I used /dev/.static/dev/md0 because it was posted in another HowTo that I used some info from. Why not just use /dev/md0? How does this affect the array down the road? id the /dev/.static needed?
Code: 
[COLOR="Blue"]cat /proc/mdstat[/COLOR]
***Personalities : [raid5] [raid4] [raid0] 
***md0 : active raid0 hda1[0] hdh1[4] hdg1[3] hdf1[2] hde1[1]
***      781443648 blocks 64k chunks
Code: 
sudo mdadm -S /dev/md0
Used to stop the array
Code: 
sudo gedit /etc/mdadm.conf
Add the following lines:
DEVICE /dev/hda1 /dev/hde1 /dev/hdf1 /dev/hdg1 /dev/hdh1
ARRAY /dev/md0 devices=/dev/hda1,/dev/hde1,/dev/hdf1,/dev/hdg1,/dev/hdh1
Code: 
[COLOR="Blue"]sudo mdadm --detail --scan[/COLOR]
***ARRAY /dev/md0 level=raid0 num-devices=5 UID=54623f8b:b3b44af:45dca40a:d0833744
This is where it gets strange...
Code: 
[COLOR="Blue"]sudo mdadm -As /dev/md0[/COLOR]
***mdadm: /dev/md0 not identified in config file.
 WHY won't this start, considering that I made the /etc/mdadm.conf file?
Code: 
[COLOR="Blue"]sudo mdadm -A /dev/md0 /dev/hda1 /dev/hde1 /dev/hdf1 /dev/hdg1 /dev/hdh1[/COLOR]
***mdadm: /dev/md0 has been started with 5 drives.
Code: 
[COLOR="Blue"]sudo mdadm -E /dev/hda1[/COLOR]
/dev/hda1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 54623f8b:b3b44af:45dca40a:d0833744
  Creation Time : Sun Feb 25 13:54:18 2007
     Raid Level : raid0
    Device Size : 0
   Raid Devices : 5
  Total Devices : 5
Preferred Minor : 0

    Update Time : Sun Feb 25 13:54:18 2007
          State : active
 Active Devices : 5
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 436417f3 - correct
         Events : 0.1

     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       3        1        0      active sync   /dev/hda1

   0     0       3        1        0      active sync   /dev/hda1
   1     1      33        1        1      active sync   /dev/hde1
   2     2      33       65        2      active sync   /dev/hdf1
   3     3      34        1        3      active sync   /dev/hdg1
   4     4      34       65        4      active sync   /dev/hdh1
This is the step by step procedure that I used. How do I configure this in the fstab? Is it different for RAID arrays? Do I need to include the UUID : 54623f8b:b3b44af:45dca40a:d0833744 in the fstab?

Also, when i created the /etc/mdadm.conf , do I use /dev/.static/dev/md0 or /dev/md0?

Finally what do I use to create a file system?
sudo mke2fs -j /dev/md0 or
sudo mke2fs -j /dev/.static/dev/md0 ?

I created /raid directory for my array mounting point, then adding this to my fstab
Code: 
# /dev/.static/dev/md0  UUID=54623f8b:b3b44af:45dca40a:d0833744    /raid               ext3    defaults  0    0
I then tried creating the FS with this:
Code: 
[COLOR="Blue"] sudo mke2fs -j /dev/.static/dev/md0[/COLOR]
***mke2fs 1.39 (29-May-2006)
***Filesystem label=
***OS type: Linux
***Block size=4096 (log=2)
***Fragment size=4096 (log=2)
***97681408 inodes, 195360912 blocks
***9768045 blocks (5.00%) reserved for the super user
***First data block=0
***Maximum filesystem blocks=197132288
***5962 block groups
***32768 blocks per group, 32768 fragments per group
***16384 inodes per group
***Superblock backups stored on blocks: 
***        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
***       4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
***        102400000
***Writing inode tables: done                            
***Creating journal (32768 blocks): done
***Writing superblocks and filesystem accounting information: done
***This filesystem will be automatically checked every 24 mounts or
***180 days, whichever comes first.  Use tune2fs -c or -i to override.

[COLOR="Blue"] sudo fdisk -l[/COLOR]

***Disk /dev/hda: 160.0 GB, 160041885696 bytes
***137 heads, 8 sectors/track, 285202 cylinders
***Units = cylinders of 1096 * 512 = 561152 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/hda1               1      285202   156290692   83  Linux

***Disk /dev/sda: 37.0 GB, 37019566080 bytes
***255 heads, 63 sectors/track, 4500 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/sda1   *           1        4314    34652173+  83  Linux
***/dev/sda2            4315        4500     1494045    5  Extended
***/dev/sda5            4315        4500     1494013+  82  Linux swap / Solaris

***Disk /dev/sdb: 74.3 GB, 74355769344 bytes
***255 heads, 63 sectors/track, 9039 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/sdb1               1        1200     9638968+  83  Linux
***/dev/sdb3            1201        3000    14458500   83  Linux
***/dev/sdb4            3001        9039    48508267+  83  Linux

***Disk /dev/hde: 160.0 GB, 160041885696 bytes
***255 heads, 63 sectors/track, 19457 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/hde1               1       19457   156288321   83  Linux

***Disk /dev/hdf: 160.0 GB, 160041885696 bytes
***255 heads, 63 sectors/track, 19457 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/hdf1               1       19457   156288321   83  Linux

***Disk /dev/hdg: 160.0 GB, 160041885696 bytes
***255 heads, 63 sectors/track, 19457 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/hdg1               1       19457   156288321   83  Linux

***Disk /dev/hdh: 160.0 GB, 160041885696 bytes
***255 heads, 63 sectors/track, 19457 cylinders
***Units = cylinders of 16065 * 512 = 8225280 bytes

***   Device Boot      Start         End      Blocks   Id  System
***/dev/hdh1               1       19457   156288321   83  Linux

[COLOR="Red"]***Disk /dev/md0: 800.1 GB, 800198295552 bytes
***2 heads, 4 sectors/track, 195360912 cylinders
***Units = cylinders of 8 * 512 = 4096 bytes

***Disk /dev/md0 doesn't contain a valid partition table[/COLOR]
[COLOR="Blue"] sudo mount /dev/md0[/COLOR]
***mount: can't find /dev/md0 in /etc/fstab or /etc/mtab
[COLOR="Blue"] sudo mount /dev/.static/dev/md0[/COLOR]
***mount: can't find /dev/.static/dev/md0 in /etc/fstab or /etc/mtab

[COLOR="Blue"]sudo gedit /etc/mtab[/COLOR]
***(((I added    /dev/md0  /raid  ext3  rw  0))))
[COLOR="Blue"]sudo mount /dev/md0[/COLOR]
***Cannot create link /etc/mtab~
***Perhaps there is a stale lock file?


What do I do to correct the /dev/md0 not having a valid partition table?
 
First, you'll probably not want to use type 83 for md devices; FD is what's generally used. If you were to create a partition with a single Linux filesystem on it, it'd be type 83, but for arrays it's important to distinguish between the array and the members thereof.
caffiendo said:
I used /dev/.static/dev/md0 because it was posted in another HowTo that I used some info from. Why not just use /dev/md0?
Click to expand...
/dev/.static is contained in an initrd, usually. This means that when your system boots, it'll know about the array even before udev starts and automatically makes the /dev entry. If you're using this as your root filesystem, you'll need .static in there, otherwise they're equivalent.
caffiendo said:
WHY won't this start, considering that I made the /etc/mdadm.conf file?
Click to expand...
Beats me. I thought the new raidtools were supposed to preclude config files; a version of that file is stored on each of the raid disks.
caffiendo said:
Finally what do I use to create a file system?
sudo mke2fs -j /dev/md0 or
sudo mke2fs -j /dev/.static/dev/md0 ?
Click to expand...
Either of these should work; "md0" is not a regular file, but an indicator to the kernel that md knows where to get blocks for this device. If you do ls -l /dev/md0 /dev/.static/dev/md0 they should show up much the same (except for the name, of course).
caffiendo said:
I created /raid directory for my array mounting point, then adding this to my fstab
Code: 
# /dev/.static/dev/md0  UUID=54623f8b:b3b44af:45dca40a:d0833744    /raid               ext3    defaults  0    0
I then tried creating the FS with this:
Code: 
[COLOR="Blue"] sudo mke2fs -j /dev/.static/dev/md0[/COLOR]
***mke2fs 1.39 (29-May-2006)
***Filesystem label=
***OS type: Linux
***Block size=4096 (log=2)
***Fragment size=4096 (log=2)
***97681408 inodes, 195360912 blocks
***9768045 blocks (5.00%) reserved for the super user
***First data block=0
***Maximum filesystem blocks=197132288
***5962 block groups
***32768 blocks per group, 32768 fragments per group
***16384 inodes per group
***Superblock backups stored on blocks: 
***        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
***       4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
***        102400000
***Writing inode tables: done                            
***Creating journal (32768 blocks): done
***Writing superblocks and filesystem accounting information: done
***This filesystem will be automatically checked every 24 mounts or
***180 days, whichever comes first.  Use tune2fs -c or -i to override.
Click to expand...
That should be okay.
caffiendo said:
Code: 
[COLOR="Red"]***Disk /dev/md0: 800.1 GB, 800198295552 bytes
***2 heads, 4 sectors/track, 195360912 cylinders
***Units = cylinders of 8 * 512 = 4096 bytes

***Disk /dev/md0 doesn't contain a valid partition table[/COLOR]
Click to expand...
caffiendo said:
What do I do to correct the /dev/md0 not having a valid partition table?
Click to expand...
That should be expected. Since the filesystem is going to sit directly on the md block device, there's no partition table associated with it. You could use LVM if you want multiple independent filesystems on this one array, but if all you want is one big chunk of space, leave it as-is and ignore the missing partition table error.
caffiendo said:
sudo mount /dev/md0
***mount: can't find /dev/md0 in /etc/fstab or /etc/mtab
Click to expand...
Again, expected. You put the .static version of this in fstab.
caffiendo said:
sudo mount /dev/.static/dev/md0
***mount: can't find /dev/.static/dev/md0 in /etc/fstab or /etc/mtab

sudo gedit /etc/mtab
***(((I added /dev/md0 /raid ext3 rw 0))))
sudo mount /dev/md0
***Cannot create link /etc/mtab~
***Perhaps there is a stale lock file?
Click to expand...
You want to look at /etc/fstab. The # you put at the front of the line is a comment start - everything from there to the end of the line is ignored. Remove it and see what happens.

HTH:D
 
Ok, I changed the mtab and fstab and the array is working, thanks for the suggestions!

As for the type of partition I created "83", what should I do about that? The array seems to be working fine, but I would like to re-do the setup the right way. What should I have done when I had to choose a partition type?

Finally, I guess I need to re-do the RAID config. how do I delete the current setup?
 
caffiendo said:
Ok, I changed the mtab and fstab and the array is working, thanks for the suggestions!
Click to expand...
Good deal.
caffiendo said:
As for the type of partition I created "83", what should I do about that? The array seems to be working fine, but I would like to re-do the setup the right way. What should I have done when I had to choose a partition type?

Finally, I guess I need to re-do the RAID config. how do I delete the current setup?
Click to expand...
You can simply change the partition type with fdisk. Nothing relies on the partition type other than the kernel; when it starts up it checks FD-type partitions for software raid arrays. No re-creation should be necessary. Reboot the machine to make sure it's detected properly.

Last but not least - have you considered the implications of reduced reliability caused by raid 0? What's your backup strategy, if your data is important to you? Especially with a 5-disk array, on which one disk failure causes immediate loss of all the data, this is a big concern.
 
Back up is going to be with a pair of WD 5000ks's.

I've changed the partitions types to "fd". do I need to re-make the file system, or change anything else after that?

Thanks again for the help!
 
Nope, should work fine. Glad to hear you've covered the backup issue.
 
FWIW, Ive made a few arrays wtih mdadm, and I never had to edit mdadm.conf by hand.
 
gurgle said:
anyone got a RAID5 tut?
Click to expand...

sure:

follow above tutorial, but create your array with
Code: 
mdamd -C /dev/md<a number> -l5 -n <number of partitions, e.g. 3> <name of partitions, eg. /dev/hd[bcd]2>
 
You must log in or register to reply here.
Back
Top