3TB+ storage system questions.

[God_Of_Death]

ok, i currently have 13 drives in JBOD config. i would like to build a back up server using 12 sata drives in raid-0. i still have not decided if i want to go with 250gigs or 320gigs. either way i will have the 12 drives in raid-0 and that will go over 2tb once formated. i don't want to make any partitions. i just want to have one big drive ^_^

i just got through reading a thread that mentions that i may have to do some windows tweaks in order to get windows to recognize a 2TB+ partition.

this will be used as a storage/backup. i would have my OS and progs in another raid setup.

iirc, windows can not recognize a drive larger than 2tb's if it's a basic drive and if it has a MBR. if it's dynamic then windows can see up to 16EB.

i will be using windows xp SP2. i still have not decided what raid card to use but i have a fair idea. i just need to know if i will encounter problems in windows when it comes time to formating 3TB's worth of space.

 

[tdg]

I must say that 12 drives in a single RAID0 array is a terrible idea, why not look into RAID5?

 

[defakto]

I must say that 12 drives in a single RAID0 array is a terrible idea, why not look into RAID5?

Quoted for truth.

For a backup server that is a very, very bad idea.

 

[God_Of_Death]

could you refresh my memory as to what raid 5 does. my memory stops at raid-1

 

[defakto]

Raid 5 rotates parity data between the drives so any single drive can fail and you can still rebuild the array.

Also, windows can recognize drives larger than 2tb, what you run into is some file systems will not allow partitions larger than 2tb.*edit* wait, not so sure on that I'll do some research for my own knowledge before I back that up.

 

[God_Of_Death]

Quoted for truth.

For a backup server that is a very, very bad idea.

it's for home use. i am not all that worried about redundantcy. this setup will just be used to hold backup files. if an hd dies i would just replace the hd, remake the arry, reformat, and then copy all of the files back from my main setup.

 

[defakto]

I understand that, but you're looking at quite a considerable amount of time to put in when something goes wrong. Wouldn't it be much easier and prefered to pull the bad drive, start the system up and let the array rebuild while you still have access to your files? You're going to spend a pretty hefty amount of cash on a raid card that can handle 12 drives as it is, might as well put some of those expensive features to use.

 

[God_Of_Death]

lol, i see i have gotten myself into something. i have always persoanally used JBOD setups. raid-1 if i feel paraniod enough. i thought raid-0 would be the simplest way to go other than JBOD.

what are the CONS of raid-5? how much of my total space will have to give up?

 

[defakto]

Raid 5 typically has lower write speeds, depends on the controller, which is something you will definately want to research. Raid 5 read speeds on a healthy array are pretty close to raid 0 read speeds. On an array with a failed drive it varies in performance from controller to controller.

For space available you take get (n-1)*size where N is the number of drives. The more drives you put in a raid 5 the less percentage of space you lose to party data. Or the percentage is (1/n)*100 where N is the number of drives. Basically you lose 1 drive worth of space. So if you use all 250, you lose 250. 300's you lose 300.

 

[God_Of_Death]

Raid 5 typically has lower write speeds, depends on the controller, which is something you will definately want to research. Raid 5 read speeds on a healthy array are pretty close to raid 0 read speeds. On an array with a failed drive it varies in performance from controller to controller.

For space available you take get (n-1)*size where N is the number of drives. The more drives you put in a raid 5 the less percentage of space you lose to party data.

i see. so in raid-5 (12x320) - 320 = 3520gigs of actual space before formating. right?

 

[defakto]

i see. so in raid-5 (12x320) - 320 = 3520gigs of actual space before formating. right?

Correct.

 

[God_Of_Death]

Correct.

i can live with that

ok, when it will comw time to formating it. will windows, using NTFS and sp2, have any problems seeing it and formating it or will i have to buy 3rd party software?

basically, will it be as simple as just adding a new hd, going into disk management, selecting the new drive and formating it?

 

[LBFlashWiz]

If you have 12 drives, Raid 5 can be a little more risky thank if you just had 3-8.

Raid 6 would give you a safer solution than the RAID 5 without loosing much space for the security.

Try this RAID card out for the 12 drives HERE

Also, if you are worrying about performance, the bottleneck in a system like this would be the PCI bus. You might want to get a PCI-X MB for full speed.

 

[Drudenhaus]

If you have 12 drives, Raid 5 can be a little more risky thank if you just had 3-8.

Raid 6 would give you a safer solution than the RAID 5 without loosing much space for the security.

Try this RAID card out for the 12 drives HERE

Also, if you are worrying about performance, the bottleneck in a system like this would be the PCI bus. You might want to get a PCI-X MB for full speed.

Just FYI...that card is around $1000 (USD)...

 

[defakto]

i can live with that

ok, when it will comw time to formating it. will windows, using NTFS and sp2, have any problems seeing it and formating it or will i have to buy 3rd party software?

basically, will it be as simple as just adding a new hd, going into disk management, selecting the new drive and formating it?

If you're using it as a boot, you'd need SP2, but it doesn't sound like you are. While I've never messed with filesystems that big it should be just as easy as you described it. Though it may take a long time for the card to build the initial array. Sometimes more than 8 hours if it's large enough.

 

[God_Of_Death]

If you're using it as a boot, you'd need SP2, but it doesn't sound like you are. While I've never messed with filesystems that big it should be just as easy as you described it. Though it may take a long time for the card to build the initial array. Sometimes more than 8 hours if it's large enough.

not as boot. just for storage.

 

[Asgorath]

For a RAID 5 card, I would go with 2x 8 drive cards. The big big cards are super expensive, and I know at least 3ware cards allow you to have multiple cards look at the same array.

Example:

15 drive RAID 5 of 300Gig drives = 4.2 TB

Card 1: 8 drives
Card 2: 7 drives

Later on you want to build a bigger array. So you say, I want 5 more drives. Then you need to buy the drives and another card. Then use the expanding feature of this high end card.

So now you have 20 drive RAID 5 of 300Gig drives = 5.7 TB

Card 1: 7 drives
Card 2: 7 drives
Card 3: 6 drives

To actually get a decent throughouput out of these crazy arrays you're going to want a fast bus such as PCI-X, PCI-Express 4x, or 64bit PCI. Not to mention the crazy ass power supply you're going to need to power this thing. Oh, and another good thing about high quality RAID cards is they support staggered spinup, which means when you first turn your computer on the drives take turns spinning up (most power hungry part) which means you can go a little easier on the power supply.

Check with the mfg to be sure your RAID card you choose supports stacking of the cards. Promise supports 2 card stacking. 3ware told me it is unlimited.

 

[unhappy_mage]

For a RAID 5 card, I would go with 2x 8 drive cards. The big big cards are super expensive, and I know at least 3ware cards allow you to have multiple cards look at the same array.
...
To actually get a decent throughouput out of these crazy arrays you're going to want a fast bus such as PCI-X, PCI-Express 4x, or 64bit PCI.

1) 2x8-way 3ware cards (I assume the 9500S-8) = 2x$500 = $1000
versus
1x16-way Areca card = $999 from http://www.topmicrousa.com
and the Areca has raid 6, ethernet management, capacity expansion, raid level migration, multi-card array support, battery backup,... Where's the benefit of the 3ware?

2) PCI-X == 64-bit pci (well, technically not quite, but for most cases.) PCI-X means pci at a higher rate or wider signals. So a 32/66 card is pci-x even though it's not 64-bit.

 

[DougLite]

Disks with a conventional Master Boot Record (MBR) cannot have partitions larger than 2TB. Using Dynamic Disks in Windows 2000 or XP will circumvent this limitation, as will GUID disks in 2003. I'm sure it is also possible to defeat this limitation in Linux, but unhappy_mage or some other Linux wizard will have to comment there.

 

[defakto]

1)

2) PCI-X == 64-bit pci (well, technically not quite, but for most cases.) PCI-X means pci at a higher rate or wider signals. So a 32/66 card is pci-x even though it's not 64-bit.

32bit/66mghx slots are not pci-x. Pci-x starts at 64 bit/66 mghz where regular pci left off and also adds, if i recall correctly, some additional instruction sets to the protocol that can be used by pci-x cards on pci-x controllers.

 

[unhappy_mage]

ftp://ftp.compaq.com/pub/supportinformation/papers/tc990903tb.pdf
On page 5 it discusses this (you're right); learn something new every day, I guess.

http://www.suse.de/~aj/linux_lfs.html
Linux large filesystem support. Short answer, use a modern distro and it Just Works (tm). XFS is designed for this (where this equals really big filesystems), use it.

XFS is a full 64-bit filesystem, and thus is capable of handling files as large as a million terabytes.

2^63 = 9 x 10^18 = 9 exabytes

Not to mention all the cool concurrency stuff they've got going. It's an all-around win, especially with multi-processor multi-adapter multi-pci-bus machines.

As for how the partition table works, I have no idea. This thread from the g2 forums talks about it, but it's from April, so I don't know if it's relevant. This just says we need new partitioning schemes, not what they are. I don't have 2tb myself, but if someone is willing to donate 5 more 300gb disks I'll be glad to test and report back.