Building a storage server, choosing controller card?

  • Thread starter Deleted member 12106
  • Start date
D

Deleted member 12106

Guest
I have decided I am going to start to build to a dedicated storage server. I currently use WHS and have 14tb of storage, however, I would like to move to something with less overhead, quicker, and more appropriate.

I am going to use a norco 20 drive case, windows 2008r2(installed on 2.5" drive", and will use either this sempron setup I have or a core2 setup I have.

The part where I am more concerned is the controller/expanders. I am not after break-neck speeds since this is going to be used primary in my house and will stream content to laptop/extenders.

Because I do not have the up-front cash to do this in one shot, I am shopping for the small items first. One of those is the controller.

I want to be able to run a 40 drive array, and have the ability to run another array externally down the road.

One of the cards I am looking at:
http://www.newegg.com/Product/Product.aspx?Item=N82E16816115056&cm_re=4320-_-16-115-056-_-Product
Highpoint RocketRAID 4320

This looks like a unit I can use a sas expander on and run 40 drives in a norco, and also expand externally if needed.

What are other cards I should be on the look out for?
 
It would take a looong time to rebuild when a drive fails. Good luck with that system.
 
danman said:
It would take a looong time to rebuild when a drive fails. Good luck with that system.
Click to expand...

What's a looong time? Couple hours? Couple days, Couple weeks? If it is a few hours or days, I think I can live with it, but if it is a week, then:eek:.

I have a RocketRaid 4320 and Chenbro CK12803 on the way.

Now to decide if I want to go sempron or core2 for the cpu/mobo. I def want a board with expandability in mind.
 
only guessing now... but probably not less than a week, but maybe within two...
 
It's ~6 days for a RAID6 array at 150MB/s. I doubt you could get that rebuild speed on that card. Reviews aren't very favorable for how much it costs. I agree with the rest, 40 drives is a lot for one array.
 
Clearly he is a subscriber to the philosophy of more spindles more iops.
 
No offense, but your guy's answers are fairly vague. I ask what color the sky is, and it appears that it is not green, white, or pink.

Like I had mentioned I am wanting to move away from WHS and would rather not have to break up larger folders across multiple arrays.

So instead of asking questions with no want to help, how about some honest, truthful, and useful information?

I am not trying to be rude, but the lack of useful information is less than enlightening.
 
I'm not familiar with how you would do that using hardware RAID.
You could do it using zfs, flexraid, or linux lvm.

How exactly do you plan to build this? Will you buy the drives one/two at a time and will they all be the same?
 
thefreeaccount said:
I'm not familiar with how you would do that using hardware RAID.
You could do it using zfs, flexraid, or linux lvm.

How exactly do you plan to build this? Will you buy the drives one/two at a time and will they all be the same?
Click to expand...

Not familair with zfs, flexraid, or linux.

My goal is to buy the drives in 1 swoop, if not, 1/2 at a time.

If the general consensus says it is a bad idea to do 20 drives in an array, what about 10 and 10?

I'd like to stick with windows if possible, I am familiar with that and also want to run a VM on top of this.
 
You could make 4 10 drive arrays then merge them all as one in Windows by making them dynamic disks. This way you essentially have one large drive.
 
danman said:
You could make 4 10 drive arrays then merge them all as one in Windows by making them dynamic disks. This way you essentially have one large drive.
Click to expand...

I should have been more clear in the OP. I am going to start with 20 drives and want to expand to a second set if needed.

So let's say I did make 4 10 drive arrays 1 disk and the OS disk craps out or one of the drives craps out, now what?

What drives are the cool kids using these days for raid? I'd like to avoid 7200rpm drives if at all possible.
 
sc0tty8 said:
I should have been more clear in the OP. I am going to start with 20 drives and want to expand to a second set if needed.

So let's say I did make 4 10 drive arrays 1 disk and the OS disk craps out or one of the drives craps out, now what?

What drives are the cool kids using these days for raid? I'd like to avoid 7200rpm drives if at all possible.
Click to expand...

That depends on what kind of array you decide to set up.

If it's a redundant array like RAID5, then if you lose one drive, you can rebuild the array, but as others have said, that would take days. If you lose more than one drive, then you'll lose all of your data and you'll have to restore from a backup.

If you do what danman says and make one large spanned volume for each of the arrays(JBOD), then if you lose one drive, then all of the data on that array will be lost.
 
sc0tty8 said:
Yeah i'm not a fan of the idea of loosing data.
Click to expand...

It is impossible to loose data. Losing data, however, is a possibility. If you are going to make an array, I would buy fast drives (like 600 Gig Raptors) and a very good RAID card with lots of RAM on it.

Most importantly, you need a backup strategy. RAID is not a backup. Twice at work, I have had a SCSI drive fail in my RAID 5. One of those times, another drive failed during the rebuild, killing the array. I would have been screwed if I didn't have another (multiple) backups to restore from.
 
Lord Nassirbannipal said:
That depends on what kind of array you decide to set up.

If it's a redundant array like RAID5, then if you lose one drive, you can rebuild the array, but as others have said, that would take days. If you lose more than one drive, then you'll lose all of your data and you'll have to restore from a backup.

If you do what danman says and make one large spanned volume for each of the arrays(JBOD), then if you lose one drive, then all of the data on that array will be lost.
Click to expand...

No I meant 4 RAID 5 arrays that are spanned.
 
cditty said:
It is impossible to loose data. Losing data, however, is a possibility. If you are going to make an array, I would buy fast drives (like 600 Gig Raptors) and a very good RAID card with lots of RAM on it.

Most importantly, you need a backup strategy. RAID is not a backup. Twice at work, I have had a SCSI drive fail in my RAID 5. One of those times, another drive failed during the rebuild, killing the array. I would have been screwed if I didn't have another (multiple) backups to restore from.
Click to expand...

LOL

Definitely spot on advice.

Snapshots sound like a good idea.

ZFS sounds like it might be a great fit. Plus SSD for l2arc
 
cditty said:
It is impossible to loose data. Losing data, however, is a possibility. If you are going to make an array, I would buy fast drives (like 600 Gig Raptors) and a very good RAID card with lots of RAM on it.

Most importantly, you need a backup strategy. RAID is not a backup. Twice at work, I have had a SCSI drive fail in my RAID 5. One of those times, another drive failed during the rebuild, killing the array. I would have been screwed if I didn't have another (multiple) backups to restore from.
Click to expand...

Raptors seem like a little overkill for storage, I am not after rip-roating speed, more dependable, low powered storage.
 
If you're using a hardware RAID card, people usually use the Hitachi's because they tend to have fewer problems. They are 7200rpm and hot-ish. I don't know which, if any, 5400rpm drives work well with that Highpoint card. I use 5400 drives and mdadm software raid at home, which is plenty fast for me. I mostly use Areca cards at work. I don't do anything over 8 drives with only RAID5. Usually RAID6 + cold spare or a hot spare. It's all about what level of risk you're willing to take.

I agree raptors are definitely overkill for what it seems like you're attempting.

If you want more helpful responses you should give more details. Off the top of my head: budget, initial space, what types of files, how many concurrent users, acceptable speeds, acceptable rebuild times, level of risk, ... what have you considered and dismissed because it doesn't meet specs?
 
I'm thinking raid 5 or 6, budget, open, looking to spend 3k total or less. Would like at least 15tb usable after format/raid config(10 drives), files are isos and mkv's, some music. Concurrent users, no more than 5-10. I'd like to be able to steam+read+write files form multiple clients, rebuild times, open. Level of rish, whatever WHS is.

WHS I am wanting to move away from because its slow, Other than the disk extender, I don't use the features. I want to move away from it because unless I have my folders set for duplication, there is no redundancy. I have external drives to do backups.

I like the idea of using something windows server 2008 r2, so I can use hyper-v to host a small web server.

So...thoughts? I'd like to start with at least 10 drives and go from there.
 
With that much storage space (assuming you're still going with the 20 drive plan first), I would recommend going with RAID 6 with two hot-spares enabled so that you can handle a two drive loss quickly and effectively. That card and the expander you bought should be fine for that. That'll still leave you with 32TB of storage space if you use 2TB drives.
 
tgrimley said:
It's ~6 days for a RAID6 array at 150MB/s. I doubt you could get that rebuild speed on that card. Reviews aren't very favorable for how much it costs. I agree with the rest, 40 drives is a lot for one array.
Click to expand...

That card is dual core IOP based, I think chances are it will perform quite well.
 
danman said:
You plan on running 40 drives in a single array?
Click to expand...

Holy smokes, I hope it doesn't break! That would take a few hundred hours to repair under RAID5/6. :eek:

Might I suggest that software RAID be used as long as your CPU is dual core, and have maybe 4 RAID arrays, rather than 1 huge one?

It wouldn't be as bad, imo, if you were using 2TB disks, but with 40 disks and sub 15TB of storage, that's a dangerous road to travel. ;)
 
Danny Bui said:
With that much storage space (assuming you're still going with the 20 drive plan first), I would recommend going with RAID 6 with two hot-spares enabled so that you can handle a two drive loss quickly and effectively. That card and the expander you bought should be fine for that. That'll still leave you with 32TB of storage space if you use 2TB drives.
Click to expand...

This would work great. Make sure you have the two (or more) hot spares configured correctly. It may not be a backup, but it's a great fail-safe.

Also, I know you want to use 2008R2 (much better than WHS, imo) but it has a ton of overhead compared to Linux.

If you use RAID on it, make sure it is software or hardware RAID, not fakeRAID. This way if you ever move to Linux or a non-Windows OS, you can transfer the data. As of right now Windows is the only OS to support fakeRAID (since it is driver-based).

I know you don't know Linux, but trust me, it is sooooo much better than Server 2003/2008/R2. In the system in my sig, I just moved from both Server 2003R2 and 2008. 2003 was fast enough, but it didn't support the apps I needed. 2008 was Vista wrapped in a silver blanket, the OS was very sluggish and slow and network speeds suffered, even on gigabit. Using Ubuntu, I get file transfers up to 56MB/s, even on my crappy (regular) Cat5 cable. I will be adding more disks to this machine soon and will probably be doing the same thing you are doing.

Unless you absolutely need Server 2008R2 for specific functions that are Windows-only, I'd definitely recommend you give Ubuntu Linux a try. I'm still kind of a noob at it, but considering all that I can do with it and the performance increases I've seen using the same hardware, I can say it's worth giving a try. ;)
 
Last edited:
Red Falcon said:
This would work great. Make sure you have the two (or more) hot spares configured correctly. It may not be a backup, but it's a great fail-safe.

Also, I know you want to use 2008R2 (much better than WHS, imo) but it has a ton of overhead compared to Linux.

If you use RAID on it, make sure it is software or hardware RAID, not fakeRAID. This way if you ever move to Linux or a non-Windows OS, you can transfer the data. As of right now Windows is the only OS to support fakeRAID (since it is driver-based).

I know you don't know Linux, but trust me, it is sooooo much better than Server 2003/2008/R2. In the system in my sig, I just moved from both Server 2003R2 and 2008. 2003 was fast enough, but it didn't support the apps I needed. 2008 was Vista wrapped in a silver blanket, the OS was very sluggish and slow and network speeds suffered, even on gigabit. Using Ubuntu, I get file transfers up to 56MB/s, even on my crappy (regular) Cat5 cable. I will be adding more disks to this machine soon and will probably be doing the same thing you are doing.

Unless you absolutely need Server 2008R2 for specific functions that are Windows-only, I'd definitely recommend you give Ubuntu Linux a try. I'm still kind of a noob at it, but considering all that I can do with it and the performance increases I've seen using the same hardware, I can say it's worth giving a try. ;)
Click to expand...

I plan on running win2k8 r2 as I need to learn it more and become more acquainted with it. It is a lot less bloated like win2k8. R2 is basically the server version of win7 in terms of bloat-ness.

Don't get me wrong, Linux is a great OS, and about into as I get is running Untangle UTM, www.untangle.com This is a great router/utm appliance that works very well, it is based on debian(like ubuntu) and it just works. I have used ubuntu a little bit as a web server, the overhead is minimal, but I do not miss the days of dos and don't care for the CLI as much.
 
Awesome, well good luck with 2008 R2. I can't wait to upgrade our severs at my job to it. You are right, it is a LOT better than 2008. If only we could move everything to Linux. ;)
 
Red Falcon said:
If you use RAID on it, make sure it is software or hardware RAID, not fakeRAID. This way if you ever move to Linux or a non-Windows OS, you can transfer the data. As of right now Windows is the only OS to support fakeRAID (since it is driver-based).
Click to expand...
Just to comment on this; virtually all FakeRAIDs (meaning onboard RAID, which IS software RAID) are supported by Linux and BSD. The metadata formats of a dozen brands are detected. So you can recover/access any windows RAID using simple Ubuntu livecd. This does not require installation of anything, you can browse your RAID contents (NTFS) right away.

The reason for this is simple: all RAIDs use metasector (last sector on each HDD) to store array information. If you teach a software driver how a dozen brands are doing it, then it supports all those configs and can detect a 'fakeRAID' without problems.
 
OP:

  • I'm using 8 of the 2TB Seagate LP drives on an Adaptec 5805 and a LSI SASX28 based Supermicro expander backplane without any issues (and I have an additional 2 drives arriving tomorrow).
  • For 40 drives, I'd definitely recommend multiple arrays and at least 2 or more global hotspares. As other people have said, your rebuild times will be pretty horrendous with 40 in a single volume. I'd guesstimate that 16 drives in a R6 would be about a one day rebuild time assuming the array is not being accessed by users.
  • In Windows, you can always mount additional logical drives to different directories, thereby not having using additional drive letters.

Krobar said:
That card is dual core IOP based, I think chances are it will perform quite well.
Click to expand...

IOP348, IIRC. Same as the Adaptec 5xxx series and some Areca cards. I evaled some Highpoint cards here at work and they performed roughly par with same-class Adaptec cards (give or take a bit depending on which type of test).

sub.mesa said:
Just to comment on this; virtually all FakeRAIDs (meaning onboard RAID, which IS software RAID) are supported by Linux and BSD. The metadata formats of a dozen brands are detected. So you can recover/access any windows RAID using simple Ubuntu livecd. This does not require installation of anything, you can browse your RAID contents (NTFS) right away.

The reason for this is simple: all RAIDs use metasector (last sector on each HDD) to store array information. If you teach a software driver how a dozen brands are doing it, then it supports all those configs and can detect a 'fakeRAID' without problems.
Click to expand...

Also worth nothing:

dmraid in linux is compatible with RAID1/0 arrays built on many fakeraid codebases, however support for RAID 5 arrays on fakeraid codebases is rather sketchy or non-existant depending on which controller your board has. Additionally, to my knowledge, there is no monitoring/management support for dmraid managed arrays in linux.
 
sub.mesa said:
Just to comment on this; virtually all FakeRAIDs (meaning onboard RAID, which IS software RAID) are supported by Linux and BSD. The metadata formats of a dozen brands are detected. So you can recover/access any windows RAID using simple Ubuntu livecd. This does not require installation of anything, you can browse your RAID contents (NTFS) right away.

The reason for this is simple: all RAIDs use metasector (last sector on each HDD) to store array information. If you teach a software driver how a dozen brands are doing it, then it supports all those configs and can detect a 'fakeRAID' without problems.
Click to expand...

You are right in one sense, but wrong in another. fakeRAID uses drivers to tell the RAID controller (with special firmware/BIOS) to run the RAID array. This does use the CPU, but this is not software RAID. Software RAID does not require any driver (thus any OS will run it) and is done through algorithms on CPU without the need of any driver.

Yeah, I suppose the driver could be edited to accept any RAID controller, but it is a huge pain to try and convert the drivers to function with Linux or Mac OS X.

Hence why I recommended using software RAID instead of fakeRAID. I'm not saying one is better than the other, though I am saying software RAID is far more backwards compatible. But that is a decision that is up to the OP.
 
Last edited:
You must log in or register to reply here.
Back
Top