Need input on my plan for a Linux iSCSI setup

[LightningCrash]

The storage needs for my house are beginning to get pretty large. With 4 computer geeks in the house and a 12mbps cable connection, we're needing more and more storage than ever. I put an additional 320GB drive into my dedicated SMB box and it was full within a few days.

My plan is to use Linux to set up a massive array, utilizing iSCSI targets. Some volumes will also not be iSCSI targets but will be shared storage exported via NFS or SMB.

The storage will serve a couple of purposes:
-PXE booting a HTPC, making it HDD-less. With the right CPU/MOBO/PSU, it will be completely silent. Also provides plenty of storage for MythTV recordings/DVD Rips
-Having a shared storage arena so we don't download the same things 4 times (my roomies and I recently discovered that all four of us had downloaded the same Ubuntu ISO within a relatively brief period of time. Talk about a waste of bandwidth)
-Having expandable, redundant, mass, cheap storage without having to worry about upgrading each of our machines
-Extraordinarily large ePenis


As of right now, I'm looking at the following
Pentium D 820
ABIT AB9 Mobo
2GB Kingston DDR2-667
ASUS GF6200TC cheapie
Cheapie 4U Chassis
CoolerMaster RS-500 500W PSU
1ea HighPoint RocketRaid 2322
8ea Seagate 7200.10 320GB SATA drives
2ea SAS to Infiniband Connector

1 of these:
http://www.pc-pitstop.com/sata_enclosures/scsat84x.asp



Of course I'm going to upgrade my network to gigabit and run Cat6
So I'll have to get a gigabit switch. I'm looking at a couple of Linksys managed GigE switches.

Total cost looks to be about $2500 right now to bring us up to 2.2TB, with the ability to increase the array by another 2.2TB later for $1500 or less.

I've seen some products to PXE boot Windows XP from iSCSI, which is a very intriguing thought. Linux already has iSCSI PXE boot support. I think it would be really neat to be able to try out new OSes/Distros on the iSCSI volumes without having to worry about what happens to local disks. Has anyone had experience with PXE booting a box to an iSCSI volume?

I know the Highpoint isn't true hardware RAID, but I don't need balls-to-the wall performance anyway. I've seen benchies that pretty well max out the controller/disks at only 5% CPU utilization on a 3Ghz Pentium 4, so I'm sure an 820D would work fine.

I'm trying to find a board with 3ea 4x PCI-E capable slots, no luck so far though
I want this thing to be 6.6TB by this time next year, so I'll need 3ea 4x+ PCI-E slots. (Or else I'll have to build another box and use iSCSI to bring the disks over to the first box, I guess)

Thoughts/Comments/Suggestions?
Anything I'm missing?

 

[Dually]

Which iSCSI target are you going to use?

 

[unhappy_mage]

Why iScsi? Sure, you can boot XP over it, but that's all the advantage you get - Linux PXE works just fine over NFS, and SMB/NFS allow multiple users concurrent access to the same data. Unless you were to export a read-only NTFS filesystem, you can't mount it more than once. Remember, this would show up like a block device. And since your goal is storage cosolidation, you want to be able to use it from multiple machines read/write. Unless an implementation of GFS or OCFS2 for Windows appears, you can't have multiple filesystem drivers accessing the same block device at the same time. Perhaps make a small block device to boot XP off of, but you'll want most of your storage accessible via SMB.

My brother built a similar machine - 8*320 7200.10s, a nice Supermicro case with power supply, a dual p3 board we had lying around, and some other stuff. Spend more money on the power supply - if you use a 4u case, you might as well put drives in it, and you'll want something that'll handle 15 drives eventually. Buying small power supplies is for wimps

 

[LightningCrash]

Which iSCSI target are you going to use?

I've been testing a couple of versions on a Linux box I have downstairs. Any recommendations?

Why iScsi? Sure, you can boot XP over it, but that's all the advantage you get - Linux PXE works just fine over NFS, and SMB/NFS allow multiple users concurrent access to the same data. Unless you were to export a read-only NTFS filesystem, you can't mount it more than once. Remember, this would show up like a block device. And since your goal is storage cosolidation, you want to be able to use it from multiple machines read/write. Unless an implementation of GFS or OCFS2 for Windows appears, you can't have multiple filesystem drivers accessing the same block device at the same time. Perhaps make a small block device to boot XP off of, but you'll want most of your storage accessible via SMB.

My brother built a similar machine - 8*320 7200.10s, a nice Supermicro case with power supply, a dual p3 board we had lying around, and some other stuff. Spend more money on the power supply - if you use a 4u case, you might as well put drives in it, and you'll want something that'll handle 15 drives eventually. Buying small power supplies is for wimps

right, i do want most of my storage SMB/NFS accessible. i'd just like to be able to take drive purchasing out of my build decisions from here on out. sure, maybe i could throw in a small local disk for swap/paging on the heavy duty desktops, but i've got plenty of spare disks that would be fine for that.

as for the PSU, there's a PSU in the external enclosure that should be capable. if it isn't, i'll drop in a beefy Sun or PC P&C unit. i know that most electronics failures are caused by power supplies somehow, so i always try to buy quality components

 

[LightningCrash]

bump

any other input?

 

[LightningCrash]

many thanks to kirbyrj for this response:

Do you have a budget?

You could always get something like the Asus P5W64 board. It has 4 x16 slots which you could use with a P-D and RAID controllers.

http://www.newegg.com/Product/Product.asp?Item=N82E16813131047

:drool:

 

[tacticus]

http://www.newegg.com/Product/Product.asp?Item=N82E16816115031 and 2 nice big stacker cases is what i am planning

though it wont really be rack mountable
but combined with a board with at least 2 x8 and 1 x4 slot you should get 32 drives and a good dual gig intel nic in there with easy

though it will probably be a bit more work than setting up those external drive boxes

something like http://www.auspcmarket.com.au/show_product_info.php?input[product_code]=CA-CSPC-41416AB&input[category_id]=1252
may work as well though it is a bit pricey (sorry can't seem to find that case in the .US)

 

[unhappy_mage]

Fixed link. vB doesn't like brackets.

I'd recommend this over that. 16 bays in 3u instead of 4, lets you use SAS boot disks, and has expanders. So you get a card that has a wide-port and you have simplified cabling quite a bit. Comes with 800W redundant power supply. I want one for Christmas, but that's not happening

 

[drizzt81]

Fixed link. vB doesn't like brackets.

I'd recommend this over that. 16 bays in 3u instead of 4, lets you use SAS boot disks, and has expanders. So you get a card that has a wide-port and you have simplified cabling quite a bit. Comes with 800W redundant power supply. I want one for Christmas, but that's not happening

You ought to get banned for posting pron links like that!

 

[unhappy_mage]

"Report Post" is right there, dude

 

[Farva]

Fixed link. vB doesn't like brackets.

I'd recommend this over that. 16 bays in 3u instead of 4, lets you use SAS boot disks, and has expanders. So you get a card that has a wide-port and you have simplified cabling quite a bit. Comes with 800W redundant power supply. I want one for Christmas, but that's not happening

You ought to get banned for posting pron links like that!

Yes, that is pretty [H]ardcore pron. I don't know if I could spend 1k on a case though.

 

[unhappy_mage]

Yes, that is pretty [H]ardcore pron. I don't know if I could spend 1k on a case though.

It's not *just* a case. Think of it as a case, 4 4-in-3 cages, and a power supply. The case might be $200 elsewhere, 4-in-3s are usually $120 each, and $300 for that power supply isn't unreasonable. Total so far $980; throw in a $50 assembly fee and a 10% markup so they can keep making them and you get pretty close to what it actually costs.

My brother bought one of the tower cases they have there - 8 hotswap bays, 645W psu, 4u instead of 3. It's really nice to work in - nice and roomy, and very enterprisey We added 5 more bays via a 5-in-3, and it's dealing with its 12 drives quite well. Power is stable, and the drives are actually cool to the touch. All air inputs are filtered, so all that ends up inside the case is dust so fine it feels like grease. It's not terribly noisy, either - I mean, it isn't exactly the quietest machine I've ever used, but it's not full-on server room loud by any means.

Can you tell I like these cases?

 

[Farva]

It's not *just* a case. Think of it as a case, 4 4-in-3 cages, and a power supply. The case might be $200 elsewhere, 4-in-3s are usually $120 each, and $300 for that power supply isn't unreasonable. Total so far $980; throw in a $50 assembly fee and a 10% markup so they can keep making them and you get pretty close to what it actually costs.

My brother bought one of the tower cases they have there - 8 hotswap bays, 645W psu, 4u instead of 3. It's really nice to work in - nice and roomy, and very enterprisey We added 5 more bays via a 5-in-3, and it's dealing with its 12 drives quite well. Power is stable, and the drives are actually cool to the touch. All air inputs are filtered, so all that ends up inside the case is dust so fine it feels like grease. It's not terribly noisy, either - I mean, it isn't exactly the quietest machine I've ever used, but it's not full-on server room loud by any means.

Can you tell I like these cases?

Nope, not at all

So I wasn't taking into account all the hardware that went into the case by just simply looking at it briefly. So sue me

My address is:

123 Fake St.
North Pole, 12345