Project: Galaxy 5.0

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
The time has commeth.


Please refer to the previous galaxy projects found here

Galaxy 4.5: (reached 16tb)
dsc06153tn6.jpg


Galaxy 4.0: (reached 12tb)
dsc05664bg7.jpg


Galaxy 3.0: (reached 9tb)
dsc045104ia.jpg


Galaxy 2.5: (Reached 5.19tb)
dsc038402vr.jpg


Galaxy 2.0: (reached 3.5tb)
dsc034803md.jpg


Galaxy 1.0: (System on far left) (Reached 2tb)
dsc028889vr.jpg




What determines the name of the project?
Good question. If a system experiences drastic physical changes, such as case, motherboard, or entire configuration, then it's a new x.0 level. If the system recieves minor changes/upgrades or just a drive configuration update, then it's a x.5 level update. Smaller upgrades are not counted for.


The goals of galaxy 5.0:

- To increase storage capacity (duh)
- To increase performance
- To increase functionality
- To modernize it
- To transition to the next phase
- Prep it for another project that is to come


This build will be a little different than the previous builds. There will be no custom case action going on, and there are no phases of build... this is a one shot deal.


The whole underlying goal is to prep this machine into a performance beast, so when I am ready, I can go for the dream case, a 50 bay rackmount 5u system. If I jumped from 4.5 to such a massive case, I need to basically replace everything, which would be very cost prohibitive.

The overall goal is 24tb right from the start, when I populate this space, I will then do a simple swap for a larger system configuration, which would be the most cost effective.


This is the build parts:

Supermicro X7DWN+ Dual 771 Intel 5400 Seaburg Extended ATX Server Motherboard
Supermicro SC846TQ-R900B 24 Hot Swap SATAII Chassis (dual 900w redundant HE PSU's)
Supermicro MCP-220-84601-0N HDD-add on module for internal mounting
Supermicro AOC-SIMSO(+) IPMI Remote lights out management
Supermicro CBL-0177L Cable Extension for IPMI
Supermicro CSE-M14TB 4-in-1 Mini SAS Hot-Swap Unit
Two Intel Xeon E5410's Harpertown 2.33GHz 12MB L2 Cache Socket 771 Quad-Core (8 cores total)
Areca 1280-ML 24 Port PCI-E SATAII Raid Controller with Mini SAS Breakout cables
Areca 1280ML 2GB Cache Memory Upgrade Kit
Adaptec 3405 4 Port SAS Controller PCI-E with Mini SAS cables
Adaptec 3405 Battery Backup Option Kit
Adaptec 3405 256MB Cache Upgrade Kit
16 x Wintec 2GB DDRII FB-DIMM DDRII-667 Memory Modules (total of 32GB Ram)
Microsoft Windows 2003 Server
4 x Seagate 15K Savvio 2.5" 73.4GB SAS Drives
24 x Western Digital Caviar GP WD10EACS 1TB Drives
Intel PCI-E PRO/1000 PT Quad Port Server Adapter
Quanta Storage Slim DVD-RW slot load Drive
Slim Optical Drive Adapter to IDE





This is a picture of the case. I like the functionality, I like the slim drive and internal drive areas, and I'm a fan of Supermicro in general. Like I said before, the main concept will be pure performance.

rackmount.jpg

All credit goes to d3vy for this picture.
 

Garr3T

n00b
Joined
Dec 10, 2007
Messages
21
Any reason your going with 2k3 server, hate to sound like a Linux fanboy (I'm Red Hat Certified tho) but ive always seen the LVM file system in Linux to be a bit more robust
 

Farva

Extremely [H]
Joined
Feb 3, 2004
Messages
38,155
And you said Galaxy 5.0 wouldn't start for a very long time (a few months later now). :p
 

swatbat

[H]F Junkie
Joined
Apr 25, 2001
Messages
12,991
Where does the slimline drive go in that case?

Yea thats a good question.

Anyway good plan so far(other then the fact you should run a raid 6 with that and just loose 2 drives worth of space).

If you plan to run it as a dc or anything for your house then yea at least run the os drives in a raid 1.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
And you said Galaxy 5.0 wouldn't start for a very long time (a few months later now). :p


Psshhh. I got a little excited when I saw the prices drop on the rackmounts. However, this is going to depend on the sale of the other galaxy parts.

Where does the slimline drive go in that case?

Next to the PSU in the back.

Yea thats a good question.

Anyway good plan so far(other then the fact you should run a raid 6 with that and just loose 2 drives worth of space).

If you plan to run it as a dc or anything for your house then yea at least run the os drives in a raid 1.

I was thinking OS drives in Raid 1 regardless just so I can have a little more comfort :)
 

unhappy_mage

[H]ard|DCer of the Month - October 2005
Joined
Jun 29, 2004
Messages
11,455
May I recommend another alternative? The Xtore XJ 2000 holds 48 drives in 4U, is connected by a pair of SAS cables, for 2+ gigabytes of read/write capacity. It's expensive as hell - over $5k - but when you need even more storage you can just buy another one and plug it in. You could even daisy chain up to the limits of SAS, which is something huge like 60k drives. You'd need a SAS controller, but you could still use sata drives. You could even use a relatively small machine as the head, if you like, based on a 12x10 board.

Just a thought.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
May I recommend another alternative? The Xtore XJ 2000 holds 48 drives in 4U, is connected by a pair of SAS cables, for 2+ gigabytes of read/write capacity. It's expensive as hell - over $5k - but when you need even more storage you can just buy another one and plug it in. You could even daisy chain up to the limits of SAS, which is something huge like 60k drives. You'd need a SAS controller, but you could still use sata drives. You could even use a relatively small machine as the head, if you like, based on a 12x10 board.

Just a thought.


That is definitely cool. We use a very similar configuration here at work... although a lot bigger (diff brand) :) Anyways, the cost is basically what would scare me, including the portability. But yeah, 48 drives in 4u is awesome.
 

Rob Black

2[H]4U
Joined
Mar 11, 2005
Messages
2,259
What does the Intel PWLA8494GTBLK 10/ 100/ 1000Mbps PCI-X PRO/1000 GT Quad Port Server Adapter card do?

Purpose?
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
What does the Intel PWLA8494GTBLK 10/ 100/ 1000Mbps PCI-X PRO/1000 GT Quad Port Server Adapter card do?

Purpose?

It's a 4 port gigabit NIC. You can team the ports together alongside the motherboard ports and you can have 6gbps on tap. Or you can use it for various other functions.
 

unhappy_mage

[H]ard|DCer of the Month - October 2005
Joined
Jun 29, 2004
Messages
11,455
Mage, quite giving him ideas. :p

No :p

Are you going to swap to RE drives? I'd really recommend it - several people with smaller arrays have lost them because of the TLER bug. You could also try changing the firmware settings to enable TLER, but that utility is for RE (-1) drives. I don't know if it works for RE-GP drives, although it does for RE-2s, which is a hopeful sign.

What RAID configuration are you thinking of? One Big Array is probably not the way to go, especially with no backups. Considered buying a tape drive?

Edit: LOL at the WD-GP promotion page:
Scenarios are based on standard applications within the industry and do not represent actual company data.
 

AMD_RULES

2[H]4U
Joined
Mar 26, 2007
Messages
3,010
this is gonna be freakin insane. What size rack you putting this in?
Do you even have 24TB of data?? That's a lot of stuff :)
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
No :p

Are you going to swap to RE drives? I'd really recommend it - several people with smaller arrays have lost them because of the TLER bug. You could also try changing the firmware settings to enable TLER, but that utility is for RE (-1) drives. I don't know if it works for RE-GP drives, although it does for RE-2s, which is a hopeful sign.

What RAID configuration are you thinking of? One Big Array is probably not the way to go, especially with no backups. Considered buying a tape drive?

Edit: LOL at the WD-GP promotion page:

I'm not swaping drives, I'm not going raid either unless if it's in regards to the system drives (raid 1). The TLER issue with the non RE hard drives have scared me away from RAID possibilities.

this is gonna be freakin insane. What size rack you putting this in?
Do you even have 24TB of data?? That's a lot of stuff :)

No rack planned as of yet, I have some ideas, but basically I enjoy rack equipment for the fact that they are more space saving and quite robust. If I do a rack system, that will be detailed in the other project idea I have.

To give you an idea of current data usage, I'm out of space currently, I have no duplicate files, and no backups other than a few critical files which are on BD and DVD. So the extra space will be much appreciated.
 

AMD_RULES

2[H]4U
Joined
Mar 26, 2007
Messages
3,010
No rack planned as of yet, I have some ideas, but basically I enjoy rack equipment for the fact that they are more space saving and quite robust. If I do a rack system, that will be detailed in the other project idea I have.

To give you an idea of current data usage, I'm out of space currently, I have no duplicate files, and no backups other than a few critical files which are on BD and DVD. So the extra space will be much appreciated.
Man, that's insane. Well i cant wait to see how this all comes out, good luck :)
So what do you use all that space for?
(see above quote)
 

pc299

Supreme [H]ardness
Joined
Jul 2, 2006
Messages
4,523
I say go all out, sell your 16 existing drives and get some Seagate 7200.11 or ES series drives and the ARC-1280ML it's getting more and more painful to lose a drive as capacities increase.

I just completed a rebuild of my 4U with 16x500GB 7200.9's and an ARC-1160ML2. I'm running *nix on it, but I did install 2003 and ran a bench or two on the array before filling it up:

areca.png


Not too shabby for PCI-X based RAID6 :D
 

AMD_RULES

2[H]4U
Joined
Mar 26, 2007
Messages
3,010
I say go all out, sell your 16 existing drives and get some Seagate 7200.11 or ES series drives and the ARC-1280ML it's getting more and more painful to lose a drive as capacities increase.

I just completed a rebuild of my 4U with 16x500GB 7200.9's and an ARC-1160ML2. I'm running *nix on it, but I did install 2003 and ran a bench or two on the array before filling it up:

areca.png


Not too shabby for PCI-X based RAID6 :D
awesome :)
 

gjvrieze

[H]ard|Gawd
Joined
Sep 8, 2006
Messages
1,423
I say go all out, sell your 16 existing drives and get some Seagate 7200.11 or ES series drives and the ARC-1280ML it's getting more and more painful to lose a drive as capacities increase.

I just completed a rebuild of my 4U with 16x500GB 7200.9's and an ARC-1160ML2. I'm running *nix on it, but I did install 2003 and ran a bench or two on the array before filling it up:


Not too shabby for PCI-X based RAID6 :D

Holy sh*t, that is great performance considering your hardware:)

I was actually thinking on going to some sort of rack mount after psrage did his, I need a minimum of 24 hdds in hot swap bays...
Leaving normal pc hardware and cases behind, is a huge advantage, with Server 2k3, it is soooo stable....
 

beowulf7

[H]F Junkie
Joined
Jun 30, 2005
Messages
10,433
What is the power consumption for Galaxy 4.5 and what are you expecting for (in terms of power consumption) for Galaxy 5.0?
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
What is the power consumption for Galaxy 4.5 and what are you expecting for (in terms of power consumption) for Galaxy 5.0?

I never checked the true consumption of galaxy 4.5 with the green drives, however, prior to that, it was around 4.5amps of continuous draw.


I have been improving since the older galaxy (dual MP processors), the draw was almost 7 amps of power.


In regards to this new system, combined with the better integration and HE psu, I'm going to be hoping for the same power consumption... however, the second xeon proc and volume of memory may just kill that a little.
 

jmroberts70

2[H]4U
Joined
Oct 15, 2002
Messages
2,953
Amazing as always to see your progress Ockie! So I take it that the 24TB system is the goal for Galaxy 5.0 and the 50-bay chassis is intended for a Galaxy 6.0 transition in the future?
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
Amazing as always to see your progress Ockie! So I take it that the 24TB system is the goal for Galaxy 5.0 and the 50-bay chassis is intended for a Galaxy 6.0 transition in the future?

Perhaps. I guess it would also depend on how fast they move with storage capacities :)
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
So what are you going to use all this processing power for?

At this moment I'm trying to decide, although, having a dvdrw and this much processing power can make it a useful secondary machine :)
 

Rakinos

[H]ard|Gawd
Joined
Jul 31, 2001
Messages
1,520
you don't even know yet :eek: How powerful is your main machine?? A good way to chew though memory could always be as a vmware server for testing things out
 

beowulf7

[H]F Junkie
Joined
Jun 30, 2005
Messages
10,433
I never checked the true consumption of galaxy 4.5 with the green drives, however, prior to that, it was around 4.5amps of continuous draw.


I have been improving since the older galaxy (dual MP processors), the draw was almost 7 amps of power.


In regards to this new system, combined with the better integration and HE psu, I'm going to be hoping for the same power consumption... however, the second xeon proc and volume of memory may just kill that a little.

Wow, 4.5A to 7A - that's a pretty good power draw! I wonder how much that will add to your monthly electricity bill. :eek: Yes, you'll save a few bucks in the long run w/ the green HDDs from WD. :cool:
 

gjvrieze

[H]ard|Gawd
Joined
Sep 8, 2006
Messages
1,423
you don't even know yet :eek: How powerful is your main machine?? A good way to chew though memory could always be as a vmware server for testing things out

With 16GBs of ram, VMware ESX would be sweet; I am planning to build a ESX box for home, if I can get a legit copy of ESX for the right price:)
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
18,001
Wow, 4.5A to 7A - that's a pretty good power draw! I wonder how much that will add to your monthly electricity bill. :eek: Yes, you'll save a few bucks in the long run w/ the green HDDs from WD. :cool:


I roughly worked out, I may be wrong, but I pay about 40ish a month for Galaxy 4.0, so galaxy 3.0 should have been almost double that. Now with 5.0, it may be at 40ish or less even with the drive savings.
 

beowulf7

[H]F Junkie
Joined
Jun 30, 2005
Messages
10,433
I roughly worked out, I may be wrong, but I pay about 40ish a month for Galaxy 4.0, so galaxy 3.0 should have been almost double that. Now with 5.0, it may be at 40ish or less even with the drive savings.

That's not bad at all considering that you leave it running 24/7. :cool:
 

swatbat

[H]F Junkie
Joined
Apr 25, 2001
Messages
12,991
If you don't plan on running a raid again for the data why get the quad nic? Single drives will not be able to take advantage of 6 nics teamed.
 
Top