Project: Alters - VMware ESX Server, 56K warning - lots of pics

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Project name: Alters - is a name for alternate egos or personalities in people diagnosed with multiple personality disorder, now given that virtualization allows one to run multiple systems with multiple distinct operating systems all one one server I though the name was fitting.

Goal of the project: The goal is to expand my working knowledge of VMware ESX infrastructure as well as to consolidate and virtualize a bunch of existing servers I have in the house such as torrent client box, smoothwall firewall, Active directory etc.

Hardware:

2x Intel Xeon e5405 2.0Ghz Quad cpus
Asus DSBF-DE mobo
4 x Kingston 2x2 4GB memory kit
Adaptec 5805 SAS Raid Controller with battery backup
9 x Hitachi Ultrastar 36GB 15K RPM SAS drives, 8 in Raid 10 and 1 as a cold spare
1 x Samsung Spinpoint F1 750GB Sata drive to hold vm templates, iso images and serve as storage for torrent client.
Pioneer slim slot loading dvd drive + supermicro slim ide to sata adapter
2x Intel Dual Port Server network cards
Supermicro 825TQ 2U Rackmountable case with 8x hot swap bays

Software:


The server will run VMware ESX Foundation edition

VMs will consist of:

Server 2008 for Ms stuff such as active directory etc
Gentoo linux distro for torrent client, game server
Smoothwall 3 to act as a router/firewall

Possibly others to be added as need arises.

Some pictures:

You can see full resolution pics HERE

Overall shot of all the hardware that will be used in this build
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Close up of various components:

Hitachi SAS Drives







Pioneer slot loading dvd




Supermicro slim ide to sata adapter


16GB of kingston fb-dimms


Intel Dual Port Server NICs


Adaptec Raid Card

With the battery backup mounted
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Interior of the supermicro case







SAS Backplane



Xeon E5405 with retail active heatsink

 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Mobo installed:


CPUs and Ram, also hooked up the power connectors





The Single sata drive




Supermicro HDD trays, very nice build quality



Hitachi SAS drives mounted in trays

 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Slim DVD





Fans used in the case( 3 of those)



Raid card and network cards installed





Not much space to work with but I tried to keep it clean





 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Shot of the back



Front shot with hdd trays removed:



Drives installed and system powered on, it's Alive!



Some overall shots of finished system:



The cable for front connection (power buttons, leds etc) is too short so I had to route it in a ghetto way









 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Yes the case is very nice, the psu is bit loud and I might replace it with something quieter but I'll wait till after I move to the new house and mount the case in the rack.
 

Blue Fox

[H]F Junkie
Joined
Jun 9, 2004
Messages
11,785
Shiny...looks like it wasn't cheap. I bet you got a good deal on the drives though.
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
Got the drives on eBay, I think after shipping and everything it was like $40 per drive so not bad for 15K SAS drives.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,991
That's a lot of horsepower. You should look at hosting it so you have the bandwidth and expanded possibilities ;)
 

LittleMe

2[H]4U
Joined
Feb 20, 2001
Messages
2,977
That's a lot of horsepower. You should look at hosting it so you have the bandwidth and expanded possibilities ;)
LOL, I bet I know who could find him a great deal on hosting too right? Very nice build too axan.
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
I would like to setup a box at coloc but ockie is not willing to walk on the dark side :p
 

Concentric

Gawd
Joined
Oct 15, 2007
Messages
1,007
Love this, especially the case. Seems like a lot of horsepower for what you're doing but I suppose it's better to be [H]arder than necessary :)
 

Adidas4275

Supreme [H]ardness
Joined
Mar 11, 2006
Messages
5,986
makes all my boxes in my living room for my upcoming WHS build look like a pile of poop :(


nice build man
 

tgrimm

n00b
Joined
Dec 2, 2008
Messages
1
Very nice setup!

I'm looking at the supermicro 825tq as a chassis for a rack mounted SAN/NAS, with the adaptec 5805 RAID card.

Of course I don't need the power of dual Quad processor motherboard so I'm looking at some ATX candidates. (Most EATX motherboards provide dual processor support.)

Do you know if the 825tq provides a mount pattern that accepts an ATX motherboard? It looks like the standoffs can be moved - do all of those holes on the motherboard tray accept standoffs?

And did you have any issues with the lengths of the provided cables?
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
The chassis will let you install atx mobo; however the psu cables are pretty short if your atx plug is not along the top edge of the motherboard you might need an extension. Also the cable for power/reset, status leds is bit too short for my liking. It will reach pretty much anywhere on the mobo but there isn't enough slack to route it neatly. Another thing is that the case was designed for redundant psu and even the single psu models use 1u psus and have an empty bay for 2nd psu. Those 1U psus are pretty loud, it's not an issue if you have a dedicated server room/rack but you don't want to be in the same room with the case when it's on.
 

Elledan

[H]ard|DCer of the Month - April 2010
Joined
Oct 18, 2001
Messages
15,885
Imagining all the hot air from the CPUs blown over the RAM and chipset makes me cringe a bit :(

Further a really nice build :) Thanks for making us all feel jealous ;)
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
yes 8 fbdimms next to each other got pretty hot so I added an asus fbdimm cooler, now the ram stays under 60 C which is well under max operating temp for fbdimms.
 

Elledan

[H]ard|DCer of the Month - April 2010
Joined
Oct 18, 2001
Messages
15,885
yes 8 fbdimms next to each other got pretty hot so I added an asus fbdimm cooler, now the ram stays under 60 C which is well under max operating temp for fbdimms.
Ah yes, they're also FD-DIMMs... I've heard that they get really warm and will generally overheat if there's not enough airflow over them :eek: To add even more heat to that is a risky business :)

Good job on solving it :) although I must say that 60 C for RAM still sounds insanely warm to me. Then again I'm only use to regular DDR2 :p
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
it's perfectly safe, usual fb-dimm chip tjmax temp spec is between 85-125° C
 

Elledan

[H]ard|DCer of the Month - April 2010
Joined
Oct 18, 2001
Messages
15,885
You could turn an egg to charcoal with those temperatures :eek:

But I guess that the buffering serves a purpose :)
 
Joined
Nov 18, 2005
Messages
663
Noticed you're using a Adaptec 5805 card. We just had an issue with our similarly configured ESX box... a drive developed a bad block, the 5805 locked up, and the ESX server followed a moment later. It did this twice.

I updated the firmware to the 16343 release and it has been stable, despite a few more bad blocks on a dying drive.

Just a heads up.
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
thanx for the heads up lizardking, so far everything is running smooth as butter. I did the adaptec firmware update anyway, I like to keep current :p
 

nuclearsnake

Limp Gawd
Joined
Mar 8, 2003
Messages
445
Quick question about the cables you used from the 5805 to the backplane. The site says that it comes with;
Cables: two internal Mini SAS x4 to (4x) SATA fanout cables w/ sideband (2247000-R), 0.5 meter
Is this what you used?
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
ya i think i used the original cables, can't recall 100% they were mini sas to 4x sata + sgpio
 

skorpiond

n00b
Joined
Jun 11, 2009
Messages
5
Project name: Alters - is a name for alternate egos or personalities in people diagnosed with multiple personality disorder, now given that virtualization allows one to run multiple systems with multiple distinct operating systems all one one server I though the name was fitting.

Goal of the project: The goal is to expand my working knowledge of VMware ESX infrastructure as well as to consolidate and virtualize a bunch of existing servers I have in the house such as torrent client box, smoothwall firewall, Active directory etc.
]
Axan,

Great thread and awesome job with the custom server build... if you were to do this again, what would you do differently? Would you still use same hardware for this project?

Dannye
 

axan

[H]ard|Gawd
Joined
Nov 5, 2005
Messages
1,935
If i had to do it all over I would use same hardware for the most part, I would probably swap the hdds for ssds. I'm very happy with this server
 
Top