Your home ESX server lab hardware specs?

Discussion in 'Virtualized Computing' started by agrikk, Dec 18, 2008.

  1. agrikk

    agrikk Gawd

    Messages:
    923
    Joined:
    Apr 16, 2002
    I'm going to be buying some new hardware to build two 64-bit capable VMWare ESX 3.5 servers for a home lab and I'm interested in what you guys are using for your labs.

    Before I go out and do some research on my own, I'm looking for your experiences with the hardware you've already purchased.


    Right now I'm thinking about two machines with:

    one or two quad-core 64-bit CPUs (Intel v AMD?)
    Motherboard type?
    8-16gb RAM
    SCSI v SATA II?

    What's in your boxes?

    I'm also trying to figure out what to use for storage for the VMWare cluster (for DRS and VMotion stuff). I have a Fiber channel SAN that I'm currently using in my production environment, but I could pull it out and redesign things there to make my lab easier, but I'd rather go with something else.

    Perhaps I could build a Linux box and use NFS as the shared storage, but I've never done it (which makes it more appealing a project :D ) and am not really sure how it works. Can I use a 32-bit linux machine (with 1.5TB of direct attached SCSI drives) as the storage host serving NFS to 64-bit VMWare servers? Which flavor of Linux works the best in this capacity?
     
  2. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,992
    Joined:
    Feb 20, 2001
    I only have my ESX lab here at home and it's a Dell PowerEdge 1950. It's got 2 quad-core 2.66Ghz chips and 16GB of memory. 2 SAS drives mirrored for booting only and it's tied to a 12TB iSCSI SAN for guest storage.

    You can put FreeNAS on your 1.5tb box and use it for guest storage (via iSCSI or NFS). It doesn't matter if that box is 32-bit or 64-bit linux if you don't go with FreeNAS.
     
  3. agrikk

    agrikk Gawd

    Messages:
    923
    Joined:
    Apr 16, 2002
    Wait, so you are saying that if I go with FreeNAS it does matter if the box is 32-bit or 64-bit but anything else it doesn't matter?

    Or was that double negative a typo?

    But looking at FreeNAS it looks really promising. iSCSI and NFS support? Booyah.
     
  4. sabregen

    sabregen Permabanned

    Messages:
    16,913
    Joined:
    Jun 6, 2005
    I have used the following for ESX, and ESXi. Many configs, so I will attempt to container-ize each one:

     
  5. sabregen

    sabregen Permabanned

    Messages:
    16,913
    Joined:
    Jun 6, 2005

    I don't believe it can do both concurrently, but has the capability to do an either-or type of setup. that being said, you could use VMWare server to run a FreeNAS VM as iSCSI and one as NFS...just a thought.
     
  6. ben chi(f4)

    ben chi(f4) 2[H]4U

    Messages:
    2,352
    Joined:
    Mar 4, 2008
    HP ProLiant DL380 G5

    6 NICs
    1.06 TB SAS Raid0 (Raid0 because I don't care about fail)
    2 Intel Xeon Quad Core Processors @ 3mhz
     
  7. Jay_2

    Jay_2 2[H]4U

    Messages:
    3,584
    Joined:
    Mar 20, 2006
    right now I have a really old Dell 1600SC with 2 x 2.4GHz Xeons 1 GB RAM and 3 x 36GB SCSI in RAID 5.

    I curently have an exchange server, DNS server and a basic XP and 2003 test image for messing around with. I am soon moving to a monster of a system though as ESXi is really addictive!
     
  8. agrikk

    agrikk Gawd

    Messages:
    923
    Joined:
    Apr 16, 2002
    I'm not planning on using both concurrently. I'll be doing one or the other, but I like having the option to do either/or.
     
  9. FibreChanMan

    FibreChanMan [H]ard|Gawd

    Messages:
    1,908
    Joined:
    Aug 18, 2001
    It's not at home (new infant at home, so no ESXi fun at home :) ), but it's my ESXi test system at work:

    Dell PowerEdge 2850
    Dual 3.0ghz Xeon processors
    2gb RAM
    2x 36gb drives in RAID 1, 3x 146gb drives in RAID 5, and 1x 146gb drive as hot spare (All U320 SCSI)
    Dual gig nics
    PERC 4 D/i RAID controller

    Currently has 1 Windows Server 2003 DC, a Vista 32-bit host and one more Win2K3 box I haven't done anything with yet. Managed by a desktop system running Win2K3 that is also a DC replicating with the VM one.
     
  10. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,992
    Joined:
    Feb 20, 2001
    Yes, typo. It would be easier to go with FreeNAS but if you don't, it doesn't matter if you go with a 32-bit or 64-bit install of Linux. ESX is only accessing it across the network, so it doesn't care what it's running as long as it conforms to NFS or iSCSI standards.
     
  11. agrikk

    agrikk Gawd

    Messages:
    923
    Joined:
    Apr 16, 2002
    I think I might try starting with FreeNAS because it seems the easiest route to shared storage, but I've also seen a Linux iSCSI target by Ardis Technologies that I'm curious about. It would be a harder implementation, since I'm a windows guy who knows only the basics about navigating around a Linux box, but that would make it all the more interesting for me.

    It also seems like people run VMWare on just about anything, so that's cool.

    Right now I'm thinking about:
    ASUS P5Q SE Plus motherboard
    Intel Xeon X3220 Kentsfield 2.4GHz Quad-core CPU
    8GB Crucial Ballistix DDR2 SDRAM DDR2 800 (PC2 6400)
    Cheapo PCIe 16x Vid card
    Whatever hard drive I have handy
     
  12. axan

    axan [H]ard|Gawd

    Messages:
    1,940
    Joined:
    Nov 5, 2005
    My esx server specs:

    ESX 3.5 U3

    2x Intel Xeon e5405 2.0Ghz Quad cpus
    Asus DSBF-DE mobo
    4 x Kingston 2x2 4GB memory kit
    Adaptec 5805 SAS Raid Controller with battery backup
    9 x Hitachi Ultrastar 36GB 15K RPM SAS drives, 8 in Raid 10 and 1 as a cold spare
    1 x Samsung Spinpoint F1 750GB Sata drive to hold vm templates, iso images and serve as storage for torrent client.
    Pioneer slim slot loading dvd drive + supermicro slim ide to sata adapter
    2x Intel Dual Port Server network cards
    Supermicro 825TQ 2U Rack mountable case with 8x hot swap bays

    build log here
     
  13. xphil3

    xphil3 [H]ard|Gawd

    Messages:
    1,212
    Joined:
    Nov 11, 2005
    a bit dated(almost 2 years), but still does the job quite nicely.
    ESX 3.5(soon to be 4.0 FINAL:cool:;))
    2x Intel Xeon R5310 1.6Ghz
    Supermicro X7DVL
    6x 2GB super talent ECC
    4x 250GB SATA1 Raid 5
    1x LSI Logic 150-4

    also a bit dated
    ESXi - Dell 2650
    2x Intel Xeon 3.0Ghz
    2x 36GB U320
    2x 1GB ECC

    I wont even post of specs of my 2.5 rigs, lol. Eventually going to do some kind of NAS solution.... Work keeps kicking my ass though.
     
  14. lopoetve

    lopoetve Imhotep

    Messages:
    28,619
    Joined:
    Oct 11, 2001
    4x DL380 G5's.
    4x Dell 2950s
    1x Clariion CX3-40
    1x MSA 1000
    1x Netapp 3060 (IIRC).

    ... ^_^
     
  15. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,992
    Joined:
    Feb 20, 2001
    LOL, that's your "lab" at home or production at work?
     
  16. axan

    axan [H]ard|Gawd

    Messages:
    1,940
    Joined:
    Nov 5, 2005
    hehe i think that's his production, otherwise we can all go home .. :D
     
  17. Gott

    Gott [H]ardness Supreme

    Messages:
    5,163
    Joined:
    Feb 28, 2008
    We're running 4xDell 2950's at work.

    I don't have an ESX or ESXi machine at my home, though.
     
  18. REDYOUCH

    REDYOUCH [H]ardness Supreme

    Messages:
    4,498
    Joined:
    Mar 17, 2001
    I guess I have somewhat of an "old" setup at home:

    SuperMicro X5DA8
    2x 3.2Ghz XEON
    12GB DDR
    LSI MegaRAID Sata with 2x80GB drives in RAID1

    Gonna throw on some 1 tb drives soon.

    Wish I had gone with 64-bit opterons though...
     
  19. lopoetve

    lopoetve Imhotep

    Messages:
    28,619
    Joined:
    Oct 11, 2001
    ... play lab at work ;)

    Although I'm looking at picking up a used Clariion and NetApp. :)
     
  20. Vito_Corleone

    Vito_Corleone [H]ard|Gawd

    Messages:
    1,730
    Joined:
    Dec 17, 2006
    I just grabbed a Dell GX620 off Ebay for ~$150. The specs are:

    P4 2.8ghz
    2.5gb of RAM (came with 512, I put the extra 2gb in)
    250gb HDD (came with 40gb, I stuck the 250gb in)
    GigE

    I only got it to play with ESX. I'd like to get some exposure to it.
     
  21. `danny

    `danny Limp Gawd

    Messages:
    386
    Joined:
    Feb 26, 2007
    Self built server:

    1) Q6600
    2) 8GB DDR2 PC26400 RAM
    3) 5xSamsung 500GB Spinpoint T series in RAID 5 using Perc 5i
    4) Supermicro ATX Server Board (Forgot model #)
     
  22. shabazkilla

    shabazkilla Lurker

    Messages:
    435
    Joined:
    Jan 4, 2004
    Specs from my lab:

    ESX hosts -
    Q6600
    Intel BOXDQ35JOE motherboard
    8GB DDR2 RAM
    1x250GB SATA hard drive
    2 x Intel NICs

    Shared Storage -
    Openfiler 2.3 OS
    Q6600 (overkill I know)
    8 GB DDR2 RAM
    2x250GB SATA drives in software RAID1 (OS)
    4x500GB SATA drives in RAID 10 using a Dell Perc5i presented as iSCSI storage

    The Intel motherboard works great for the ESX hosts. The only thing that isn't compatible with ESX is the onboard NIC.
     
  23. geiger

    geiger Limp Gawd

    Messages:
    412
    Joined:
    Jun 23, 2005
    Homebrew running ESXi:
    Supermicro X7DVA-E-O
    Dell Perc5i
    8GB ECC DDR2(A-Data)
    Dual Intel E5430(Quad 2.66GHz)
    Supermicro CSE-743T-645B Case
    5x 500GB 7200.10
    2xOnboard Intel 1000
    5x 1gb links between 2 dual port and 1 single port intel nics(1 dual PCIE, 1 dual PCI-X, Single is PCI).
     
  24. Adakos

    Adakos Gawd

    Messages:
    545
    Joined:
    Sep 9, 2008
    H8DAE w/2x Opty 275
    12 GB of ECC RAM
    External - HP Netstore SCSI enclosure(8x20GB drives)
    ARK-600 4U Rackmount server chassis
    2x Onboard Broadcom NIC

    everything runs spectacularly.
     
  25. leward

    leward n00bie

    Messages:
    54
    Joined:
    Sep 1, 2008
    2x Athlon MP 2400+
    Gigabyte GA-7DPXDW-P
    3.25GB DDR266
    2x Intel e100 NICs
    Adaptec 2410sa SATA RAID controller
    2x Maxtor 160GB in RAID 0

    An oldie, but it's a great way for me to learn ESXi with stuff I already have.
     
  26. The Spyder

    The Spyder 2[H]4U

    Messages:
    2,629
    Joined:
    Jun 18, 2002
    Dell Precision 690
    Dual Xeon Quad 2.33
    20gigs FBDimms kingston
    2x 640gig AAKS in raid 0 (Everything is backed up on the fileserver)
    2 Intel Pro 1000 nics
    2 Intel Pro 1000 Fiber nics
     
  27. jtg1993

    jtg1993 Gawd

    Messages:
    812
    Joined:
    Apr 25, 2008
    Hey, I have enough money from my bday and christmas to upgrade my server for running vm's I found 4GB ddr2, a E5200 (will overclock) and a Gigabyte GA-EP43-DS3L, does anyone know if the GA-EP43-DS3L is pretty much fully suppored by esxi 3.5? If it isn't I will haveto run server 03 with vmware server.
     
  28. _tar

    _tar n00bie

    Messages:
    44
    Joined:
    Apr 13, 2006
    3800+ X2
    Some MSI mobo (It's an nForce chipset with an onboard Quadro)
    4GB DDR2 800
    1x80 GB
    1x200 GB
    1x320 GB

    It's no super server but I work with what I have. The onboard SATA controller is not on the HCL but it is one of those that still works.

    I plan to change it around a bit by taking out the hard drives and booting ESXi off USB and throwing the hard drives into an OpenFiler system (Sempron LE-1100, 512mb).
     
  29. Sniper|3d-R|

    Sniper|3d-R| [H]ardness Supreme

    Messages:
    4,618
    Joined:
    Oct 30, 2000
    I have an ESX 3.5 server that's been running solid for the past year+. Nothing to special with the hardware... Just stuff I had laying around... I need a better SATA controller for starters.....

    Code:
    P4 3.0Ghz
    Mobo: D875BZ
    3.5GB Mem
    GeForce3 Ti 200
    HDD: 
    Storage 3 SATA 120's,OS is on an IDE 120GB.
    Intel Corporation 82547EI Gigabit Ethernet Controller
    Intel Corporation 8254NXX Gigabit Ethernet Controller
    Intel Corporation 8254NXX Gigabit Ethernet Controller 
    Intel Corporation 8254NXX Gigabit Ethernet Controller
    
    I have just acquired a free Dell Poweredge 2650 from a good friend of mine. I've already added the dual 3.2Ghz xeons, and 2GB of memory. I'm still waiting on storage. This may be my new ESX server, or just another ESX 3.5 server. I have 2 Intel D3368 dual PCI-X NIC's that I got from my buddy as well... These have TCP off loading so very cool indeed!

    Thats all for now! :)
     
  30. Soulstice

    Soulstice [H]ard|Gawd

    Messages:
    1,106
    Joined:
    Jun 26, 2001
    Mine is in the sig. And, I don't run ESX or ESXi anymore. I've switched fully to Hyper-V Server 2008. It hasn't caught up to ESX just yet (e.g. live-migration, etc.), but it will with the release of Win2K8 R2 (which includes MS's own version of live-migration, and a new dedicated VM disk systems... similar to VMFS).
     
  31. Soulstice

    Soulstice [H]ard|Gawd

    Messages:
    1,106
    Joined:
    Jun 26, 2001
    ooops, my sig doesn't include drive and chassis info.

    Here it is:

    Chassis: http://usa.chenbro.com/corporatesite/products_detail.php?serno=41 (4U, 16-drive SATA backplane hot-plug)

    Drive Setup:

    Hyper-V Server 2008 OS RAID1 Array: 2 x WD Velociraptor 150 GB
    VMs Array 1, RAID10: 4 x WD Velociraptor 150 GB
    VMs Array 2, RAID10: 4 x Maxtor 300 GB
    VMs Array 3, RAID1: 2 x Maxtor 300 GB

    I use my environment to support my consulting business, perform test deployments, practice with new technologies (SQL 2008, Exchange 2007, etc.) I have roughly 12 machines running at all times.
     
  32. sam-sgc

    sam-sgc [H]Lite

    Messages:
    83
    Joined:
    Oct 29, 2005
    Q6600
    P5Q Pro
    12 GB Ram
    4x Intel Gbits cards
    1x 250gb disk - OS
    1x 1tb disk - Backup
    1x 320gb disk - SCVMM Library
    2x Raid-0 WD 640gb disks - VM Storage
    2x Raid-0 120Gb - ISCSI Storage

    Pretty fast so far !

    Planning on building a 2nd identical box for Host clustering soon.
     
  33. PurduEE

    PurduEE BIOSonic Man

    Messages:
    1,572
    Joined:
    Oct 9, 2000
    I built this server up just before Christmas to run SBS 2003 R2 (which was on my old server) and WHS:

    AMD 4850e
    Asus M2N-E
    8GB Geil DDR2 @ 800Mhz
    LSI 8344ELP, LSI SAS3041-e sata controllers in the x16 and x4 slots, respectively
    2x Intel Pro 1000 pci-e x1 Gb NICs
    4x500GB in RAID-5 on the 8344ELP (VM's stored here)
    4x1.5TB on the MCP55
    2x1TB on the SAS3041
    4GB USB key for ESXi installation.

    Total space in WHS is like 7TB, with another 500 or so for SBS. Works outrageously great and total cost was probably less than $1500. I debated quad-core, but this thing sits in a closet, so I went the lower wattage route. If I added some more VMs, I'd probably need to revisit the issue.
     
  34. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,992
    Joined:
    Feb 20, 2001
    You're almot ahead on drive space. Nice setup and at a great price.
     
  35. agrikk

    agrikk Gawd

    Messages:
    923
    Joined:
    Apr 16, 2002
    My question about processors is, is it worth it to go quad-core Xeon for a VM host, or will a simple Core 2 Quad suffice?

    Can anyone point me to some benchmarks or something for VM performance?


    Also,

    How do you get 12 GB RAM on a Motherboard with four slots? a pair of 4GB sticks and 2GB sticks? What memory did you use for your 4GB sticks? The ASUS web site says the Max Memory on this board is 8GB: how stable has it been with more than 8 in it?
     
  36. luie

    luie Limp Gawd

    Messages:
    469
    Joined:
    Apr 21, 2002
    Home use:

    CPU = Q6600 3.0Ghz
    RAM = 8GB OCZ Reaper HPC PC2-6400
    HDD = 750GB SATA2
    Motherboard = Gigabyte P965-DS3 rev3.3

    running 4 VMs. 2 Linuxes, 2 Windows.
     
  37. Dangman

    Dangman Ninja Editor SuperMod Staff Member

    Messages:
    46,670
    Joined:
    Dec 15, 2005
  38. sam-sgc

    sam-sgc [H]Lite

    Messages:
    83
    Joined:
    Oct 29, 2005

    P5Q Pro support 16GB of ram

    It's actually pretty stable.. even overclocked to 3.2Ghz

    Sam
     
  39. lizardking009

    lizardking009 Gawd

    Messages:
    663
    Joined:
    Nov 18, 2005
    Do you get ESXi health status information for your RAID arrays on the PERC4?