Complete Network Upgrade - Xeon D vSAN Cluster - PICS!!

jimphreak

jimphreak

[H]ard|Gawd
Joined
Nov 27, 2012
Messages
1,714
I'm getting married next year and thus this is my last opportunity to spend a good chunk of money on my "toys" before I need wifey approval for everything (queue the tears and boos). I've been saving up for a network overhaul for some time now (since before I proposed :thumb:)


My plan is to build a 3-node VMware vSAN cluster for the purpose of having full redundancy of all my VMs and Docker appdata. I want to be able to take a single server off-line while still keeping the Linux VM running my media dockers (mainly Plex) online as well as the rest of my VMs (Windows AD, DNS, etc.). I also just want to generally play around with VMware clustering, HA, and vMotion for my own personal knowledge as I'd like to implement some of this at work next year.

So without further ado, this is the hardware that will be going into a new server rack that will sit in my home office (I will build out a full network diagram once I have everything in place and will update you all with that in a subsequent post).

** Check the second post of this thread I will post what my current network looks like as it stands just so you can see where I'm coming from. **

Rack: 22U Linier Server Rack

3101-3-024-22-SIFF01_zpsm4wfqdoh.jpg



Equipment Going in the Rack (from bottom to top)

Cooling: AC Infinity CLOUDPLATE T7 - [2U]

UPS: CyberPower 2U 900w rack mountable UPS - [2U]

ESXi Node #1: vSAN Storage Only Node + UnRAID VM - [4U]

ESXi Node #2 & #3 (Identical Nodes): Hyperconverged Computing/Storage Nodes - [2U each]

10GbE Switch: Dell X1052 Smart Managed Switch (48 x 1Gb, 4 x SFP+ ports) - [1U]

pfSense Box: Router and Firewall - [1U]

24-Port Patch Panel: Cable Matters Rackmount or Wallmount 24-Port Cat6 RJ45 Patch Panel - [1U]



PICTURES UPDATES

Cable Matters Rackmount or Wallmount 24-Port Cat6 RJ45 Patch Panel

IMG_20160329_182102_zps0jttwaly.jpg



32GB Samsung DDR4 2133 RDIMMs

IMG_20160329_192826_zpsjbzvvemo.jpg



Hitachi 400GB SLC Enterprise SSDs

IMG_20160329_192214_zps6rr7ihdc.jpg



ICY DOCK MB994SK-1B ToughArmor 4 x 2.5" SATA 6Gbps/SAS HDD/SSD Mobile Rack / Cage with key lock in 1 x 5.25" bays

IMG_20160329_182810_zpsqg3xobl7.jpg

IMG_20160329_182848_zpssxlmtpg8.jpg

IMG_20160329_182909_zpseamdzcwb.jpg



360w Seasonic Gold PSUs

IMG_20160329_182327_zpsvtrkzt3n.jpg



iStarUSA D Value D-214-MATX Black Steel 2U Rackmount Chassis'
- Pre-Installation

IMG_20160329_182547_zps28hzjf3z.jpg

IMG_20160329_182605_zpsl4zvmmnt.jpg

IMG_20160329_182622_zps24e7odqh.jpg


iStarUSA D Value D-214-MATX Black Steel 2U Rackmount Chassis'
- Internal 2.5" drive mounts removed. ICY DOCK, PSU, and case fans installed.

IMG_20160329_190455_zps2y4o3qsn.jpg

IMG_20160329_190912_zpssc2au76t.jpg

IMG_20160329_190926_zpsoigoa4ko.jpg
iStarUSA M-4160-ATX 4U 16-Bay Trayless Chassis

M-4160-ATX-01_zpsbs6lubct.jpg

M-4160-ATX-02_zpsepugckha.jpg

M-4160-ATX-03_zpsiod1w9lt.jpg

M-4160-ATX-04_zps47nn208j.jpg

M-4160-ATX-05_zpsl8n39yph.jpg

M-4160-ATX-06_zpsrooa4xex.jpg



CyberPower 2U 900w rack mountable UPS

CyberPower%20UPS-01_zpsh6adnxvh.jpg

CyberPower%20UPS-02_zps0yf14jem.jpg



CyberPower RMCARD303 OL Series Remote Management Card

RMCARD303_zps7xdq9q7m.jpg

rYIwUSG.jpg

8nwncIU.jpg

kZFzjjU.jpg

W7pEcGb.jpg

6XyHMqv.jpg

zSbCGcu.jpg

zbUSoux.jpg

LkvTFlB.jpg

wvjYdt8.jpg
NQjCVs4.jpg

BLMIWdC.jpg

kOEwUuA.jpg

ALyXjmk.jpg

5LpJQSJ.jpg

6CamWjz.jpg
 
Last edited:
As an Amazon Associate, HardForum may earn from qualifying purchases.
As promised, this is what my network looks like at this moment before this overhaul.


Site #1 Network (My Condo)

Network Devices:
  • pfSense Firewall
  • Netgear ProSafe Plus switches (vlan tag support)
  • UniFi AC Access Point
  • Various Media Streaming Devices (Chromecasts, Tivo BOLT, Samsung SmartTV)

Server:

56TB UnRAID Server running an assortment of dockers (Apache, Plex, PlexPy, PlexRequests, Sonarr, CP, NZBGet, Hydra, Madsonic, Miximux, UniFi, etc.). Also run a bunch of test VMs inside the build-in KVM hypervisor.

PC:

Main everyday workstation. Was built for gaming but I never game on it. Mainly just use to for web browsing and as a terminal for configuring/testing on my UnRAID server.


Site #2 Network (Parents House)

Network Devices:
  • pfSense Firewall
  • Netgear ProSafe Plus switch
  • Linksys E1000 AP (DD-WRT)

Server:

56TB UnRAID Server backs up to this over site-to-site VPN nightly)
 
Last edited:
 
Last edited:
As an Amazon Associate, HardForum may earn from qualifying purchases.
Installed most of the rack units including 2U AC Infinity Cooling unit.

NQjCVs4.jpg

BLMIWdC.jpg

kOEwUuA.jpg

ALyXjmk.jpg

5LpJQSJ.jpg

6CamWjz.jpg
 
I know I could look up all the parts ... but how much did all that set ya back? Damn that's a lot of shit at once. One thing I would do different is for the storage server, run Linux on it, and ZoL and just do my own storage management with ZFS manually instead of using unraid, but hey to each their own.
 
extide said:
I know I could look up all the parts ... but how much did all that set ya back? Damn that's a lot of shit at once. One thing I would do different is for the storage server, run Linux on it, and ZoL and just do my own storage management with ZFS manually instead of using unraid, but hey to each their own.
Click to expand...

The UnRAID server is just for my bulk media. I have no desire to run ZFS on my bulk array because I want to have my unused disks spun down which you can't do with any kind of stripped array. My VMs/Dockers will all be stored on the vSAN cluster.

As for the cost of this project...a LOT :eek:. But I've been saving for a bit for this upgrade so I'm not freaking about over it.
 
Looks pretty good man. See I don't give a shit if my disks spin down or not, and I prefer to manage things manually myself rather than have a UI and have it all done for me. But hey, like I said, different strokes for different folks.
 
extide said:
Looks pretty good man. See I don't give a shit if my disks spin down or not, and I prefer to manage things manually myself rather than have a UI and have it all done for me. But hey, like I said, different strokes for different folks.
Click to expand...

I totally get it. I personally want a quickly accessible UI to check on things as I rarely ever make changes to that array. And since my disks are really only in use for 6 hours a day (at night) on average I see no reason to have them spun up 24/7. I can write to that array (via cache pool) at 10Gb speeds even so I see no performance hit.
 
hey Jimphreak, I love it. I really do. Seems like a very solid setup to me!

I'm glad to see that I'm not the only one who uses a "Hot / Warm" storage configuration with ZFS and UnRaid. The majority of home bulk storage doesn't need to be spun up all the time, and the power savings are considerable. (Also, less heat being generated)
 
Last edited:
iamwhoiamtoday said:
hey Jimphreak, I love it. I really do. Seems like a very solid setup to me!

I'm glad to see that I'm not the only one who uses a "Hot / Warm" storage configuration with ZFS and UnRaid. The majority of home bulk storage doesn't need to be spun up all the time, and the power savings are considerable. (Also, less heat being generated)
Click to expand...

Thanks iamwhoiamtoday. Yea it makes sense for those of us who separate where we store our different types of data. I used to store everything on the same array and I'd want my drives spun up all the time for immediate access. But for my media, I can wait the 5-10 seconds it takes to spin up the drive(s).
 
I've completed my network overall. Wound up with 4 nodes in the vSAN cluster. Here is a pic of the finished product and a full post about it can be found here.

1EU4Aga.jpg
 
You must log in or register to reply here.
Back
Top