![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Hello there,
I have been roaming the forum for some time and thought I should introduce myself, along with my cluster build for discussion. I am in the works of building a small Tier-2 Data center. The building should be finished next year by February, and I plan to open sometime in March. I have a starting budget of around $50k for hardware and software.
For starting, I have put together the following:
Two hypervisor servers:
These are great 2-Socket boards. Each CPU has 12 DIMM slots for up to 786GB of RAM with 32GB RDIMM/LRDIMM (or up to 1.5TB with 64GB LRDIMM apparently) per host. I really wanted to start off with 4-Socket processor configurations for the density on the Hypervisor computes, but the price difference Intel wants for the processors was hard to justify for now. I can justify their cost in the future because it is easier to manage more computing power per host rather than it being split into two, 2S hosts with the same total compute power (and slightly higher power consumption from duplicate components from having two boards, etc). Also, I will eventually buy 32GB or 64GB modules and move the 16GB DIMMs into the storage servers.
Two storage servers:
I want to run ZFS on these storage servers. I looked into Nexenta's offering and their licencing cost for a small business such as mine just didn't make sense. They have no cost effective SMB transition, and sadly, I will not be using them. I have been looking into OmniOS with Napp-it Pro, and then either buy the HA plugins for it or build them myself with some help to complete the build.
For SLOG/ZIL, I have been keeping my eye on the Intel SSD DC S3700 200GB at $480 or Seagate 600 Pro 200GB at $290. They both have fantastic write performance for the price (although the Seagate is rated at 500TB max writes vs 3.65PB on Intel). However, recently I found a drive that seems to compete very well and sometimes even best the SMART Optimus 400GB drive for writes (which is fair on price but has an amazing performance/endurance ratio). The drive I am refering to is the "SuperSSpeed Hyper Gold S301 128GB SLC Enterprise SSD" (http://www.tweaktown.com/reviews/53...per-gold-slc-enterprise-ssd-review/index.html). Since this drive has Intel SLC 25nm chips and no DRAM cache, I think it is well suited for SLOG/ZIL device (I bought two for testing at $260 each).
For L2ARC, I was thinking that the Samsung Pro 256GB drives are great for their price.
For the actual pool, I still have not decided. I'd think that the latest 500GB VelociRators are good for entry level. I have looked into enterprise SAS 10k drives from Seagate, WD, etc, and they're decently priced for 450-600GB ($350 vs $150).
Network equipment:
These Mellanox devices are IB QDR-only switches as far as I am aware. They all have an integrated Subnet Manager, which is great. The 1GbE EX4200-24T switches will handle all but storage traffic. The InfiniBand network will only handle storage traffic. Perhaps the IB network will also handle vMotion if I find IPoIB to be decent.
Well, I need to get some sleep. I will be back with more details and updates. Ever since I found out about that SLC SSD I've been up all night and morning doing research on it because I really did not want to spend $2,300 for a ZeusRAM of the new 2.5" ZeusZIL that I think goes for $1,800.
I have been roaming the forum for some time and thought I should introduce myself, along with my cluster build for discussion. I am in the works of building a small Tier-2 Data center. The building should be finished next year by February, and I plan to open sometime in March. I have a starting budget of around $50k for hardware and software.
For starting, I have put together the following:
Two hypervisor servers:
Code:
Supermicro 6027R-3RF4+ (2S/24xDIMM/4x1GbE)
1x Intel Xeon E5-2650 v2 (2.60GHz 8C 20MB 8.0GT)
4x Hynix HMT42GR7MFR4A-H9 (16GB 1333MHz 1.35v RDIMM)
1x Mellanox MCX354A-QCBT (ConnectX-3 VPI IB QDR / 10GbE)
Either USB Flash drive for local boot or iSCSI network boot.These are great 2-Socket boards. Each CPU has 12 DIMM slots for up to 786GB of RAM with 32GB RDIMM/LRDIMM (or up to 1.5TB with 64GB LRDIMM apparently) per host. I really wanted to start off with 4-Socket processor configurations for the density on the Hypervisor computes, but the price difference Intel wants for the processors was hard to justify for now. I can justify their cost in the future because it is easier to manage more computing power per host rather than it being split into two, 2S hosts with the same total compute power (and slightly higher power consumption from duplicate components from having two boards, etc). Also, I will eventually buy 32GB or 64GB modules and move the 16GB DIMMs into the storage servers.
Two storage servers:
Code:
Supermicro 2027R-E1R24N (2S/24xDIMM/4x1GbE)
1x Intel Xeon E5-2620 v2 (2.10GHz 6C 15MB 7.2GT)
4x Hynix HMT42GR7MFR4A-H9 (16GB 1333MHz 1.35v RDIMM)
1x Mellanox MCX354A-QCBT (ConnectX-3 VPI IB QDR / 10GbE)
Either USB Flash drive for local boot or iSCSI network boot.I want to run ZFS on these storage servers. I looked into Nexenta's offering and their licencing cost for a small business such as mine just didn't make sense. They have no cost effective SMB transition, and sadly, I will not be using them. I have been looking into OmniOS with Napp-it Pro, and then either buy the HA plugins for it or build them myself with some help to complete the build.
For SLOG/ZIL, I have been keeping my eye on the Intel SSD DC S3700 200GB at $480 or Seagate 600 Pro 200GB at $290. They both have fantastic write performance for the price (although the Seagate is rated at 500TB max writes vs 3.65PB on Intel). However, recently I found a drive that seems to compete very well and sometimes even best the SMART Optimus 400GB drive for writes (which is fair on price but has an amazing performance/endurance ratio). The drive I am refering to is the "SuperSSpeed Hyper Gold S301 128GB SLC Enterprise SSD" (http://www.tweaktown.com/reviews/53...per-gold-slc-enterprise-ssd-review/index.html). Since this drive has Intel SLC 25nm chips and no DRAM cache, I think it is well suited for SLOG/ZIL device (I bought two for testing at $260 each).
For L2ARC, I was thinking that the Samsung Pro 256GB drives are great for their price.
For the actual pool, I still have not decided. I'd think that the latest 500GB VelociRators are good for entry level. I have looked into enterprise SAS 10k drives from Seagate, WD, etc, and they're decently priced for 450-600GB ($350 vs $150).
Network equipment:
Code:
2x Juniper SSG-520M-SH
2x Juniper EX4200-24T
2x Mellanox IS5022-IS5035 (depends on price/ports).These Mellanox devices are IB QDR-only switches as far as I am aware. They all have an integrated Subnet Manager, which is great. The 1GbE EX4200-24T switches will handle all but storage traffic. The InfiniBand network will only handle storage traffic. Perhaps the IB network will also handle vMotion if I find IPoIB to be decent.
Well, I need to get some sleep. I will be back with more details and updates. Ever since I found out about that SLC SSD I've been up all night and morning doing research on it because I really did not want to spend $2,300 for a ZeusRAM of the new 2.5" ZeusZIL that I think goes for $1,800.