First look: Supermicro F617H6-FTL+ Ultra high density storage/cluster

P

packetboy

Limp Gawd
Joined
Aug 2, 2009
Messages
288
They say I got the first two of these from the factory:

48TB *and* 12 cores PER 1U!

HhfArn5.jpg



X4 per 4U chassis gives you almost ~200TB and 96 (hyperthreaded) cores.

Some initial comments:

* Amazing density...amazing price for what you get

* Great power efficiency...E2600 CPUS *and* 80Plus Platinum PS

* 12 (yes 12) internal drives per 1U...NOT hot swappable...however, drives are installed in a metal carrier and then snap into the blade, then ONE screw to secure to chassis...so drives can be swapped pretty fast, esp. if you purchase some spare carriers.

* Only two of the 12 drives can be accessed by partially sliding the blade out...the other 10 are underneath a metal top that covers most of the top of the blade...not really a big deal

* Big error I made..did NOT realize these were extended length chassis...barely fit in cabinet and even then the back end of the enclosure blocks tons of power plugs on the "zeroU" APC power distribution strips I'm using....fortunately I have plenty of outlets to work with and am not trying to fill the whole cabinet with these things.

* Other bit error I made...doh...PCI slots are on the FRONT of these servers, NOT the back. I've got a 40Gbps Infiniband card in each blade and estimated my IB cable length to be 1m when in reality I need 3m to reach the front of the enclosure...now have $500 in IB cables.

We have two of the eight nodes operational...waiting on longer IB cables to fire up the rest of the nodes...this augments our existing 5 node Hadoop cluster:

Code: 
CLDB Status : RUNNING
Cluster Capacity : 258 TB
Cluster Used : 175.06 TB
Cluster Available : 82.94 TB
Cluster Used Percentage : 67

Cluster capacity will be 600TB once we get those cables...woot!
 
Backplane speed? there is no backplane.

As you can see each drive is wired itself to the motherboard, so 3g/6g/sata/sas, it will be whatever you have available.
 
New Infiniband cables are in:

Xwt6awd.jpg


They were a bit more expensive ($90/each), but note the ribbon looking IB cables on the left...these are from 3M and are *significantly* easier to install and have a much more liberal max turn radius that typical round IB cables.

5 of 8 nodes now up:

Code: 
Cluster Capacity : 341.74 TB
Cluster Used : 178.76 TB
Cluster Available : 162.98 TB
Cluster Used Percentage : 52

Code: 
CPU	4%	11 cores	264 cores
Memory	30%	433.9GB	1.4TB
Disk Space	52%	178.8TB	341.7TB
 
packetboy's ops guy and assistant, here. 7/8 nodes up. So much space. So. Much. Memory. Power Overwhelming! Etc. :)

Code: 
	   %	Utilized  Total
CPU	   2%	7 cores	  288 cores
Memory	   30%	479.9GB	  1.6TB
Disk Space 40%	182.2TB	  449.9TB
 
I'm a little late to the party, but what is/was the intended purpose of this cluster? It can't just be storage because oh god cores. Is this a Hadoop cluster?
 
Pr0n storage? Or is it for simultaneously watching many cat movies on the youtube? What is it for? :)
 
Well this is for his hardoop cluster.

I'm thinking they could make some very nice imap servers though :)
 
You must log in or register to reply here.
Back
Top