Spring '05 Build (supercomputer inside)

man, that must suck when you have to take apart, transport, and put this farm back together.. Is there some sort of easy way to do it?
 
slowbiznatch said:
They're all in the back. Every system has at least 1 power cable, 1 ethernet cable, and 1 fibre cable. All the quads have an additional ethernet cable for serial monitoring (hooked to a Cyclades switch) and the large quads have 3 additional power cables for the redundant power supplies. That all makes for about 3000 cables in varying lengths, all behind or under those racks.

By the way, everything is up and running. All nodes have been repaired and we have no downed nodes. Last we checked, the first order is pulling about 4 teraflops with no tweaking. That number should grow quite a bit within the next few days.
Click to expand...

Sorry I think I wasn't clear. They aren't shipped with the cable's connected are they? I mean those fibre's are kind of fragile IIRC.
 
slowbiznatch said:
Yep, each rack is shipped with fibre attached. We'll lose about 20 cables during shipping, but 20 out of 400 isn't too bad.
Click to expand...

I am sure you guys have this worked.....but it is actually more cost effective/time effective to lose them than not? I guess it is though afterall how much are 20 fibre cables to a few million dollar budget anyway........if only I had money like that.
 
im wondering why you didn't use the G5 processors, i remember that virginia tech built a system with about 2200 procs that cost them just about 5mill, and they hit 10 teraflops
and what OS will this system be running?
 
mewannafastpc said:
im wondering why you didn't use the G5 processors, i remember that virginia tech built a system with about 2200 procs that cost them just about 5mill, and they hit 10 teraflops
and what OS will this system be running?
Click to expand...

That cluster also didn't have ECC......and they tore it down to replace it with systems that did support ECC IIRC.
 
Just a question, because I've never quite understood it. Why does each system use both ethernet and fibre?
 
slowbiznatch said:
From the way I understand it (I'm not a software engineer so I don't get involved with this stuff) the ethernet connections are used to access the nodes (i.e. ssh, send commands, etc.) and the jobs are run across the fibre connection.
Click to expand...
fibre = storage
 
make sense to use the fibre for networking. but i still don't understand the need for the ethernet. can they not use the fibre to do the same things, or is it to keep that traffic seperate from the work traffic?
 
Maybe redundancy?
Or maybe because the RJ45 gets saturated during the message passing between nodes?
I am really curious about this as well.
 
Perhaps it is to keep costs down for aspects of the network that don't need the performance that fibre allows.
 
but wouldn't it cost more to purchase those cisco core switches than to just pass the traffic on the fibre?
 
You must log in or register to reply here.
Back
Top