We also use infiniband extensively. It is used in every serious wall street / financial firm in the world like mine, as well as a lot of HPC environments.infiniband really??? WHY!?!?!?!?!?!?!?
We don't care about the cost savings. The latency difference (especially due to RDMA) is huge. RDMAoE/iWARP isn't quite there yet.
BTW -- nice stuff spyder. I've been looking at this thread from time to time. You seem to be a jack of all trades.