Welcome to InfiniBand...

[awsiemieniec]

Ok, InfiniBand 101 question here. I have two servers, both with InfiniBand cards in them. I know traditionally I'll need an InfiniBand switch. But can I get by with direct connect of the two servers to each other (bypass the switch) or does the switch need to be in place? Kinda like with Ethernet when you have a cross over cable... (?) I have a switch coming but would like to start this connection w/o having to wait.

Does InfiniBand *requre* a switch between devices?

Thanks!

edit: whops - can't edit the title to correctly spell "InfiniBand"

 

[Aluminum]

No, you can P2P infiniband, I recommend doing it that way for 2-3 computers. You use the same cables, for 2 close computers a passive copper DAC is like $20.

You will need to run a subnet manager on at least one computer (ideally 2) fortunately OpenSM is pretty simple and available for most OS as a daemon or service. After that you can use IPoIB to get the familiar TCP/IP setup you would have with ethernet, unless you have extreme hardware it will be faster than your drives by a lot, even most SSDs.

Also keep in mind many protocols (older SMB etc etc) have shit tuning that most people don't know about because they are stuck at 1Gbps, look there first (and second, third, maybe fourth) if performance is not what you expect.