• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Question about "cascading" with supermicro cases

R

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,223
Been looking at a solution for the future for a SAN. My biggest challenge is finding a card of sorts that allows me to connect so many drives without the card costing more than the server itself. Even the 1k cards only have 16 ports, I need 24.

Then, I found this:

http://www./products/?sku=77587&vpn=BPN-SAS2-936EL1&manufacture=SuperMicro

From what I understand it has a SAS expander built in so I wont need such a large card, and it has cascading. With cascading does it mean I can buy multiple of the same case and it will automatically add more drives without that case needing it's own motherboard/cpu/ram/controller? Or am I understanding this wrong?


I also have an old IBM SAN enclosure and other than the fact that the drives are proprietary, I love the way it works. Fiber channel straight to the PC and the PC sees all the drives. Is there such enclosure out there that would work the same way? Anything to save me from having to figure out how to connect 24+ individual drives myself. That seems to be the biggest challenge with building these high data density setups.
 
SAS expanders is a way to go especially if you use a hardware Raid controller ex with Windows.
I would use SAS disks with expanders and not Sata ones.

But you should now think about your future SAN solution first and decide the hardware afterwards.
If you use a solution with software raid like any ZFS solution, there are solutions that are faster and cheaper with Sata disks.

Example
My newest Storage and Backupsystem with a RAW capacity up to 200 TB is a pair of
- Chenbro cases with 50 disk bays (no expander)
- Supermicro Socket 2011 mainboard, Hexacore, 64GB RAM+, 7 pci-slots
- 6 x IBM M1015 SAS Controller (about 120 Euro each, LSI 9211 IT mode)
- 10 GBe Intel Nic

Hardware cost about 5000 Euro each without disks (can use cheap 4 TB Sata disks)
NAS/SAN OS: ZFS and OmnIOS, OpenIndiana or Solaris 11

Connectivity: can be filebased (SMB, NFS) or blockbased (iSCSI, FC)
 
I'm planing software based, I don't need a raid controller, I just don't know where to find a card that allows me to connect many drives at once. What do I need to be searching for exactly? HBA? Expander? Most of those that I find only have capacity for 4 drives via 1 SAS port. Most motherboards only have a handful of expansion slots so it's not like I can buy 4+ of these cards and have them work in one system.

Basically I want the cheapest way to plug 24+ drives into a single computer system. I'll let the OS do the raid. Probably use ZFS or Linux MD raid.

I'm still debating if I even want to go the SAN route and just stick to putting storage on the server level but the nice thing with going the SAN route is I can get non hot swap servers rather cheap, even a new Dell that has no hot swap drives goes for really cheap. I'd just have to put in a SSD and I'd be set since the storage would be on the SAN.
 
Your cheapest way to go is probably:

Rackable Systems SE3016 (search Ebay)
16 Drive Jbod enclosure *with* built in SAS Expander
Has SAS wide *in* and *out* ports

Use with LSI SAS 9200-8e or LSI SAS 9205-8e HBA

Get two enclosures and you can connect 32 drives using a single HBA (HBA will support something like a total of 1000 drives).

Cost:

Enclosures: $200*2 = $400
HBA: $300
Cables: $60

Thus: ~$750

Because the enclosures have the SAS *out* port, you could easily daisy-chain another 32 drives by picking up another 2 enclosures and cables...this would NOT double throughput as you'd be constrained by the approximate 1000MB/s throughput of each of the expanders in the original enclosures. However, you could just as easily get a second HBA and connect the enclosures to that. I can confirm that Hitachi 4TB drives work in these enclosures.

Downside of these enclosures is that they are only 3Gbps, NOT 6Gbps..do NOT have redundant power supplies, and are in general a bit 'ghetto' (e.g. latch mechanisms can be a bit flaky...status leds can be hard to see).

If you need a 'professional' class enclosure with built-in expander, the cheapest way to go is the Supermicro SC847-JBOD ... this gives you 45 drives in 4U with TWO 6Gbps expanders (one for the 24 drives in the the front, and one for the 21 drives in the back. Because you have 6Gbps uplinks to your HBA (assuming you are using all 6Gbps drives) you can get about 2000MB/s *per* expander. Additionally, if you connect both SAS wide ports of each expander to your HBA (you'd obviously need two HBAs to connect all 4 ports), then you able to achieve an x8 wide port for your HBA uplinks...unfortunately this does not double your throughput, but does seem to increase throughput from 2000MB/s to about 2700MB/s.

Each expander has an *out* port for daisy chaining, so you can also cascade *many* enclosures off a single HBA SAS wide port. Enclosure also has redundant hot-swap power supplies.

Downside: They are $1800 ... so more than 3x per drive bay than the SE3016.


Another option is the Norco 24 drive enclosure...see the official Norco owners guide in this forum.
 
The supermicro route seems good, I don't mind spending a bit over 2k if it's something that is commercial grade. My biggest thing is I want something robust with redundant PSUs and don't want to need to buy like 6+ sata cards considering the amount of slots it would take up on the motherboard.

I found it on NCIX, is this the right one? Does not mention jbod though...

http://www./products/?sku=55977&vpn=CSE-847A-R1400LPB&manufacture=SuperMicro

And two of these cards:

http://www./products/?sku=37999&vpn=AOC-SASLP-MV8&manufacture=SuperMicro

And then mobo/cpu/ram combo, and I'm set, right?


So when a case says it's jbod is that what it means, that it has a sas expander so I don't actually need 1 port per 4 drive?


Edit: I just stumbled on this: http://www.tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=1369003&CatId=109

Finally, a card with more than 2 connectors. This could work to make the disks show up individually right? Or do I HAVE to use the raid? I had bought an 8 port (well 2 connectors) adaptec card a while back for my server and could not use it as I could not simply just make the drives show up, had to make a raid volume.

I guess I'm probably jumping the gun planing this out given I don't plan to do it for a while... stuff changes all the time.
 
Last edited:
Weird seems NC IX is being blocked again. The issue is the stuff that's easily available in the states aint in Canada so I can't find most of the stuff people link to, here. That case I cannot find on any Canadian retailer.

I'm only wanting to use sata drives so don't need the dual-pathing.

What about that card I posted, that link should work:

http://www.tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=1369003&CatId=109

Could I put this in any case and use that to make all the drives show up?
 
You must log in or register to reply here.
Back
Top