Need some advice on DAS and HBA cards

[MrCrispy]

I want to use a SuperMicro 24 bay chassis as a DAS. It will be used with a server on which I want to use unRaid. The specs are as follows -

SuperMicro:
846E16-R1200B chassis
BPN-SAS2-846EL1 backplane
it also came with a LSI 9223-8i card

server:
HP Elitedesk 800 G4 SFF

I'm new to these things. I did some research and have a few questions -

1. the HBA goes in the HP, and then connects to the 8087 port on the backplane. since its a SAS expander one port/cable can control all 24 drives?
2. with the 9223-8i, there are no external ports, so I'd need an adapter which converts it to external 8088 like this - PCI adapter 8087-8088, and then a 8088 cable between the 2 machines?
3. the other option is to buy an HBA card that has external ports. There are lots of LSI cards. e.g. 9207-8e, 9207-414e, 9205-4i4e. The newer HBA cards are going to be more expensive and support more expansion and higher speeds I guess, but are they needed for using regular SATA hdd's? Which one do you recommend?
4. also do these cards need active cooling?

whats the best route here?

 

[Grebuloner]

1. Yes. If the expander has additional input ports, more bandwidth can be realized by using 2 cables.
2. Yes.
3. You're fine with the card you have and the internal-external adapter. You'll be more bottlenecked by 24 drives into a 4 (or 8 with 2 cables) drive port. There's not a substantial difference between 9200 series cards, buying a 4e or 8e card would only be visually cleaner.
4. If you are going to have heavy constant use, a 40mm fan is recommended, but for everything else, just make sure there is airflow from the case going over the heatsink.

 

[MrCrispy]

1. Yes. If the expander has additional input ports, more bandwidth can be realized by using 2 cables.
2. Yes.
3. You're fine with the card you have and the internal-external adapter. You'll be more bottlenecked by 24 drives into a 4 (or 8 with 2 cables) drive port. There's not a substantial difference between 9200 series cards, buying a 4e or 8e card would only be visually cleaner.
4. If you are going to have heavy constant use, a 40mm fan is recommended, but for everything else, just make sure there is airflow from the case going over the heatsink.

thanks. About the bandwidth, am I right that a single 8087 cable supports upto 24Gbps (4 lanes) via SAS2? Thats 3000MB/s so with all 24 drives, thats 125MB/s with all drives in parallel theoretical max bandwidth? That should be more than enough for most practical usage? With 2 connectors it becomes 250MB/s which is pretty much max for most consumer SATA3 drives? so do you recommend 2 connectors and thus 2 8088 cables between the machines?

how does SAS3/12G figure into all this? I'm guessing since the backplane is only SAS2 it won't help. I see the LSI 12G cards are about the same price.

and about DAS - does it stay powered on 24/7, does it go to sleep, and wake up when there's disk access (ie. just like a disk)?

 

[Grebuloner]

thanks. About the bandwidth, am I right that a single 8087 cable supports upto 24Gbps (4 lanes) via SAS2? Thats 3000MB/s so with all 24 drives, thats 125MB/s with all drives in parallel theoretical max bandwidth? That should be more than enough for most practical usage? With 2 connectors it becomes 250MB/s which is pretty much max for most consumer SATA3 drives? so do you recommend 2 connectors and thus 2 8088 cables between the machines?

how does SAS3/12G figure into all this? I'm guessing since the backplane is only SAS2 it won't help. I see the LSI 12G cards are about the same price.

Closer to 2400MB/s. The overhead's a bitch, but it's still plenty for spinners and probably higher than your network speed.

If 2 connectors are supported (I'm on mobile and didn't look up the details), depending on the expander/backplane architecture, you could potentially double the bandwidth. If it's supported, I'd do it, especially as cost of cables isn't that high.

I've never had opportunity to test an SAS3 expander with SATA3 drives, so I can't say if it can use the higher link speed to provide more bandwidth or not over SAS2, sorry.

 

[MrCrispy]

this is a pic of the current server. It looks like its using 2 cables from the LSI to the backplane, so maybe I should use 2. The PCI adapters to convert the 8i will cost ~$20 and I can get a used LSI 9205/9207-8e for not much more, maybe thats a better option?

I would also need to remove the current mobo/cpu etc, they would be no use in a DAS. and make the thing a little quieter, so this is going to be a long project I guess.

 

[Grebuloner]

Nice little server you've got there! Nehalem/LGA1366?

I'm on my main rig now and having the picture helps out with some pdf crawling. There seems to be some missing information on the part of SuperMicro (not unusual), but despite two cables present, it seems that only one cable is actually working between HBA and backplane, while the other is for situations where you have multiple expanders and are cascading them. That said, clearly the system worked in this configuration, so using both cables isn't bad. You would be able to see whether the HBA is using just one or both channels in its BIOS menu.

There's no functional difference between the 8i and 8e cards aside from cleanliness of looks and whether losing a PCIe slot or two to the adapters matters to your future plans. Flip a coin maybe? A card in the hand is worth two on ebay?

Removing the old DAS hardware can certainly present its challenges, but I see standard ATX power connectors, so kludging a way to power the system without an internal MB should be relatively easy. You can swap out the nasty server fans and a fan controller will take care of running them if still too loud.

Oh! and on that SAS3 front: Still don't know if it increases total bandwidth when using SATA3 drives, but at a minimum, both the card and expander must be SAS3 compliant. Fun fact, there is a chassis compatible SAS3 model of that expander if you want to spend more money

 

[MrCrispy]

yup, its a X8DTE-F mobo with 2x Xeon L5630 cpu's. At the time I picked this since these were low power versions. Also upgraded PSU to the SQ version.

Haven't really had a chance to use this much, life happened, had to move etc. Today it seems underpowered and a power hog, esp with dual cpus and 2GB dimms, esp compared to an Intel 7th/8th gen cpu and modern chipset.

The chassis, drive caddies and the backplane are still great IMO.

Does a DAS stay powered on all the time when connected to a host? Does it have sleep modes and sleep? I will also be losing fan control and pwm without that SM mobo, as well as IPMI, these things really are amazing. Also will need to replace the fans, there are plenty of posts about that.

 

[Grebuloner]

Does a DAS stay powered on all the time when connected to a host? Does it have sleep modes and sleep? I will also be losing fan control and pwm without that SM mobo, as well as IPMI, these things really are amazing. Also will need to replace the fans, there are plenty of posts about that.

Depends on how the DAS is connected to the host. Purpose-built ones generally will, having additional data connection, but one that is only connected through SAS cables is on its own. Drive spindown will happen if configured in the OS, at least.

(Thinking designs in my head here) If your HP used a PSU with an ATX 24 pin (I doubt it), a dual PSU adapter could be used between the systems to (de)activate the SM PSU with the host. If you are crafty, you could rig a USB cable/device that is plugged into the HP to short the power-on pin on the SM ATX connector. Something may also already exist that you can buy.

Fan control is easy. Use a PWM fan hub, run the cable to a spare connector on your HP MB, then use speedfan to set your curves.

I think IPMI PCIe cards exist, check your manual for compatibility.

 

[MrCrispy]

So I had an idea. SuperMicro makes a JBOD motherboard - CSE-PTJBOD-CB2 for this use case, but its ~$50 and it only has 2 fan headers, no other features. the CB3 version of that with IPMI etc goes for $200+.

Instead, could I use something like this - https://www.ebay.com/itm/203591158474, X8SIE-F ? I wont add a cpu or ram, it will only be used for power on. But it comes with IPMI, and it has more fan headers, and they can be controlled via IPM and PWM.

so now I can do full remote management over IPMI. Do HBA's support the DAS being turned on after the server? so IF my idea above works, I could write a simple script that runs when the server comes on, it will turn on DAS if its off, and vice versa.

 

[Grebuloner]

Never tried a board's IPMI without anything installed. Interesting idea; will be interested in the results if you try it out.

The number of fan headers is not strictly an issue when not using 30W server turbines. You can easily run several regular fans with splitters or a hub off of each header.

Yes, your HBA will absolutely support the DAS being turned on second. Just make sure that drive hotplugging support is enabled, and of course no software/data so critical that your server won't run without it is loaded from those drives. I've made that mistake before...

 

[Dopamin3]

Never tried a board's IPMI without anything installed. Interesting idea; will be interested in the results if you try it out.

You can usually get into the IPMI. The BMC is basically a total separate thing and doesn't require CPU or RAM to be installed. SuperMicro locks a lot down in the IPMI without paying for a software key, but I bypassed on it on the X11SSM-F by following this guide.

I'm not following the reason to try this. But it should power on to the IPMI, you'll see the IP address become active on your network and maybe that can be used for scripting.