Planned Norco 4224 fileserver build, stuck on SAS controllers

Crest

Limp Gawd
Joined
Oct 7, 2009
Messages
433
Hello there,

I've been researching SAS/Sata cards and expanders for months now, I'm been reading loads of topics in this section since January and finally decided to post a topic to ask my specific questions as I can't find a clear answer. I'm planning on building a pretty big fileserver based on a Norco 4224 with FlexRaid. I'm not set on what OS I want to use, but I figured 2008 R2, Server 2012, or maybe the latest of Windows Home Server. Not sure yet. My sticking point has been SAS cards. I want to eventually run 24 drives, but for right now I only have about 8-9 drives I'll be using. Mostly 1-2TB drives. So if that means I need three 8 port cards or a host card and an expander, fine. I just want to be sure I'm getting what I need.

People recommend all sorts of cards and I'm losing my mind with all these different items, but my needs are: I do not care about RAID at all due to my plan to use FlexRaid, and I want/think I need SMART pass through to the OS. FlexRaid monitors SMART and can message me if there is a issue with a drive before it happens. I want to be sure the card I get will allow this. Speed is not really an issue, hence my lack of concern with RAID. The average speed of a single drive is plenty, I plan on streaming movies, music, video files, and doing backups onto the server from 3 machines in the house.

So, what card or combination of cards will allow me to run SMART to the OS from the drives and will support 24 drives? I do plan on using 3TB drives or higher, so that's another concern.

The other thing, is that I eventually might want to get another 4224 and place it ontop and expand into that case for more drive storage. I'd like to figure out which cards I could use/get in order to run external SAS to another chassis. I don't think I'll need that for a while, but it's something I'd like to have. I think I could budget up to $500 for this, but if there is specific reason to go higher, I think I could. Though I don't need something really special, I would think this would be a simple thing to come up with, but I have yet to, so I've come to you guys.

____


On a side note, most if not all of the hardware I plan on using hasn't been chosen as I haven't gotten to a point where I know what controllers I'm using. So if there is a suggestion you have for a card that doesn't work nice with a motherboard or something, then please post it and I'll keep that in mind.

I appreciate any help you can provide. I waited a long time to post because as a mod on other forums I know how annoying it is to have someone ask an obvious question. But I can't find a complete answer that I've been seeking.

Thanks.
 
Hello,

These late days that has been my issue too.
The choose of a good Card, although mine should have some Raid features.

Well, i have been choosing between the most known SAS Cards and those who seem to me as the most reliable and with better features are the Areca ones.

ARC-1880ixl-8
ARC-1882-ix-12

These two cards may do what you need.
Both have 2 and 3 respectively internal miniSAS (8087) ports
And 1 external port for case expansion miniSAS (8088) port.

Perhaps you should take a look at the website for more specifications
http://www.areca.com.tw/products/1882.htm

Both are SMART capable, and also RAID capable.
They use a PCIe x8 interface which 99% of motherboards have.
 
I am not a particular fan of Flexraid (Especially if you are planning on expanding to 40+ drives) but I would suggest reading through the relevant threads here and on their page for the best compatible hardware. That said, for the expander I would recommend going with the HP (you can read the thread on HOCP about it). As to the HBA itself, since you are looking to stay away from hardware RAID, a lot of people seem to have good success with the IBM M1015 http://www.servethehome.com/ibm-m1015-part-1-started-lsi-92208i/ which is a very flexible SAS HBA for all sorts of software raid applications (flexraid, mdadm, zfs etc) and can be found for about $75 on ebay for pulls from existing servers.
 
I am not a particular fan of Flexraid (Especially if you are planning on expanding to 40+ drives) but I would suggest reading through the relevant threads here and on their page for the best compatible hardware. That said, for the expander I would recommend going with the HP (you can read the thread on HOCP about it). As to the HBA itself, since you are looking to stay away from hardware RAID, a lot of people seem to have good success with the IBM M1015 http://www.servethehome.com/ibm-m1015-part-1-started-lsi-92208i/ which is a very flexible SAS HBA for all sorts of software raid applications (flexraid, mdadm, zfs etc) and can be found for about $75 on ebay for pulls from existing servers.

I've been going between FlexRaid, Unraid, and Stablebit Drive pooling since like december. FlexRaid has been what I've always ended up liking more.

What is your issue with flexraid, if I may ask?

I've seen the M1015 mentioned loads of times and I was about to buy one, but information on how many drives it supports, SMART functions, and it's performance when handling this many drives was unclear. I'm reading that link now.

Edit: Alright so for my uses, I should get a M1015, flash it to LSI9211-IT. Then get an HP expander and hook it up. I'll go re-read the HP thread, though it's so big, it's confusing. Is there only one HP expander? Which one do I get?
 
Last edited:
@OP

The M1015 is a good choice, along with the LSI 92xx-8i series, which also support expanders. Taken from this link for the HP SAS expander.

Non-RAID HBA cards
LSI SAS 9211-4i, 9211-8i (SAS2008): YES (dual linking supported on 9211-8i model)
LSI SAS3081E-R: YES
Intel SASUC8I (cross-flashed to LSI 1068e firmware): YES
Supermicro AOC-SASLP-MV8: YES
SuperMicro AOC-USAS-L8i: YES (Confirmed by Tau)
Areca ARC-1300-4X: YES (Not recommended due to mediocre performance and no SMART passthrough)
Adaptec 1045 (1 x SFF-8088): No
Adaptec 1405 (1 x SFF-8087): No

The Dell SAS 6/iR will likely work too, due to the fact that it is based on the LSI 1068e. The newer Dell H200 is also LSI-based, so will likely work too.
 
I assume I should ask these questions in the HP expander topic, but I see two HP SAS Expanders listed.

http://www.amazon.com/HP-468406-B21-24-port-RAID-Controller/dp/B005WJC5ZW/ref=pd_sim_sbs_e_4
http://www.amazon.com/HP-468406-B21...Q16K/ref=sr_1_4?ie=UTF8&qid=1339694177&sr=8-4

What is the difference? Both are 468406-B21.

And along with this, if I bought another Expander later for another case, I could run a 8088 cable from the first expander to the second and it would work? Or do these cards have issues with that? Should I run a 8087 to a 8088 adapter and run one to the second expander separately?

Since I'm not doing high bandwidth operations, dual linking shouldn't be needed or a must.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
They are both the same card. As to why the same seller is selling the same card for different prices, you would have to talk to them. One may include SFF-8087 cables and the other may not for example. As to how to do the expansion, I have NEVER had success cascading expanders and recommend against it. You should put the HBA and one expander in one case, and use a backslot sff-8087 to 8088 adapter (connected to the second HBA port for example), then run that to the external port of the expander in the other chassis.
 
In that case, I would see 6Gbps per SAS cable, or would I technically be getting 24Gbps with all four SATA3 ports running. Again, I'm not a speed freak when it comes to "NAS" storage, but curious how that would work. I figure in my case, having 4 lanes is fine per expander since I'm only accessing maybe 2-3drives at once under most situations.

So, I just need to buy a M1015, and the HP Expander on Amazon for 260. That would bring me to about $350, then probably 8 8087 cables for now since I'd be running a single expander. Glad I finally have a clear idea of what I need.

Thanks so much for clear answers. If you could mwroobel, what concerns did you have with Flexraid.

Edit: I ask now because he's having a sale that ends today. And i've been waiting to buy.
 
Last edited:
BTW Last I checked cables were cheapest on monoprice. If this is wrong someone correct me cause I keep going there to buy SAS cables.
 
Theoretical bandwidth in a system such as this with SATA spinning disks is largely irrelevant, you will never touch 3Gb/s speeds let alone 6Gb/s. As to the cables, you will need 1 8087-8087 cable to connect HBA to expander, and then 1 SAS-SATA fanout cable for each 4 drives (or portion thereof) you plan on adding. As to my issues with Flexraid, I know a few people who tried it over time (specifically the windows version) and had stability issues. I also know some people who had no problems. For windows machines, I tend to prefer HW raid over SW raid, but that would preclude your ability to mix and match drive sizes within a drive group.
 
Theoretical bandwidth in a system such as this with SATA spinning disks is largely irrelevant, you will never touch 3Gb/s speeds let alone 6Gb/s. As to the cables, you will need 1 8087-8087 cable to connect HBA to expander, and then 1 SAS-SATA fanout cable for each 4 drives (or portion thereof) you plan on adding. As to my issues with Flexraid, I know a few people who tried it over time (specifically the windows version) and had stability issues. I also know some people who had no problems. For windows machines, I tend to prefer HW raid over SW raid, but that would preclude your ability to mix and match drive sizes within a drive group.

I figured I'd simply get 8x 8087 since the Norco has internal MiniSAS 8087 connections on the 6 backplanes. Then I might as well use two 8087 to connect to the Expander. Unless you think that's pointless.

http://www.monoprice.com/products/p...=10254&cs_id=1025410&p_id=8189&seq=1&format=2 This is the right 8087 cable, I believe. Their listings are a bit confusing.

Thank you all for the help with this. So glad I have this sorted. Now I need to get my other hardware sorted and see if I'll have any compatibility issues with these cards on particular mobos.

Edit: Does anyone here sell these cards and expanders?
 
Last edited:
Before you buy the cables, talk to whatever reseller you decide to go with because some retail kits with the expander include up to 6 8087 cables with the package. A couple of people sell them here (Check the expander thread). As for the cable length, get the length that makes cable management easiest (ie .5m may be better than 1m or 2m). Wherever you buy, make sure it is the latest board rev (green) and includes the latest firmware (2.05 or later) if you can because you can only update the firmware with an HP SAS HBA. Again, most of the info can be found in post #1 of the expander thread.
 
Last edited:
While you mentioned that this is a large file server, you haven't told us what exactly you are looking to store/do with it. Also, what is your budget? If it's only a few hundred bucks, you pretty much are forced to get 3x IBM M1015.

While I've railed on FlexRAID before, I have forked out the $55 for snapshot + storage pooling, but that's strictly for my HTPC/Media Box since most of the data ia static.
 
If your motherboard has the PCIe slots to handle it, I'd highly recommend the 3xM1015 solution. This is definately the lowest cost approach. All three cards together should cost less than the expander alone (assuming you can find them for about $75-85 on eBay - lately it appears they have been running a bit higher).

Don't worry even if your MB only has 2x PCe x8s and the third card has to go into a x4 slot. Unless you are running SSDs you won't overwhelm a PCIe 2.0 x4 slot with 8 spinny disks.

You noted in the original post that you might want to add a second chassis in the future. Add the expander if/when you actually do this. There is no reason to add an expander now just to make that transition easier. Its just a waste of money - and there will likely be better and/or cheaper expander on the market when you want to pull the trigger.
 
While you mentioned that this is a large file server, you haven't told us what exactly you are looking to store/do with it. Also, what is your budget? If it's only a few hundred bucks, you pretty much are forced to get 3x IBM M1015.

While I've railed on FlexRAID before, I have forked out the $55 for snapshot + storage pooling, but that's strictly for my HTPC/Media Box since most of the data ia static.

The price of a M1015 and the expander is perfectly fine for me. It looked like it would come to roughly $350 including possible shipping. Though as PigLover mentioned, I planned on possibly adding a separate chassis and expanding to more drives. But as I'm starting out with about 8x drives that might not really be a concern right now.

As for what I'm storing, I wasn't sure it was needed. This server is going to be storing backups of 3 computers, all my media that I generate on my computer, music collection of 3-4 people, ripped movies, etc. I also was looking to setup a nightly dump from my machine. I produce content on youtube and depending on the project it could be multiple terabytes. Even though I'm on a RAID10 right now and I have external harddrives, I'd like an automatic dump of my recorded media and project files.

I plan on putting a decently powerful system in it since I plan on using it to transcode media and do general streaming processes for devices in the house and perhaps while I'm out of town. I don't know yet on the specifics of that.

None of the media will be changed frequently, I'll change it a lot at once, but then I'll have it sync late at night probably. Once I fill it up, it will be mostly a matter of reading speed for maybe at most 2 movies, but doubt it.

If your motherboard has the PCIe slots to handle it, I'd highly recommend the 3xM1015 solution. This is definately the lowest cost approach. All three cards together should cost less than the expander alone (assuming you can find them for about $75-85 on eBay - lately it appears they have been running a bit higher).

Don't worry even if your MB only has 2x PCe x8s and the third card has to go into a x4 slot. Unless you are running SSDs you won't overwhelm a PCIe 2.0 x4 slot with 8 spinny disks.

You noted in the original post that you might want to add a second chassis in the future. Add the expander if/when you actually do this. There is no reason to add an expander now just to make that transition easier. Its just a waste of money - and there will likely be better and/or cheaper expander on the market when you want to pull the trigger.


Not sure of what Mobo I'll use but I'll keep that in mind. I won't be buying this until I have some of the rest of my system planned, but this point was a obstacle. So I suppose I could buy two M1015's now. Then go from there. If I need another chassis, I can persue an expander.
 
The speed the sas cable goes will be the speed of the SLOWEST device connected to it. If you have one sata 1 drive, everything will go 1.5gbit, If everything is atleast 3gbit, then it can go 3gbit. I thought all the hp expanders where 3g, can't remember if they started making 6g ones.

And I agree with the other one above, I wouldn't cascade expenders, if you have any way to avoid it.
 
Currenlty running here as fileserver & for client backups.

Norco 4224 (all fans changed to quiet ones)
Cosrair HX750 PSU
Intel Intel S3210SHLX Dual Q/C Xeon LGA775 Motherboard (new old stock)
Intel E8200 CPU
8gb ECC Ram
2 X Broadcom onboard NIC's
Intel PCI-x Dual Gigabit NIC's
M1015
HP Expander
6 X 2tb HD​
11 X various sized HD.​
Total 18TB available on Flexraid drive​
Intel Sata
2 X 1TB Mirroed for O/S​
1 X 2TB (parity for Flexraid)​
1 X 500gb (Server backup)​
WHS2011
Flexraid

Working like a charm so far. Draws 210 watts @ 240v (Australia) from the wall with light use. A smaller PSU would have been more efficient and my be changed later on, advanced power management through the HP Expander does not appear possible at this stage to power down drives not needed, only the ones on the onboard Sata power down when not in use.

Cheers
Jamie
 
I thought all the hp expanders where 3g, can't remember if they started making 6g ones.

All HP SAS Expanders using firmware 1.52 or later support 6g.

Source: HP SAS Expander Owner's Thread first post:

Q: Why do I need firmware 1.52 or higher on the HP SAS Expander?
A: The most notable change in firmware 1.52 brought upgraded SAS/SATA speeds from 3Gbps/1.5Gbps to 6Gbps/3Gbps respectively. Most newer harddisks ship defaulted to 3Gbps, and for compatibility and performance reasons firmware 1.52 or higher is recommended.
 
Back
Top