• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Suggestions for high-end SAS disk setup?

U

unhappy_mage

[H]ard|DCer of the Month - October 2005
Joined
Jun 29, 2004
Messages
11,455
As seen in MP systems, I'm building a high-capacity, high-availability disk-to-disk backup server, and I'm looking for experience with expensive toys like this. Here's my current build:
Maxtor Atlas 10K V SAS, model 8J300S0 - dual-port SAS drives, 300GB, $800 (3 for a start)
Seagate Barracuda 7200.10, model ST3750640AS - single-port sata drives, 750GB, $350 (2 for a start)
Boot disk * 2 - not sure about these. Two in raid 1, either SAS or SCSI.
Supermicro enclosure/case - dual SAS expanders, 15 hotswap SAS/SATA bays, 760W redundant power supply, $1050 or so.
LSI MegaRAID SAS 8408E (~$800) or 8344 (~$700) - two options. First option is to connect both ports of the first card to the two SAS expanders, second is to connect the one internal port of the second card to one expander, then later get another one and connect it to the other expander. The first is more redundant now, the second is even more redundant later, and allows for expansion to an external array box over a redundant connection later. I'm leaning toward the 8344; is there a difference in the raid kernel between the 8300 and 8400 series?

Rest of the system:
ATP memory, 1GB * 2 @ $165
Supermicro H8DCi - pci-e, pci-x, dual opteron, cheap.
Opteron 265 * 1 @ $333
EVGA 6200 @ $55
Eventually I'll move to another 265 and 2 more sticks of ram.

Okay, so there's that. The goal here is massive redundancy - if this machine works out okay, I'll be building another one of the same specs. So, the question is, are there component choices that would be better for uptime?

 
I'm not sure what your timeframe is, but soon LSI will have a single chip, native PCI-E 8-port SAS RAID controller available. This will have higher performance, both due to the native PCI-E connection and a higher performance built-in IOP (that will also support built-in RAID 6 support, since redundancy is your thing..). I'll have to check and see what that will be released for sale.

Also, if you really want speed as well, look into the Fujitsu MAX series SAS drives. They're by far the fastest I've seen.

Also, the RAID firmware used in both the cards you have chosen is exactly the same.
 
That new card sounds like a definite winner. I won't be buying for at least a month to make sure I've got the software end straight. Raid 6 is a definite win.

Density is important for this application. So I'd like to stick with 300GB drives; Fujitsu makes SCSI 300s but no SAS (unless I'm missing something...). At $200, though, the 36GB MAX may just be a good boot disk. A little steeper than say a pair of Raptors, but I'd rather keep WD away from this machine on general principles.

 
i know this is a bit off topic, but im just curious, what do you have against WD? i ask because sometime in the future i plan on building a 16-drive server and WD is the cheapest solution.
 
I'm not aware of any SAS drives that WD makes. That's probably why they weren't included. They're much more of a desktop company than a server company. Prices on SAS drives aren't cheap either so you probably don't want them for a large low-cost array.
 
Anarchist4000 said:
I'm not aware of any SAS drives that WD makes. That's probably why they weren't included. They're much more of a desktop company than a server company. Prices on SAS drives aren't cheap either so you probably don't want them for a large low-cost array.
Click to expand...
Exactly. I don't mind using WD stuff for home stuff, but they don't make any enterprise drives (no, the Raptors don't count). I want utter reliability out of this, which is why I'm going SAS rather than sata in the first place.

 
I see that you stated density is paramount for this application and the Seagate 750 is certainly unbeatable so far. However, if I may suggest an alternative: the Seagate NL35 enterprise class drives. True you would lose a bit of space since the largest they make so far is a 500GB but you might get another boost in reliability (on paper at least).


http://www.seagate.com/products/enterprise/nl35.html
 
Good suggestion, I didn't think of using nearline drives and it's a really good idea. Oh, and the Barracuda ES series does go to 750. ZZF even carries them. OOS, but they'll be back sooner or later, I hope.

 
Hmm I did not even realize that Seagate made other SATA enterprise drives other than the nearline series.

What is the difference anyway between the ES and NL35 series anyway? All I can find on Seagate's site are marketing descriptions.
 
UICompE02 said:
I'm not sure what your timeframe is, but soon LSI will have a single chip, native PCI-E 8-port SAS RAID controller available. This will have higher performance, both due to the native PCI-E connection and a higher performance built-in IOP (that will also support built-in RAID 6 support, since redundancy is your thing..). I'll have to check and see what that will be released for sale.

Also, if you really want speed as well, look into the Fujitsu MAX series SAS drives. They're by far the fastest I've seen.

Also, the RAID firmware used in both the cards you have chosen is exactly the same.
Click to expand...


Anymore info on this LSI card? Would love some price and other info on it...
 
Mellman said:
Anymore info on this LSI card? Would love some price and other info on it...
Click to expand...

Well, it's based on the SAS1078 Single Chip RAID Controller, of which I can tell you any sort of technical information you want (within the bounds of things that have already been publicly released). But, I'm in engineering, not channel marketing, so I'm not sure about the price it will be sold at until I actually see it in a store (even though I've been playing with these chips for the past two years...)
 
Well, the 1078 has been discussed since January, and Fujitsu is going to use them, but that's all the information I can find. Other than the LSI guy who submitted the entry to the Linux PCI ID database made a mistake, and Google likes finding the various mirrors of the PCI ID database. So all the information I have is that it's pci-e native, SAS, and raid 6. And maybe includes an IOP333 (but wait, that's bridged to pci-x internally...?). Any information at all that you can give us is welcome. I can't find *anything* about it on LSI's website, or google.

 
unhappy_mage said:
Well, the 1078 has been discussed since January, and Fujitsu is going to use them, but that's all the information I can find. Other than the LSI guy who submitted the entry to the Linux PCI ID database made a mistake, and Google likes finding the various mirrors of the PCI ID database. So all the information I have is that it's pci-e native, SAS, and raid 6. And maybe includes an IOP333 (but wait, that's bridged to pci-x internally...?). Any information at all that you can give us is welcome. I can't find *anything* about it on LSI's website, or google.
Click to expand...

The 1078 does not use an Intel IOP, it's actually based off of a PowerPC core. Also, there is no bridging of any type, allowing the full PCI-E x8 bandwidth to be used. For caching it has an integrated DDR-2 interface running at up to 667MHz as well. Expect to see this guy showing up in quite a few OEM server designs very soon! I can't get too detailed on the internals (out of legal concerns), but if you have any specific questions, I'll answer them as completely as I can.
 
Very cool! Any idea what timeframe it'll be released in? I'll be building in January, so there's a bit of leeway here, and I could wait another month if I really have to. I don't need a specific date, but are we talking one month? three months? six months? I'll definitely buy one of those if they're available at the time I'm buying.

 
Any updates on this? This page is still on the first page of "lsi 1078" at google :p
 
HP has SmartArray adapters available now using the SAS1078 chip. Of course they use HP's SmartArray RAID stack instead of LSI's MegaRAID, but that's up to you to decide if that matters (I think they're both great, but that's just a personal opinion -- just make sure they're supported by the OS of your choice)

Newegg has some.
http://www.newegg.com/Product/Product.asp?Item=N82E16816401016
Has two SAS x4 internal connectors, 512MB of DDR2-533 memory and a battery backup.

I still haven't found any information about when the LSI/MegaRAID version of these cards will be available beyond "any day now"
 
Thanks for the links! I'll wait for the LSI-branded versions, but it's nice to see there are cards out now.
 
Noticed LSI now lists the 1078-based adapters (here and here). They're quite cool-looking. Pci express x4 or x8, 500 mHz processor, DDR2 onboard.

I applaud LSI for catering directly to the "adult video" market - to quote the brochure: "the 8708ELP delivers impressive performance numbers – up to 900MB/s throughput – allowing steaming video" :p

One more question I'd like to address. I emailed Supermicro tech support about the case I'm considering and they said an LSI 3442X would work with it. That card is pci-x, doesn't do raid in hardware, and those are two things I don't like about it. However, it has a different connector type than the new cards - 1 SFF8470 for the 3442X, versus 2 SFF8087 connectors for the 8888. The case in question has two LSI SASX28 sas expanders, and I'm not sure what kind of connector it has - the manual (skip to page 44) shows six seperate SAS connections, 3 primary and 3 secondary. Since it is a 28-port expander with 15 drives on it, does this mean 12 "upstream" connections are available? Would it work with only 4 used? What kind of cables should I buy to connect the 8888 to the Supermicro chassis if indeed they will work together?

Sorry for the long discombobulated post. I'm a little confused as to how all this stuff works together, and I think it shows. :eek:
 
I've always thought the remaining "extra" ports a waste as most enclosures do not have a chain port/upstream connection.

As I understand it you must use a fanout expander if you want to use more than one array and the so called port expander included in the enclosure is really an edge expander.

So, you can do it two ways:

SAS Card > enclosure/edge expaner (1 to theoretical max of 128 drives)

Or:

SAS Card > fanout expander (1 to a max of 128) >> enclosure/edge expaner (1 to a max of 128)

Max possible = 16384 drives, or 12,288TB using 750Gers!

My understanding anyhow, let me know if you see any 128 Port fanout expanders on rebate at Circuit City :D
 
Hmm, you're right, it's not clear from the manual what type of SAS ports those are and how they're connected. We have a few of those SuperMicro cases sitting around in the lab at work, I'll take a look at one in person and see if I can figure out how everything is connected... If not, well, I know the person that helped SuperMicro with the integration of the SASx28 onto that board, so he can give me some info for sure.

My guess just from looking at it is that there are 12 ports from the host controller to the backplane (3 x4 sas ports), then 15 target drives. I would guess that the other three ports use the dual-porting feature of SAS and are there for fail-over if something happens to the first controller. That's just a guess and I'll double check, like I said.

Keep in mind that the 1078 only supports wide ports up to 4-wide. So to take full advantage of this backplane, it looks like you'd need three separate controllers (or six if you want fail-over on the secondary port!) Though, yes, you could definitely use the 1078 with this. In fact, you could connect just one port and still be able to talk to all 15 drives through the expander.

The 3442X actually does have hardware RAID (well, more correctly, firmware RAID), but it's fairly limited in that it only supports certain RAID 1 and 0 levels and only up to two volumes with up to 8 drives total allowed between the two volumes. We don't want it to compete too much with the MegaRAID controllers, after all (well, that and there are some hardware limitations you run into with the non-full-blown-RAID controllers, it's not all marketing, I swear!)

Nice catch on the Steaming video... Heh, I wonder how that one made it through, but hilarious nonetheless.
 
So I just checked out the Supermicro box. My suspicions were right. There are two ports for each of the target drives. This allows for failover from primary to secondary. Also, I noticed a handy little cable that goes from the internal SAS connector on the backplane to the PCI bracket, which provides an external SAS connector for the expanders to cascade out and communicate with other external backplanes.

All the connectors on the backplane are internal iPASS (or iSAS or miniSAS depending on who you ask) connectors. That is, SFF 8087.
 
Sweet! Thanks for checking that out, it's a load off my mind. :cool:

I've got another project enqueued for the month of January now, so it'll be a little while before I can get this started. But now I know what to buy. I am confident and unafraid :D
 
What software are you going to use for the disk-2-disk backups?
 
You must log in or register to reply here.
Back
Top