Does this look good for a storage server?

Well, the Norco 4224 is ATX but it holds my X9SCM fine - it should work as long as the standoffs are in the same position. The case me even allow you to move the standoffs around. Grab the X9SCM manual and look at the hole positions and check with the manual/diagram for the case you're looking at, and hopefully they should match!

As a datapoint, check out this thread - note that the .sig of the original poster lists the X9SCM and an SC846 case: http://lime-technology.com/forum/index.php?topic=24697.0
 
BTW you should read that whole thread - the OP looks like he got almost the same case you linked to in your previous post. Note the performance limitation that might be in effect, accessing 24 drives through one expander. But that probably won't be a big deal.

EDIT: It still isn't clear to me from the SC846 manual ( http://www.supermicro.com/manuals/chassis/tower/SC846.pdf ) if it has an expander. There is a chapter for it, but not sure if that applies to the model you linked, as the manual seems to cover a few variants. So if it isn't using an expander, then never mind! Either way, it looks like a great case - at $1200 it better be! hehehe
 
Yeah I think the last one I linked does not have the expander, just 6 ports. I originally linked to the one with the expander though. I may as well do this right and go with the separate cards.

Come to think of it, I have built PCs where I used a micro ATX board in an ATX case, so pretty sure the standard is to provide the holes for mATX too. It's intersting how the Micro ATX boards tend to be the ones that have the more useful slots.
 
Last edited:
For some reason when I bought the IBM cards the other day the purchase never went through. No ebay confirmation email, not showing up in profile, no money transferred from account etc... weird. So re bought again. Better deal this time!

http://www.ebay.ca/itm/IBM-M1015-Se...sk_Controllers_RAID_Cards&hash=item485523fdd8

Just a heads up, the Ebay item you have linked to is a RAID 5 feature key for an IBM M1015 and not the card itself. These are a small module that plugs onto an M1015 to activate the ability to use RAID 5 on these cards.
 
Ah crap seriously? I had a feeling that deal was too good to be true. I wish that first order had just went through properly, since now there's not enough of them to place it again.
 
Glad you got it sorted out :)

Slowly putting together a home ESXi/storage server myself, buying a few bits here and there as funds allow, aiming for the following:

Norco 4224 (maybe Xcase RM 424 Pro)
Supermicro X9SCM-iif
Xeon E3 1230 v2
4x 8GB ECC RAM (probably samsung M391B1G73BH0-CK0)
Samsung 840 Pro 256GB (VM OS drive)
3x IBM M1015 Flashed to latest IT firmware (vt-d pass-through to the FreeNAS VM)
12x WD red 3TB (running ZFS raidz2 or raidz3, future upgrade to 24x drives)
Intel PRO/1000 PT quad port NIC (vt-d pass-through to pfsense VM)

Have bought or already have:
X9SCM-iif
3x M1015
Intel PRO/1000 PT

Hopefully soon buying, PSU, CPU, RAM, SSD so i can atleast get things powered up to have a play/test the hardware I have bought.

Will be running:

ESXi 5.1
FreeNAS 8.3.1 (Home media & general storage server)
pfSense 2.0.2
Windows Server 2012
Assorted proof of concept/testing VMs

All good fun, keep us up to date with how you get on and any questions that come up along the way.
 
I just checked with Supermicro and the case DOES support mATX, so I'll switch back to the MBD-X9SCM motherboard. Updated the list in the OP. I think I pretty much have everything in order now, so think I will wait a few more pay checks and then order. I don't want to wait too long though as stuff changes all the time. Way I see it is this is an investment for the far future since I'll have this case for a very long time I'm sure.

As long as they don't come up with another standard to make Sata obsolete lol.
 
Well everything but the case is ordered now. I want to wait till I'm done with the basement before I start on this (don't want to rack an expensive brand new server in a dusty under construction environment) but with how motherboards and stuff change all the time figured I'd get at least that stuff ordered. The 3 IBM sata cards are on the way too.

Also in the process of switching banks so my money is all over the place so I want to wait till everything is in order before I make a big purchase. All the stuff but the case came up to a bit over 1k. Then the case is another 1k. Not too bad though, 2kish for a 24 bay powerful file server. Can't wait to put this together. Then I'll have to start getting drives for it, but I'll be broke for now lol. I'll most likely move my existing raid array to this server so my main server no longer has to process it. Probably as simple as just moving the drives over and then deciding between NFS or iSCSI. I'm thinking NFS.
 
Well it's official! I just pulled the trigger!

All this stuff put together is pretty much the cost of my car. :eek: But it will be worth it. Have not treated myself in a while. :D

Now to pray I don't have any DOA parts and that everything "just works"(tm) after spending all that money.
 
Most of the parts came in now, just waiting for the SSD bracket since I did not realize I had to order it separately. The system is in a state where it can POST since I got the ram and motherboard the other day (kinda important eh) and overall things look promissing. Was able to boot off my virtual cdrom external drive. (this thing is awesome, you can load ISOs and boot off em).

Ran into an annoying snag though. my SFF8087 cables are too short! So I'll have to order longer ones. NEver even took length into consideration when I bought them.

Speaking of those cables, is is there an easy way to know what order to plug them into the cards? I want the drives to be in order, I don't care if it's up down, or across, I just want them to be a consistent order. Or do I just plug them in any way and not worry about that?

I also did not get to the point of flashing the IBM cards yet, guess I can probably do that while I wait for the longer cables. This is an exciting build, anyway. But expensive. I went way over budget with that. Between this, and trying to troubleshoot a video issue with linux on my PC, my credit line is like at 2k right now. :p
 
All good fun indeed, got some of my kit now too, set up in a temp spare case so i can test hardware and have a play around. Essentially just missing the proper case and bulk storage drives.

You might have seen it already (or something similar) but here is a really good guide to flashing the M1015 on an X9SCM (also works on the X9SCM-iiF which I have, have done it to all 3 of my M1015 already). You cant just boot to a DOS disk and do the whole thing as you will likely get some resource conflicts and/or a "PAL Error" when trying to flash the card.

http://lime-technology.com/forum/index.php?topic=26774.0

Have a good read of that, there is a section specifically headed "M1015 Cross Flashing from EFI Shell"

Hope it helps, let us know how you get on :)
 
Thanks reading through that now. Also, how long does this process take? Wondering if I need to worry about running an extension cord to my big UPS or if my main UPS will be fine. (5-10 min runtime). Juuuust in case the power goes out. That would be my luck.

So in my case I'm guessing I need to do the EFI shell method?
 
Wow this seems pretty complicated. So may ifs and buts. Getting late, think I'll have to do this later when I'm more awake. It looks like I have to update the motherboard bios too? Yikes. This is scary stuff, had I known I would have looked at alternative cards where I don't have to take so many risks.

Also anyone know how I can make a DOS boot USB stick from within Linux?
 
Dont remember exact times, but you are looking at less than a minute (probably less than 30s) for each of the 2-3 steps that write to the M1015.

There are a few steps to do for each card but I found it really quite easy, once you pick the relevant parts out of that forum guide there are only really a few steps to doing each card (dont follow this list incase i missed something, refer back to the guides :) ):

Boot to USB
2x megarec commands

Boot to EFI
Start full EFI
Flash Card
Set SAS address

Done

Didn't have to do the BIOS on the motherboard myself as my X9SCM-iif came with the latest, however BIOS updates these days are pretty painless and I've only heard of issues through powerloss on the whole. Also the Supermicro boards come with a BIOS recovery mode should the worst happen and a flash fail (Appendix C in the manual).

Get all the stuff together you need along with consolidated notes of what you need to do on your cards (saves getting lost in that rather large forums post), take it steady and you should have no issues.

Not sure on making a suitable boot USB in Linux im affraid, I'm mostly a windows guy and picking up FreeBSD etc for the first time to run FreeNAS etc.

Good luck, any questions feel free to ask and I'll help where I can.
 
I think my X9SCM is still on BIOS 1.2. Never felt the need to go to 2.0 (and read some horror stories about that update - I don't know if that is still an issue now).
 
Without thinking, I went and plugged in 3 of the SAS cables (waiting for the other 3 to come in)... now it wont boot, it just sits at the boot screen of the card. Can I still flash with the cables plugged or do they have to be unplugged?

I can not get them to come out anymore, it's like if they are fused there. If I pull with any more force the port will rip out from the card. I'm pushing down on the little tab.

Still trying to see how I can make a DOS disk from within Linux, and I will bite the bullet and do this.
 
Yay managed to make a DOS boot USB stick. 1st step done... wow, that was a pain. Ended up having to boot in a windows partition and use a 3rd party utility.

Still can't get those cables out of the card but managed to remove them at the backplane. Hopefully I never have to change those cables because once they go in the card it's a done deal it seems.
 
Ok, I got all the files together, managed to make the dos boot disk, so is this all the steps I need? (going to skip the bios update and hope it works)

<Boot off DOS USB stick>

Type in the following exactly:

megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0

<reboot, go in UEFI boot>
FSx:
execute: Shell_Full.efi


sas2flash.efi -l Flashlog.txt -o -f 2118it.bin -b mptsas2.rom
sas2flash.efi -o -sasadd 500605bxxxxxxxxx
<reboot>

Done!
 
Odd about the cables, they certainly should come out, i've never had any issues with removing SAS cables from any cards, should just push on the tab and slide out with minimal force. If you look closely there will be 2 small holes on the metal cable surround on the card, 2 small hooks on the cable clip fit into these holes, I can certainly understand how these might get a bit caught up.

Other than your addition of "-l Flashlog.txt" which seems entirely reasonable, that is exactly what I have written down on my notes for when i did the job :)

Of note, if you do not want the OptionROM on the cards you can remove the following:

-b mptsas2.rom

So you would have:

sas2flash.efi -l Flashlog.txt -o -f 2118it.bin

The OptionROM allows you to boot from a drive(s) attached to the card, enter the BIOS settings during boot, this is usually where you can set up RAID arrays, cache settings, set boot arrays and the like, all fairly redundant on an IT mode card. It also increases boot time by 5-10 seconds or so per card. In reality it serves no real purpose on an IT mode card short of taking time and displaying attached drives during the boot process. With or without is entirely personal preference tbh.

I omitted the OptionROM from my cards with no issues.
 
Have had a quick "play" with some SAS connectors here at work and I think I can see the issue you might be having getting your plugs out.

If you try and pull the plug out, even a little before pushing the tab down the 2 little metal hooks can easily get hung up in their holes, if they do pushing the tab doesn't disengage them properly it just flexes the tab and no amount of pulling (short of breaking something) will get them to move.

Your best bet is to push the plugs into the socket, push the tab down and then try to slide the plug out (a small wiggle as you do this can help).

Failing that you can take the card out and use something small to poke the clips back into their holes as you try and get the card out, they are literally the only thing holding the plug into the socket.

Good luck :)
 
Good to know I'm on the right track, I will do these commands today and hope for the best!

I finally did manage to get the cables out, I had to use a flat screwdriver to slightly pry the top of the port up as it was not disengaging the hooks. I'm thinking the ports were maybe slightly bent inwards so when it clipped, it did not have enough play so when I press the clip the hooks go down far enough.

My other 3 cables are on the way, they're in Toronto area so should be here by tomorrow. Hard drive bracket came in late yesterday so I should be able to go pick it up today. slowly coming together. Annoying how they keep sending one box at a time though. I pretty much had to drive to Purolator for each and every part individually. :p
 
Neme - thanks about the trick with the optionROM - I didn't realize you could leave it out! I probably flashed mine with the optionrom included. Optionrom can be a big pain because some motherboards won't have enough space in the BIOS and could cause the boot to hang. I'll have to remember for next time if I ever use the cards in a different machine.
 
If I leave that option out, I can't boot off the drives though right? Not really an issue for me though as I have a SSD to act as boot drive and it will be in the chassis.
 
Neme - thanks about the trick with the optionROM - I didn't realize you could leave it out! I probably flashed mine with the optionrom included. Optionrom can be a big pain because some motherboards won't have enough space in the BIOS and could cause the boot to hang. I'll have to remember for next time if I ever use the cards in a different machine.

No problem, learning this stuff as I go myself, just been reading many many forums/blogs/guides :)

If I leave that option out, I can't boot off the drives though right? Not really an issue for me though as I have a SSD to act as boot drive and it will be in the chassis.

That is correct, no OptionROM will mean no booting from any drives on that card.
 
This is weird... apparently I don't have enough memory to run the flash tool? I do have 8GB...



Do I need to add anything to autoexec.bat to allocate more memory or something? Have not dealt with dos in a long time. Kinda feels nostalgic, in a not so good way. :p

Gotta like Microsoft's solution: "Close some files and programs you are not using, then retry the operation." LOL
 
Got it!

Had to add this line:

device=c:\dos\emm386.exe 2048 ram

In config.sys. 2048 is just an arbitrary number I chose, chances are I could have gone much lower.
 
Hmm, bummer.

Didn't get any such issues when i did mine, have searched around a bit and the only usefull reference I can find is here:

http://forums.laptopvideo2go.com/topic/29166-sas2108-lsi-9260-based-firmware-files/

Post #19 seems to indicate not all DOS USB boot images are created equal...

Personally made mine using the program suggested in that long thread I posted with the guides on it "Rufus_v1.3.2" though this does rely on having a windows box you can run it on.

I could certainly knock up a fresh USB stick and check if there is anything funky going on in the Autoexec.bat or Config.sys files (been a while since I've had to mess with these anoying files :)).

Anyway off to bed for now... getting late in the UK. Let me know if you want me to try the above for you.

Edit: Ah ha, seems you beat me to it :) nice going.
 
Damnit why do things never work out smoothly for me. Another error:



I only have one card in the system, I did not want to mess around trying to do 3 at once. Know the fix for this? I'm googling as well. Thanks for the help btw.
 
Just as a quick thing to try:

MegaRec -adpList

This should list all the adaptors it can see in the system and their IDs I believe, cant down my server to try it at the moment.

Really off for sleeps now... good luck, i'll catch up in the morning.
 
Yep it showed that it found one.

I'll try the other cards and see if I have better luck, maybe this one is bad?

Edit: does it on all 3, tried different slot for shits and giggles.

Going to try to update the bios as suggested, maybe that's why it's not working. Though I don't think this step has anything to do with the bios, that's for the UEFI step.
 
Last edited:
Similar error on another system.




Also, two of the cards have the exact same SAS address. Will that cause issues if I even do manage to get this to work?

I'm leaning towards another card. Any other cards that will do jbod, perhaps without needing to jump through so many hoops?
 
Got it!

Had to add this line:

device=c:\dos\emm386.exe 2048 ram

In config.sys. 2048 is just an arbitrary number I chose, chances are I could have gone much lower.

Holy crap, memories. Good lord it's been a long time since I've messed with a config.sys file!
 
Haha I know eh. Funny thing is, I don't think I've EVER had to do that before. For some reason my gut feeling told me it was something I had to do with that file so I googled config.sys and memory and found that lol.

It amazes me that even in 2013, tools are made to depend on DOS lol. Do companies hate Linux THAT bad? And they think they can get support for DOS nowdays? lmao.
 
Bah this blows. I have the worse luck. Nothing ever works. Don't get my started with my workstation build which I've been troubleshooting for over a year.

Another error.... while trying to update bios and running AMI_2.BAT as per instructions.



I'm starting to feel really stupid for spending all this money. Did not realize it was this complicated. I thought everything would just be plug and play.
 
I'm starting to lean towards ram. Memtest (old version I forget) wont run, it just reboots. Another version runs flawlessly, newest version, memtest86+ just freezes right when it starts to load. Time to spend more money.
 
This is the closest thing I could find that matched something in the compatibility list. There's only like 3 modules in the list on the SM site so it made it very hard to find ram that will match.

Kingston ValueRAM KVR1333D3E9S/4G 4GB 1X4GB DDR3-1333 ECC 240PIN CL9 w/ Thermal Sensor Memory

Hopefully this works. If it's even the ram that's the issue. This server is hitting close to 4k. Almost better off just buying a SAN if these issues continue. Getting expensive.

If this does not work I suppose I can pay someone to flash the cards for me.
 
Bummer, as you say I suspect something more basic than card flashing issues may be at work here, given your problems running basic diagnostic tools like memtest86. I assume you have tried running memtest with only basic hardware installed (e.g. CPU RAM etc no HBAs)?

I know the supermicro boards can be a bit funny about RAM, while im not running the exact same board as you (X9scm-iif vs X9SCM) they are pretty close, im running 4x "Kingston® ValueRam 8GB PC3-12800 DDR3 ECC RAM". Reports are generally good with Kingston Value RAM in these boards. Did a 24hr soak in Memtest86 with no errors.

I would offer to flash the cards for you, but I suspect you can find someone closer and cheaper to get them to/from than someone in the UK :)

Hope you get this nailed soon
 
Yeah everything I've been doing so far is with either only 1 HBA or nothing at all, and 1 stick of ram only, in the proper slot as per documentation. I'm going to mess with bios settings and see if I have any luck but then call it quits for today and hope that ram comes in soon.

The Purolator lady thinks I'm building some kind of super computer since I'm going there almost every day to pickup parts LOL.
 
Back
Top