Changing raid controller in Ubuntu 14.04?

dgingeri

dgingeri

2[H]4U
Joined
Dec 5, 2004
Messages
2,830
I have a server at work that is acting up. The raid controller, a LSI 9261-8i, is frequently dropping all the logical drives offline. (Actually, this has been happening every since it was put in place, and we'd just reboot the thing, but only recently my boss wants to resolve it.) Looking up info, it is a frequent problem with the LSI driver and some of the raid controllers and motherboard combinations cause this issue, and Avago/Broadcom isn't going to work on a driver level fix for this since it is an older controller. So, my boss wants to change out the raid driver with a different one. We're buying a Broadcom 9361-8i controller to replace it, along with BBU and cables.

The problem is that the new controller uses different drivers, and I have no idea how to change out the boot storage drivers in Linux. My boss knows, but is unwilling to tell me, saying I need to look it up to learn it. (He does this a LOT.) I can't find a thing on this through multiple Google searches.

How would I go about doing this? Does anyone here know?
 
You should probably move this post to the Linux subforum.

It's one thing to learn and another to teach, but forcing someone to learn where precious data is involved is bound to end in tears.

I hate pseudo hardware raid cards under Linux for this reason, preferring to use mdadm in the case of Ubuntu (Debian) derivatives for the simple fact that software raid is so much simpler where hardware failure is a problem. Due to this fact it's difficult for me to offer advice, although sometimes you can get lucky and install the hardware (software) raid card, install the drivers, connect the drives in the same order and everything just works - Although there is no way I would offer this as advice where precious data is concerned, your boss is an idiot and should not be putting you in this position. ;)
 
Consideration #1 immediately backup
Consideration #2 very rarely do hardware RAID (especially pseudoRAID) use compatible disk layout....

I know server admins that buy a couple of RAID cards, even from mainstream hardware controllers to mitigate their aweful compatibility.
Unless you need the throughput software RAID is more then viable and is compatible across setups
 
Consideration #1 - this is storage for large projects for our post production group. It was specifically put up with the contingent that it would not be backed up. (Not my choice.) It is actually rarely used, and has frequently been found to be locked up after months of idle time.
Consideration #2 - I have worked with these controllers very frequently, and I know for certain that it is easy to migrate groups from one to another. Disk sets can move from the 9261 to the 9361 without issue, and backwards is only a problem if they used RAID 6 on the newer controller. This works great under Windows, even when the boot drive is on the RAID controller. I just install the newer drivers before the change out, and as soon as it boots up, it re-recognizes the devices, reboots, and all is at 100%. The Dell versions of each work that way well, too. It is easy to import a disk set from a Dell H730 on a Broadcom 9361 or a LSI 9261, as long as they're SATA drives. SAS makes things a bit more complicated, but not much. Of course, using 12Gb SAS drives on a Dell H710 or LSI 9261 is not possible, but that's the only complication.

Oh, and it wasn't my choice in how this machines was built, as it was built before I ever started with this company. So, software RAID wasn't a choice for me. Also, with the performance needed for this, editing large video files over a Linux SMB share, software RAID would not work. It just doesn't perform well enough. I know this from direct experience.
 
dgingeri said:
Oh, and it wasn't my choice in how this machines was built, as it was built before I ever started with this company. So, software RAID wasn't a choice for me. Also, with the performance needed for this, editing large video files over a Linux SMB share, software RAID would not work. It just doesn't perform well enough. I know this from direct experience
Click to expand...

That doesn't stand to reason.

If you have to install drivers to use the raid card, you are using a variant of software raid. With the level of CPU performance that's been around for the last ten years or so I've rarely seen a situation where software raid via mdadm didn't perform adequately. If the cards (replacement vs original) are compatible just bung the card in, connect the drives, install the drivers and you're good to go.

Your boss is still a flog using ridiculous teaching methods.
 
Consideration #1 needs to be revisited if possible. It doesn't need to be in a super disaster recovery capacity. However, if it's conceivable to push a copy of the contents (at least work data) over to an external drive or system while you learn whatever (or decide to yolo it), it'll take a lot of the pressure off.

Do you have any similar hardware to practice this scenario or will no one really care if you mess something up? I haven't played with RAID cards all that much, but don't they have their own schemes for writing metadata across the drives? (making portability between different controller types practically impossible) That would make the drivers at the linux level (if boot volume is on the array) not really the first problem to overcome.

If not, at the OS level, I guess your reading adventure will be about the contents of /boot (grub, initrd/vmlinuz) and reading the documentation for setting up the new controller in Linux. I'm just pulling an idea from the air with no experience, but if it's not baked into the default kernel, then you'll have to install some sort of package and then regenerate initrd/initramfs so that their supplied driver will be available as the system starts to boot.

Personally, I think your blocker will be before that at the RAID hardware & metadata level though. Unless the boss shares the secret handshake, the shortest path will likely be backing up data and starting with a new install. You could also pitch for bumping to a newer version (16.04?) for added value. Just my $0.02

Edit: betcha another $0.02 that the guys in the SSD & Data Storage forum would have more first hand experience.
 
Heh, experimenting with a work dataset without backup... prepare a good rope and some grease for yourself too.

Your boss may be so dumb that he says you don't need a backup. But rest assured if that data is gone, your head is going to fall. It always goes like that.
 
BulletDust said:
Your boss is still a flog using ridiculous teaching methods.
Click to expand...

Well, yeah, and now he's on vacation, leaving me with many things I haven't been able to figure out. He's a much younger kid, late 20s, who has been through college, and he is quite smart, but not as smart as he thinks he is. He doesn't know people all that well, and doesn't yet realize that while figuring something out on their own might get people to remember it better, it does cost a lot of time and is sometimes impossible for certain things.

This same guy left me with a Salt config for the production stack that he wanted me to just adjust for the QA stack, and didn't even tell me how to get the config files from the git repository onto the QA management servers. I did finally figure that part out, but was unable to get Salt to work with the QA systems at all.
 
Last edited:
B00nie said:
Heh, experimenting with a work dataset without backup... prepare a good rope and some grease for yourself too.

Your boss may be so dumb that he says you don't need a backup. But rest assured if that data is gone, your head is going to fall. It always goes like that.
Click to expand...
It's more like a workbench type setup. It is for very large uncompressed video files, to have a location locally that they can decompress a video to, then run it through processing to add captions, and then it gets burned to DVD and sent back to the customer. When it is not in use, there is no useful data on it. However, the boot drive is a lun on the raid set.

The raid set is raid 6, which is then divided up into 2 luns, one for the OS and one for storage. So, if I just swap the controllers and import the raid set, it probably won't boot. I don't know if installing the drivers for the new controller first, then shutting down and swapping the controller, would be possible or advisable. That's how I'd do it with Windows, but I don't know if Linux could handle it.
 
Oh, and as far as my experiences with Linux software raid, a while back I wanted to build a Linux file server, but didn't want to have to pay for 2 expensive (at the time) 4TB drives. I could get 1TB laptops drives for cheap, and I figured with 8 of them, I could run RAID 10 and get as good of performance and redundancy. The results were that I was getting write speeds at less than half the rate of a single laptop drive, in the range of 40MB/s. I got a little better from a hacked version of Windows 2012r2 server and Storage Spaces at about 60MB/s. The same drives with a hardware raid controller, a Dell H710 I picked up off ebay for $200, still keeping me below the cost of 2 4TB drives, was able to do 280MB/s, and automatic rebuild if I were to hot swap a drive.
 
dgingeri said:
Oh, and as far as my experiences with Linux software raid, a while back I wanted to build a Linux file server, but didn't want to have to pay for 2 expensive (at the time) 4TB drives. I could get 1TB laptops drives for cheap, and I figured with 8 of them, I could run RAID 10 and get as good of performance and redundancy. The results were that I was getting write speeds at less than half the rate of a single laptop drive, in the range of 40MB/s. I got a little better from a hacked version of Windows 2012r2 server and Storage Spaces at about 60MB/s. The same drives with a hardware raid controller, a Dell H710 I picked up off ebay for $200, still keeping me below the cost of 2 4TB drives, was able to do 280MB/s, and automatic rebuild if I were to hot swap a drive.
Click to expand...

Probably some configuration problem there with the software raid. You don't get write cache for example by default (and enabling it is risky in case of crash / power loss). So a proper raid with a battery backed write cache is pretty much the only way to go if you're doing something serious.
 
You must log in or register to reply here.
Back
Top