Supermicro IPMI failover, does it occupy the whole NIC?

I

iroc409

[H]ard|Gawd
Joined
Jun 17, 2006
Messages
1,385
I recently picked up a X10SLM+-F to "update" my file server (add more drives). It has two Intel i210-AT NICs and a Realtek dedicated to IPMI. In the BIOS it says something about "Failover" behavior in the BMC/IPMI config, but it isn't something I can change in BIOS so I sort of ignored it (just wanted to get the system up).

When I connected it to the network, I naturally used the first i210. I have FreeBSD on the server, and ifconfig seems to indicate only one NIC, em0 (maybe igb0? Too many NICs lol). I was going to set up the two NICs in a lagg failover (not a multi-link), so I tried ifconfig igb1 up and it says the device doesn't exist.

Tried to Google it quick, and it seems like this auto IPMI failover mode is a jumper setting or something so I'll have to tear the machine apart, but does the IPMI failover mean that the other on-board NIC is not available to the operating system? It seems that is the case but I want to make sure its not a wetware error.
 
I'm not sure about your failover question, but my IPMI nic had to be jumped on on my supermicro board.
 
The failover option makes the nic act like a three port switch with IPMI, normal i210 nic and the connection to the rest of your network. If you go into the IPMI web interface you can change the failover mode.
 
Gotta love hard. I just got the same board and I am sure I will run into this.
 
jschuricht said:
The failover option makes the nic act like a three port switch with IPMI, normal i210 nic and the connection to the rest of your network. If you go into the IPMI web interface you can change the failover mode.
Click to expand...

Thank you for the info! I will have to fire up IPMI, as I haven't played with it (recent servers are Dell).
 
You should be able to create a LAGG with the two Intel ports. Do this in FreeNAS GUI under Network -> Interfaces -> Add and for the type do Link Aggregation. I can maybe help you more with this. What's your entire output of ifconfig -a from an SSH shell?

Whether you can get the Realtek IPMI NIC working in FreeNAS, don't actually use it. Realtek NICs are notorious for causing weird issues with FreeBSD/FreeNAS and should be avoided. ServeTheHome has a good list here of NICs that are well supported and work well: https://www.servethehome.com/buyers...as-servers/top-picks-freenas-nics-networking/ Also having a dedicate IPMI port is awesome for running it headless.
 
Dopamin3 said:
You should be able to create a LAGG with the two Intel ports. Do this in FreeNAS GUI under Network -> Interfaces -> Add and for the type do Link Aggregation. I can maybe help you more with this. What's your entire output of ifconfig -a from an SSH shell?

Whether you can get the Realtek IPMI NIC working in FreeNAS, don't actually use it. Realtek NICs are notorious for causing weird issues with FreeBSD/FreeNAS and should be avoided. ServeTheHome has a good list here of NICs that are well supported and work well: https://www.servethehome.com/buyers...as-servers/top-picks-freenas-nics-networking/ Also having a dedicate IPMI port is awesome for running it headless.
Click to expand...

Thanks Dopamin3! I should be able to set it up... lol. I'll try and get this fired back up by the weekend and see what it says. I only remember ifconfig showing localhost and igb0 (or em0...). Once I get it plugged back in I'll do some troubleshooting there and see what pops up and report back.

No GUI here; just me, Beastie and rc.conf. I'd definitely like to keep the Realtek on IPMI and not for the operating system, and I agree FreeBSD has never played nice with Realtek. I just figured I should have the option, so why not fail over the extra NIC?
 
iroc409 said:
Thanks Dopamin3! I should be able to set it up... lol. I'll try and get this fired back up by the weekend and see what it says. I only remember ifconfig showing localhost and igb0 (or em0...). Once I get it plugged back in I'll do some troubleshooting there and see what pops up and report back.

No GUI here; just me, Beastie and rc.conf. I'd definitely like to keep the Realtek on IPMI and not for the operating system, and I agree FreeBSD has never played nice with Realtek. I just figured I should have the option, so why not fail over the extra NIC?
Click to expand...

If you're not wanting to use a GUI at all, don't use FreeNAS. Go with Linux (since it has ZFS support more readily available now) or just straight FreeBSD. You might notice when you login via SSH shell the MOTD for FreeNAS literally says "Warning: settings changed through the CLI are not written to the configuration database and will be reset on reboot." Now instead of SSH if you access it via IPMI or an actual monitor you can make some configuration changes. But you can't act like it's FreeBSD and modify files like /etc/rc.conf , /etc/resolv.conf , etc... unless you want to redo your changes everytime you reboot or do an update.

I manage all my iocage jails and bhyve VMs manually through SSH and execute commands inside them which is fine. But you shouldn't be making configuration changes to the actual FreeNAS host. Even if you got your LAGG to work briefly setting it up CLI it will stop working.

Does the interface show in the GUI under Network -> Interfaces? For me igb0 is my Intel NIC onboard and mixen0 is my 10Gb Mellanox card.
 

Attachments

  • freenas interfaces.PNG
    freenas interfaces.PNG
    24.9 KB · Views: 0
We have our wires crossed. :D I am not running FreeNAS, just plain old FreeBSD 12.1, hence the lack of GUI. I actually haven't gotten as far as editing rc.conf, as the first steps in the handbook are on the command line; then update the conf if it works. I have to find the interface first. :)
 
iroc409 said:
We have our wires crossed. :D I am not running FreeNAS, just plain old FreeBSD 12.1, hence the lack of GUI. I actually haven't gotten as far as editing rc.conf, as the first steps in the handbook are on the command line; then update the conf if it works. I have to find the interface first. :)
Click to expand...

I'm so sorry lol. I don't know where I read that you were using FreeNAS but swear I saw that... must be out of it.

Does it show when you run pciconf -lv ? Can you make sure it's not disabled in the BIOS?
 
Well, so far the extra NIC just doesn't show up at all. It doesn't bring up a link, isn't shown in pciconf, etc. I haven't had a chance to play with IPMI yet so when I eventually get around to that I'll see what's going on.

Interestingly, nmap doesn't see the IMPI NIC at all either when scanning my local network, even though it has a link and plugged into the same subnet. I'm sure that's by design but I just figured it would at least pick it up. Usually the idrac on my Dells do the whole IP address thing and are somewhat visible, but the Dells I have with it share the NIC with the system.

Anyway, everything is running how I need it to with no issues, so I don't want to fix it until I break it but I will figure it out one of these days.
 
You must log in or register to reply here.
Back
Top