Virtualized router?

[Rivera]

Hello, it's me again, and again with questions
Anyone using router running in VM? Since i got pretty powerful hw now and faulty router i decided to try pfsense and found it, well, pretty amazing.
So i got idea of "tunneling" my uplink via switch with VLAN to esxi boxes, configuring CARP in pfSense (also pfsense seems the only x86-based router that can sync configs, all similar software can only do IP failover) to do failover in case of hw crash and let it work. Oh, and put it on local storage, not NAS.
Still dunno if it good idea or not. Most interesting thing is what i should do if both routers crash...

 

[peanuthead]

You can virtualize a router. Some says it's not best practices but I think every situation needs to be analyzed and judged appropriately. AutoLab uses a virtualized SCO router and I run that on local storage in VMware workstation (but can on an ESXi host)

 

[extide]

That plan will work. It's a bit complex but if you can handle it then you should be able to do it no problem.

pfSense is AWESOME! I run it at home, at my in-laws, and at my clients too.

 

[Rivera]

The most interesting question is about stability... Well i think if both routers fail i can just configure static ip on my gaming notebook and connect via vCenter Client to troubleshoot it. My current router is dying anyway...

 

[bds1904]

What you are talking about is commonly refered to as "router on a stick".

I have been running pfsense and zeroshell on my ESXi cluster for almost a year now and have never had any problems at all. My pfsense box has 6 virtual NIC's (2 of which are WAN's), i use it for load balancing and much more. I woun't have it any other way.

Just remember that unless you are doing NIC passthrough to pfsense, the VLAN's are handled at the ESXi level, and then you set up the pfsense VM with multiple NIC's.

 

[DarkDiamond]

I also have a virtualized pfSense router and have had no issues, although I'm not doing any load balancing... Thanks to some advice here at [H], I set up VLANs for my WAN connection and am able to vmotion my router across my two hosts, also with no issue.

 

[MrGuvernment]

isn't one key to have a NIC with a dedicated port for the router side for incoming to keep traffic separate in on the virtual layer?

 

[Rivera]

Just remember that unless you are doing NIC passthrough to pfsense, the VLAN's are handled at the ESXi level, and then you set up the pfsense VM with multiple NIC's.

Oh. Thanks for it. Thought about what i'm doing wrong. After your post i got virtual router working, having my gaming nb connected to virtual router That PVID parameter is messy..

 

[Child of Wonder]

I'm running a virtualized Windows 2008 R2 RRAS server and it works great.

 

[-Dragon-]

There's one minor issue with doing this if your internet comes from a cable modem, and that is cable modems are little bitches about latching onto the first MAC address they see chatting in any way, and pretty much any switch that supports VLANs will probably also have stuff like STP, CDP, LLDP, at least one of which needs to be disabled for it to even work at all (pretty sure it's STP). You'll find yourself needing to reboot your modem any time either your switch or virtual router reboots, not really a huge issue but assuming you stick your server/modem in some out of way place in your house it'll be something that annoys you the few times it happens.

 

[Rivera]

There's one minor issue with doing this if your internet comes from a cable modem, and that is cable modems are little bitches about latching onto the first MAC address they see chatting in any way, and pretty much any switch that supports VLANs will probably also have stuff like STP, CDP, LLDP, at least one of which needs to be disabled for it to even work at all (pretty sure it's STP). You'll find yourself needing to reboot your modem any time either your switch or virtual router reboots, not really a huge issue but assuming you stick your server/modem in some out of way place in your house it'll be something that annoys you the few times it happens.

I'm happy i got provider with proper network - they use vlan-per-customer scheme, so just plug cable into any DHCP-capable device and it will work. Since i have only one switch i do not use STP (however my router also acts as switch, there is two port groups with two switch chips...)
Thanks for info, btw. I'm looking into second provider for fallback/balancing, found our main telephony company offering 200mbps GPON and they use some kind of GPON modem (looks like another unnamed chinese crap, no docs, no specs). I wonder if i can just purchase proper SFP module and plug it directly in my switch, tunneling it as i done with my current provider, eliminating need in another CPE.

 

[-Dragon-]

I highly doubt it, as the CPE needs to differentiate your traffic from the rest of the traffic on the fiber, depends on how the provider does it but it's doubtful they just put you all on different unecrypted vlans, there's probably some sort of encryption going on and authentication going on when the CPE starts up.

 

[mayhem87]

I currently run this setup with pfsense. I just plugged my cable modem into a NIC that was on the esxi box. Put that nic on its own vswitch with only pfsense on it. From there I created another switch for my inside traffic and put my pfsense lan interfaces on that. I guess you could do this all in one switch but I like to break out my vlans into switches so it makes it easier for me to read rather then having to look at the portgroups.

 

[Nate7311]

What you are talking about is commonly refered to as "router on a stick".

I have been running pfsense and zeroshell on my ESXi cluster for almost a year now and have never had any problems at all. My pfsense box has 6 virtual NIC's (2 of which are WAN's), i use it for load balancing and much more. I woun't have it any other way.

Just remember that unless you are doing NIC passthrough to pfsense, the VLAN's are handled at the ESXi level, and then you set up the pfsense VM with multiple NIC's.

QFT! Just got to playing with this over the weekend, and found that out the hard way Set up a new port group on whatever Network and assign that port group a VLAN id. Then you VM NIC gets assigned the new VLAN port group.

 

[Rivera]

I highly doubt it, as the CPE needs to differentiate your traffic from the rest of the traffic on the fiber, depends on how the provider does it but it's doubtful they just put you all on different unecrypted vlans, there's probably some sort of encryption going on and authentication going on when the CPE starts up.

Found out they use something called "IPoE". Not sure what it is. I know EoIP, IPIP, but... IPoE? Maybe that's abstract name for DHCP-based authentication? Here is scheme of GPON ONT, comments are in russian but scheme is english...

Related main theme: i managed to get "virtual" router (RouterOS too) act as fallback router for my HW RB493G. WiFi is down of course, but at least i can continue to listen music at work After replugging power cord (seems like HW problem, even watchdog can't reboot it) main router re-takes VRRP and all OK again. Not sure if i will make main router virtual too, but at least now i have fallback mode, which already saved me today
Thanks guys, you're truly helpful

 

[NetJunkie]

I run Untangle in a VM and it's working just fine. Have an internal VLAN and an external VLAN. Only thing on External is my cable modem set in bridging mode.

 

[artbird309]

I have been playing with pfsense and vlan,'s and found that there is a few ways of doing it. You can setup different nic's and putting them in different port groups to access different vlan's or you can setup a nic with the special vlan id of 4095 to enable trunking so that VMware will tag all the packets on that nic and setup the vlan's on pfsense.
http://kb.vmware.com/selfservice/mi...nguage=en_US&cmd=displayKC&externalId=1004074

I found that a combination of those was the best for me to make it easy to follow and make sure the traffic is separated. Before I can set using this at home I need to get another vlan enabled switch or move my modem and probably have to rewire the phone jack.

 

[Rivera]

I have been playing with pfsense and vlan,'s and found that there is a few ways of doing it. You can setup different nic's and putting them in different port groups to access different vlan's or you can setup a nic with the special vlan id of 4095 to enable trunking so that VMware will tag all the packets on that nic and setup the vlan's on pfsense.
http://kb.vmware.com/selfservice/mi...nguage=en_US&cmd=displayKC&externalId=1004074

I found that a combination of those was the best for me to make it easy to follow and make sure the traffic is separated. Before I can set using this at home I need to get another vlan enabled switch or move my modem and probably have to rewire the phone jack.

So basically if i add interface with vlan 4095, traffic will be bypassed "as is" and i can configure VLANs in guest?

 

[DarkDiamond]

I'm happy i got provider with proper network - they use vlan-per-customer scheme, so just plug cable into any DHCP-capable device and it will work. Since i have only one switch i do not use STP (however my router also acts as switch, there is two port groups with two switch chips...)
Thanks for info, btw. I'm looking into second provider for fallback/balancing, found our main telephony company offering 200mbps GPON and they use some kind of GPON modem (looks like another unnamed chinese crap, no docs, no specs). I wonder if i can just purchase proper SFP module and plug it directly in my switch, tunneling it as i done with my current provider, eliminating need in another CPE.

I'm guessing my switch must have STP turned off by default. I'll have to check that when I get home. I use Comcast as my provider and I've only rebooted my modem one time in 2 or 3 years.

 

[artbird309]

So basically if i add interface with vlan 4095, traffic will be bypassed "as is" and i can configure VLANs in guest?

Correct, you can then create the vlan interface on the guest and assign them how you like. The guest will tag them and they will be passed through to the vswitch with the tag.

 

[Rivera]

Correct, you can then create the vlan interface on the guest and assign them how you like. The guest will tag them and they will be passed through to the vswitch with the tag.

So much to learn. Great feature, thanks, just what i looked for. Extremely great for network experiments

 

[athlon1.2]

There's one minor issue with doing this if your internet comes from a cable modem, and that is cable modems are little bitches about latching onto the first MAC address they see chatting in any way, and pretty much any switch that supports VLANs will probably also have stuff like STP, CDP, LLDP, at least one of which needs to be disabled for it to even work at all (pretty sure it's STP). You'll find yourself needing to reboot your modem any time either your switch or virtual router reboots, not really a huge issue but assuming you stick your server/modem in some out of way place in your house it'll be something that annoys you the few times it happens.

I passed through Comcast cable modem on a cheap Linksys switch and it's been working without any problems.

pfSense is on a physical box but I've been thinking about virtualizing it.

 

[bAMtan2]

you CAN have too much stuff running on one server and at risk of total failure

 

[Rivera]

Ok, current status:
VRRP works for default GW, works for switching me to another WAN (well, same connection but different IP got by DHCP client). So i get _almost_ instant recovery. Sad part - existing connections dropped. I think if i will somehow manage to VRRP my primary (static, but still need DHCP for auth) ip recovery will be faster (ie inbound VPNs from work, my servers, etc)
And now where is fun comes. How i can balance same WAN IP?
Provider uses DHCP for auth so i can't just bind static IP. And as i can see i can't put DHCP client on VRRP interface? Any ideas?

 

[peanuthead]

Ok, current status:
VRRP works for default GW, works for switching me to another WAN (well, same connection but different IP got by DHCP client). So i get _almost_ instant recovery. Sad part - existing connections dropped. I think if i will somehow manage to VRRP my primary (static, but still need DHCP for auth) ip recovery will be faster (ie inbound VPNs from work, my servers, etc)
And now where is fun comes. How i can balance same WAN IP?
Provider uses DHCP for auth so i can't just bind static IP. And as i can see i can't put DHCP client on VRRP interface? Any ideas?

Is DDNS an option? (I haven't had the morning coffee so my brain may have read the above incorrectly.)

 

[Rivera]

Is DDNS an option? (I haven't had the morning coffee so my brain may have read the above incorrectly.)

Sure - that's how it works ATM, using fallback IP (my provider allows connecting 5 devices directly, assigning dynamic white IPs to them (but only one mac have static binding). So now i think if it possible to balance main IP...

 

[Nate7311]

Ok, current status:
VRRP works for default GW, works for switching me to another WAN (well, same connection but different IP got by DHCP client). So i get _almost_ instant recovery. Sad part - existing connections dropped. I think if i will somehow manage to VRRP my primary (static, but still need DHCP for auth) ip recovery will be faster (ie inbound VPNs from work, my servers, etc)
And now where is fun comes. How i can balance same WAN IP?
Provider uses DHCP for auth so i can't just bind static IP. And as i can see i can't put DHCP client on VRRP interface? Any ideas?

What is the internet connection? Assuming standard cable, the connections dropped due to the different IP address. A new IP was assigned because a NIC with a different MAC contacted it for an IP. For things to function as you desire, you'll need a static address.

 

[Rivera]

What is the internet connection? Assuming standard cable, the connections dropped due to the different IP address. A new IP was assigned because a NIC with a different MAC contacted it for an IP. For things to function as you desire, you'll need a static address.

IP is mapped to MAC on ISP's BRAS. Auth performed by DHCP, technology is called "IPoE", i think it's CIS-related thing (IP over Ethernet, lol, basically means DHCP auth, vlan-per-customer, call it anything you want) because i haven't heard about it anywhere except Russia and Ukraine.
I do have static IP but i can't just bind it to interface without DHCP. DHCP client do not work on top of VRRP (at least on mikrotik, i'm thinking about implementing workaround kludge)

Oh, and my core router died today. Rest in peace