Help a VLAN noob?

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Hey all,

So I have a need to have one client on my network be isolated from the rest, but still be able to access the outside internet via the gateway.

My setup is:

pfSense Router (two ports, LAN & WAN) -> HP Procurve 1810G24 switch -> all clients

I know I can probably do this with VLAN's somehow, but I ahve yet to find a good resource from which to learn how that would be accomplished.

Is there any way I could set port one on the switch (the pfsense LAN port) as accessible from multiple VLANs at the same time, and thus have two VLAN's, one for the isolated client, and one for all the others both access the gateway through that one port?

Would this require some sort of tagging on the pfSense side?

Much appreciated!
 
Last edited:

Grayson

n00b
Joined
Oct 1, 2017
Messages
39
I don't use pfsense, but generally you create the vlan. Then tag the switch port on your procurve that the client is connected to for the vlan. Then you may need to create firewall rules to block vlan traffic. Some block by default some do not.
 

Vengance_01

[H]ardness Supreme
Joined
Dec 23, 2001
Messages
5,899
I am guessing WAN port goes to your modem correct? So your LAN port going to the switch will be your uplink port to the procurve switch. This port will tag for both VLANs. Lets call all clients VLAN 1 and then the isolated VLAN 100. All clients connected to VLAN 1 then you would tag those ports as access or switch ports with VLAN 1. Then the single client you would make its port a switch port with VLAN 100 access. Then just make sure both ports can route out on the pfsense side and you should be good. The computers will not be able to talk to each other unless you create a route on pfsense to allow this traffic.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I am guessing WAN port goes to your modem correct? So your LAN port going to the switch will be your uplink port to the procurve switch. This port will tag for both VLANs. Lets call all clients VLAN 1 and then the isolated VLAN 100. All clients connected to VLAN 1 then you would tag those ports as access or switch ports with VLAN 1. Then the single client you would make its port a switch port with VLAN 100 access. Then just make sure both ports can route out on the pfsense side and you should be good. The computers will not be able to talk to each other unless you create a route on pfsense to allow this traffic.
Yes, my bad.


I write WAN when I meant to write LAN.

Will edit.
 

jardows

[H]ard|Gawd
Joined
Jun 10, 2015
Messages
1,794
I used ALAN. Extra Ethernet port in the pfsense router, with dedicated physical connection. The option in pfsense is OPT1.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I used ALAN. Extra Ethernet port in the pfsense router, with dedicated physical connection. The option in pfsense is OPT1.
Yeah, that was going to be my first choice, but I'm using a mini-ITX board with two on board Intel NIC's, and a compact case without a PCIe slot:

IMG_20170401_141021.jpg

I have plenty of spare intel NIC's kicking around, but just can't add one to this box.

Maybe I can accomplish something similar by setting up multiple LANs on the pfsense box with different DHCP on each, and having one LAN on untagged default VLAN and the other on a tagged VLAN.

I've heard of people doing stuff like this using the same NIC in pfsense for both WAN and LAN using VLANS, for slower connections, so I'd imagine setting this up for two LAN's on the same NIC shouldn't be impossible. I just need to figure out how.

There are very few good "guides for beginners" out there when it comes to VLAN's
 

jardows

[H]ard|Gawd
Joined
Jun 10, 2015
Messages
1,794
Yeah, that was going to be my first choice, but I'm using a mini-ITX board with two on board Intel NIC's, and a compact case without a PCIe slot:

View attachment 42539

I have plenty of spare intel NIC's kicking around, but just can't add one to this box.

Maybe I can accomplish something similar by setting up multiple LANs on the pfsense box with different DHCP on each, and having one LAN on untagged default VLAN and the other on a tagged VLAN.

I've heard of people doing stuff like this using the same NIC in pfsense for both WAN and LAN using VLANS, for slower connections, so I'd imagine setting this up for two LAN's on the same NIC shouldn't be impossible. I just need to figure out how.

There are very few good "guides for beginners" out there when it comes to VLAN's
Nice setup! What's the CPU being used?

I used to support a router that had VLAN's, but it was discontinued pretty soon after I started working for this company, and I don't think I ever got a call on it. Sorry that I can't be more help on this particular issue.

My current role at work is knowledgebase article creation. I've thought if I had some time - may have some next week - I may take some of that same approach and create some detailed How-To's for setting up pfsense. Netgate may not like it, since it could take away sales from their book, but I've found much of the publicly available and free documentation for pfsense to be inadequate or out of date.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Nice setup! What's the CPU being used?
Since I have gigabit internet and wanted to use OpenVPN I went a little bit overkill on the specs:

(The stock CPU cooler only just BARELY fit inside the case)

IMG_20170331_215359.jpg . IMG_20170331_215912.jpg


Turns out I'm limited by my VPN service to about 600Mbit even with this overkill CPU at 3.9Ghz. The CPU rarely hits more than 10% even when I try to max out the VPN connection.

The RAM was also total overkill. I only needed a gig or so, but it turns out itretty much impossible to buy DDR4 sticks smaller than 4GB, and I wanted to populate both channels...

The PicoPSU is the best thing since sliced bread. It's close enough to the actual power use of a limited system like this that it stays in the efficient range, and thus uses only about 6w at the wall at idle. Love it!



I used to support a router that had VLAN's, but it was discontinued pretty soon after I started working for this company, and I don't think I ever got a call on it. Sorry that I can't be more help on this particular issue.

My current role at work is knowledgebase article creation. I've thought if I had some time - may have some next week - I may take some of that same approach and create some detailed How-To's for setting up pfsense. Netgate may not like it, since it could take away sales from their book, but I've found much of the publicly available and free documentation for pfsense to be inadequate or out of date.

Yeah, I usually have to spend time fishing in the forums to get the answers I need, and even then the solutions are not perfect and require me to do further research and alter the methods. I at least try to post back when I solve my problem with what I did for purposes of posterity.

The more documentation - IMHO - the better. It sucks to have to be a google champ and a trouble shooter to get basic f done sometimes.
 
Last edited:

pek

prairie dog
Joined
Nov 7, 2005
Messages
1,029
Will your box handle trunks? Using cisco cli (so your syntax may vary), but:

create your vlans so they can talk to each other

vlan xxx
name vlan xxx

vlan yyy
name vlan yyy

then switchport configs:

interface (your uplink name here)
switchport
switchport mode trunk
switchport trunk encapsulation dot1q *******if you want all vlans allowed *********
switchport trunk allow vlan xxx, vlan yyy *******add if you want to restrict the trunk to specific vlans only, and not all******

and then do each separate vlan switchport

interface for xxx
switchport
switchport mode access
swtichport mode access vlan xxx


interface for yyy
switchport
switchport mode access
switchport mode access vlan yyy
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Will your box handle trunks? Using cisco cli (so your syntax may vary), but:

create your vlans so they can talk to each other

vlan xxx
name vlan xxx

vlan yyy
name vlan yyy

then switchport configs:

interface (your uplink name here)
switchport
switchport mode trunk
switchport trunk encapsulation dot1q *******if you want all vlans allowed *********
switchport trunk allow vlan xxx, vlan yyy *******add if you want to restrict the trunk to specific vlans only, and not all******

and then do each separate vlan switchport

interface for xxx
switchport
switchport mode access
swtichport mode access vlan xxx


interface for yyy
switchport
switchport mode access
switchport mode access vlan yyy
I get confused with the terminology, as I have Procurve gear and I understand that a Procurve "trunk" is a very different thing from a Cisco "trunk".

Not sure what Procurve calls a Cisco trunk though.


I'll need to look into this.

My switch will be a little bit more limited than your example above though as it does not have a full cli, only a web interface.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
That's a pretty slick looking setup for ~$500. Now you are giving me ideas haha
My total cost at the time I bought it was ~$393.31, but I may have had some sales or an open box discount. I can't remember.

Yeah, I agree, it's a decent amount of kit, in a small footprint with very low power use. I'm very happy with it.

Here is some OpenVPN performance data if interested:

Code:
[2.3.4-RELEASE][root@router.localdomain]/root: openssl speed -elapsed -evp aes-128-ecb
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-ecb for 3s on 16 size blocks: 231135962 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 64 size blocks: 207661908 aes-128-ecb's in 3.01s
Doing aes-128-ecb for 3s on 256 size blocks: 69556140 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 1024 size blocks: 17789148 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 8192 size blocks: 2216901 aes-128-ecb's in 3.00s
OpenSSL 1.0.1s-freebsd  1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) 
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-ecb    1232725.13k  4418613.90k  5935457.28k  6072029.18k  6053617.66k
I ran this test without taking it offline though so there would have been a slight amount of load on it. Real results would probably be ever so slightly higher.
 
Last edited:

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
So if it supports 802.1q, then it's good to go. (Which the HP does) Regardless of what Cisco coins "trunking" you're really only worried about vlan tagging.



So the hardest part you're going to have to deal with, is that since you'll need to modify the port which through you're connected, if you get it wrong you'll basically break your ability to get into the box. For that reason you'll temporarily want to enable HTTPS access to the web port from the web interface (WAN interface), and have a way to get into that from say your phone outside of your network. I'll try to reword that to make more sense. If you're using the LAN connection to connect to the web interface, making changes to that interface will likely cause you to lose connectivity. Thus locking you out of your box in an unusable state.


With that out of the way, setting it up is basically a cake walk.

First let's just backup the config so if it all goes south, we can just roll it back. So login to your PFSense web interface. Go to diagnostics > backup and restore. You can uncheck the backup graphing data if you care about that or not, then click "download configuration as XML". Save that for later in case everything blows up, and you can come back to this screen and just click the browse to find the file, and restore it.

With that out of the way Go to interfaces > assignments. The first thing you'll need to do is create the "vlans" which will create sub interfaces of the primary interface. Once again with only 2 nics, this will be a bit different, but definitely doable.

So once you're in assignments, click on the VLAN tab. Click the add button in the lower right corner. For the parent interface, you will have 2 choices. Right now one will say WAN, and the other will say LAN. This will change later so just document the actual interface name. (em0, fxp0, or one of the other hundred combinations) So we need to create a VLAN to replace your current LAN. VLAN 1 is usually reserved as the "trunking vlan" so I wouldn't suggest using that. But your choices are from 2 - 4096 so people have plenty of room to work with.

We're going to call your new LAN replacement "VLAN 2" so type 2 into the VLAN tag box. Give this a description of "LAN" so you know what it is later. Click Save. One down, one to go for now. You should be back on the VLAN tab now, so click the add button again. Pick the same adapter as before, which will still say LAN on it for the parent interface. Set this one as VLAN tag "3". Description "New LAN". Click save.

So now we have 2 VLANs, but they are not assigned to anything yet. Now we need to go back to the "interface assignments" tab so we can enable these two new vlans. So here's the magic, the current "LAN" will still have whatever the name of the interface you've been using to create vlans from. So if it was "em0" right now LAN will say "em0" as the network port. But now we have more options in the drop down that we did have before. If you click the drop down for LAN and now you'll see "VLAN2 on em0" and "VLAN3 on em0" but instead of em0 whatever that interface is called. So we'll set the LAN to "VLAN2 on xxx" for your configuration. Then you'll need to click the save button at the bottom.

So to recap what we just did, we switched the LAN port from being untagged to tagged. When it was untagged and any all traffic on your interface will be directed in and out of that interface. Now that the vlan is tagged and we set what tag we want, the only traffic that will go out of that interface is stuff that has a VLAN tag of "2". Remember that now this is a virtual interface, and is only a portion of the traffic that will exist on that physical interface. (It's clearly made up because it's "VLAN2 on xxx")

So at this point if you were attempting to do this from the old "LAN" interface, you would be screwed because the PFSense "LAN" interface no longer accepts untagged traffic. It will only talk to devices which are talking using a VLAN tag "2". Since your machine isn't setup to tag traffic, you'd no longer be able to talk to that LAN anymore, hence why you MUST do this from your WAN connection. (Normally I toss in a 3rd NIC for configuration purposes, but you don't have that option)

So that's one interface out of the way, let's add the 2nd one while we're in here. You'll notice that at the bottom of the interface assignments tab, you still have the ability to create another interface. You didn't have this option before we started creating vlans because you were all out of interfaces to assign. Now that we have more interfaces, you have stuff to pick under "available network ports". So we just need to pick "VLAN3 on xxx" and click the Add button next to it. Then we'll click save. You should now have "OPT1" with a network port of "VLAN3 on xxx".



So that's a great place to stop this post, and I'll create another post to cover setting up the HP switch.
 
Last edited:

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
So the HP switch part is more of less trivial to setup, but at the same time I'll try to refer you to some configuration guides as I don't have one in front of me, nor can I tell you the last time I've had to configure one. The basic idea we have to go through is this:

We need to connect the physical port that was your "LAN" port on your PFSense box, to a specific port on your switch. The physical port for your PFSense box is no longer just a plain old untagged port, it's a special "tagged" port which if you hear most people talk about that port they'd probably just call it a trunk port, since it has the ability to carry multiple virtual networks over one cable. It important to know that interface is special now because we need to tell the switch that it's special, and tell it what information you should be expecting to receive from it.

So I don't know how you connect into your HP switch, but I'm going to assume the web interface since it's probably the easiest way. The same caveat's apply as they do with the PFSense box, if you are using a web interface to get into your switch, you'll want to have a port you can dedicate to configuring through the web interface that will remain unchanged. When we start tagging traffic we will break your connection to the web interface or telnet / ssh connection if you were using an untagged port and switch it to a tagged port like "2" or "3". So if you are planning on using something to connect to the switch that IP based, we should dedicate a port for management. For this I'll just toss out a number and keep referencing it, you can pick any interface you want, but I'll just pick interface 5 as your management interface. We need to make sure we don't touch this or you can lock yourself out of your switch requiring you to factory reset it. (Far less unlikely since you have lots of interfaces to pick from, and if you have a console port then you can't lock yourself out)

The default ip for the web interface is 192.168.2.10, with a 255.255.255.0 subnet mask. By default any port we're plugged into will be able to reach that, but we're going to plug our cable into port 5, and then configure a static IP of 192.168.2.11 with a 255.255.255.0 subnet mask on whatever device you want to configure the switch from. (Using multiple computers will make this easier)

So same stuff applies, backup the switches configuration at this point. There is probably 100 tutorials for that, so google will help out. Here is two ways people said you can do it, but there are likely several more.
https://support.auvik.com/hc/en-us/articles/212152486-How-to-back-up-configuration-and-update-firmware-on-HP-ProCurve-1810g-series-switches
https://community.hpe.com/t5/Switches-Hubs-and-Modems/Procurve-config-backup/td-p/4797498

Here is the first link verbatim:

To back up your configuration:

  1. Log into your HP switch web console.
  2. Go to Maintenance > Backup Manager.
  3. For Backup Method, select HTTP.
  4. Leave Server IP and File Name fields blank.
  5. For Backup Type, select Configuration.
  6. Click Apply.
  7. Don’t disturb the browser window while the backup is in progress. It may take about three minutes for the backup to complete.

With that out of the way, I'm basically referencing the manual for an HP 1810. You should be able to click on VLANs > VLAN configuration. We'll need to create those two VLANs we made on the PFSense box to match here. Click the create VLAN checkbox, and the VLAN ID will be "2", since that is the same tag number as we used for the "LAN" on our pfsense box. You can probably name it on this side, and in our case you'd name it "LAN", just to be consistent between places. The names don't have to match for this to work. Click apply, then do the same steps again with a VLAN ID of "3", and this we called "New LAN" back on the PFSense box.

The VLANs have been created, so now we need to assign them to something. Click on VLANS > Participation / Tagging. Then you'll have a terrible looking interface that should hopefully have what we need on it. In the drop down next to VLAN I'm going to assume you'll have the numbers 1,2, and 3. Pick "2" in the drop down list. To make this easy, I'll use switch port 2 for VLAN 2. So we're going to find port 2, and click the Letter under it until it displays a "T". That means that port is now specified for Tagged traffic for VLAN 2. Click apply. Let's click the VLAN drop down again, but select 3 this time. Now find port 3, and click the letter until that says "T". Click apply again. That should have set port 2 to VLAN 2 and port 3 to VLAN 3.

So now we need to create the "trunk" interface. I can clearly tell that "trunk" in your HP manual relates to LACP, which is bonding multiple interfaces together for increased bandwidth, but has nothing to do with VLANs or tagging. So my best guess is that creating a "trunked" or tagged interface for our example should be as simple as continuing to use the participation / tagging screen we are already on. So what we need to do is let a certain port talk to both VLAN 2 and VLAN 3 at the same time. I'm just going to pull a number out, and say we'll use port 20. This port is going to be whatever port you have plugged into the physical "LAN" cable you were using on PFSense. You can either move the cable to port 20, or just configure the number of the port that cable is plugged into. So we need to go through the same steps as before, but this time when we select VLAN 2, we need to put a "T" for port 20, click apply, then select VLAN 3 and also put a "T" for port 20 and click apply again. We should now have tagging enabled for VLANs 2 and 3 on port 20.

At this point, you should have a working connection for your virtual "LAN" connection, but it will only work from port 2 on your switch. So we'll need to plug a device into port 2 on your switch, and verify you get a DHCP lease from your PFSense box, and you can connect to the internet. If that works then we'll have succeeded in getting this started. If it doesn't work, we'll have to try to troubleshoot some things. This is were it can be a bit frustrating because you have multiple parts in play, so you're not really sure where to start. The thing to remember is that when you're using tagged traffic, you cannot see that traffic with an untagged connection. So plugging a computer back into the physical "LAN" port on your PFSense should now be broken, and plugging a computer into the port that you set the multiple tags on shouldn't work either. I'm actually not sure how HP handles this, so hopefully this is the correct way. If it's not working then I'd probably look for some configuration guides to two switches with VLANs. If you had another switch it would make testing those VLANs easier, but a lot of times you'll just have to play around until it works if you don't have the stuff to test with.


So assuming that is now working, we'd just need to go back into the VLAN > Participation / tagging page, click on VLAN 2 again, and now you can set the rest of the ports you want to include on your "LAN". Click apply and all of that should be working.
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
That's all and good now, but obviously I left out any mention of VLAN 3. We essentially did a bunch of work to accomplish exactly what we had before. All of the ports that are on VLAN 2 can talk just like they did before, except for port 3. At this point port 3 is not working, because there are more steps. On the switch side VLAN 3 is configured. You can verify this by putting another port on VLAN 3, and the two computers will talk to each other. We set the participation for your "trunk" to include this VLAN, and we even set the interface in PFSense for VLAN 3. What we need to do now, is actually enable that interface in PFSense, give it an IP, and then put some firewall rules in place to allow it to go places. All of that is taken care of by default for you when you setup your PFSense box, but we're going to have to do all of that now for the new interface.

So login to your PFSense box, and then go to Interfaces > OPT1. The first thing you'll need to do is click the checkbox to enable interface. This is important to know because PFSense will hide it on other screens if you don't have it enabled. I'm pretty sure it doesn't show up in the firewall rules until it's enabled. Let's give this an IP. For the IPv4 configuration type, choose static IP. Then you can set the static IPv4 configuration to something, I'll just pick 192.168.3.1 /24 for now. Make sure the block private IPs checkbox at the bottom is unchecked, and click save.

Now click on Firewall > Rules, and you should have an OPT1 tab that you didn't have before. It will say something about No rules defined at this point. To just make stuff work for testing, we're going to go with the wide open configuration, so you can make sure it work. So click either one of the add buttons, make sure the action is set to "pass" the protocol is set to "Any". The other stuff should be any by default, so make sure it is and click save. The basically turned off the firewalling on that port, so later we need to put rules in place, like a rule that blocks outbound traffic to your LAN subnet, so stuff on "New LAN" (OPT1) can't talk to stuff on the original LAN.

Chances are high that you want to hand out leases for this interface, so we need to go to Services > DHCP server. Once again you'll now have an OPT1 tab that didn't exist before. Click the box for enable, and setup your pool however you want. Click save and that should start the DHCP server for that interface.

So now we are ready to test VLAN 3. From port 3 on the switch, you should be getting a DHCP lease from the pool you just created. Since we opened up the firewall rules, you should be able to ping the internet and anything that is on the "LAN" which is VLAN 2. If that works then you're in business and you've successfully gotten it all to work!

At this point creating new VLANs is as simple as going through the same process, making sure you create both sides of it, setting the port as tagged for the connection that goes between PFSense and the switch, configuring a port to use it, and then doing the configuration on the PFSense side to enable it. There's definitely a few steps, but once you have stuff working the new interfaces is a lot easier. You'll probably want to setup firewall rules to start blocking things, and for reference the PFSense web GUI will be accessible from any IP it has. So you can get into it now from VLAN 3 if you type 192.168.3.1 into your browser. This means you can probably turn off access from the WAN again, but you might also want to block web access from VLAN 3 as well.

EDIT: Also, After you determine that VLAN 3 is working, that is the time to make another set of configuration backups for both the switch and the firewall. Cause when you break it after that you now have a point to revert back to where everything was working with VLANs in place.
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Holy shit, you weren't kidding. This is quite the writeup!

I've been sick all weekend so I haven't had a chance to tinker, but I will definitely be reading through this in the next couple of days.

Thanks!
 
Last edited:

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
Bumping this thread a month later to see if you had time to play with this. Just curious if you were able to get stuff figured out or not.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Bumping this thread a month later to see if you had time to play with this. Just curious if you were able to get stuff figured out or not.

Appreciate the bump.

I got tied up with a hell month at work and have not yet had a chance to tinker, unfortunately.

I will soon!
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Bumping this thread a month later to see if you had time to play with this. Just curious if you were able to get stuff figured out or not.

So, I've started playing around a little bit today.

I didn't follow your guide exactly (and that may be why it isn't working) but what I tried to do was to use the untagged igb0 interface for LAN traffic and a tagged (VLAN7) on that same interface for the VLAN.

I did this because I had read somewhere that every port on the switch can have one default untagged membership and several tagged memberships. Is this not possible? Do I absolutely have to have two separate tagged VLAN's for this to work? I'm trying to set this up without disrupting the many existing servers and clients on my network.

Here is what I did:

On PFSense:

Created VLAN7:

upload_2018-1-1_17-59-0.png


upload_2018-1-1_17-59-51.png


Then assigned it as OPT2:

upload_2018-1-1_18-1-57.png



Then on the HP Switch, I created VLAN7 (and kept the default VLAN1)

upload_2018-1-1_18-4-48.png


I kept all ports untagged on VLAN1 as the default setup

upload_2018-1-1_18-3-41.png


Then on VLAN7 I excluded everything except port 1 (the pfsense box) and trunk2 (the box I need to be able to access trunk 7) These I added as tagged:

upload_2018-1-1_18-10-59.png



On trunk 2 is my Proxmox virtualization server. On it, I need most VM's to continue to use the LAN as normal, but this one VM to be on VLAN7 only, so I set it to tage VLAN 7 in the config for the specific VM I need to be isolated:

upload_2018-1-1_18-17-33.png



I appreciate any feedback you might have, fully understanding that I intentionally disregarded your instructions, so feel free to smack me back and tell me I cant do it that way, stupid :p

What happens now is, when the VM is not set to VLAN7, it can access everything (as expecte, and not wanted), but when set to VLAN7 it can't ping anything at all, not even the router.

I wonder, does my OPT2 interface need its own IP address instead of the default router IP (currently at 10.0.1.1)?

Appreciate all of your help!

--Matt
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I might have figured it out.

I went poking around in the networking settings in Proxmox.

Proxmox creates internal linux bridges to ethernet devices which it uses for its VM networks. My VMBR0 internal network (which communicates with Bond0, which is a LACP interface connecting to "trunk2" on the switch (in HP language a trunk is a group of aggregated ports) has a little checkbox which says "VLAN aware". This box was previously unchecked. I have now checked it, but it seems to want a reboot to make it effective.

upload_2018-1-1_19-4-21.png



The reboot will ahve to wait. I have too many other VM's running right now to want to reboot. This isnt a production box, but it might as well be given how many things in my house depend on it. Let's call it "home production".
 

octane

[H]Lite
Joined
Jun 10, 2017
Messages
94
I've donne this before setting up a WiFi hotspot using PFsense, although PFsense was used purely as the platform to negotiate the hotspot and was not used as a gateway as such.

Basically I created a DHCP server assigned to a port on the rear of the router, creating a new network for anything connected to that port.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I might have figured it out.

I went poking around in the networking settings in Proxmox.

Proxmox creates internal linux bridges to ethernet devices which it uses for its VM networks. My VMBR0 internal network (which communicates with Bond0, which is a LACP interface connecting to "trunk2" on the switch (in HP language a trunk is a group of aggregated ports) has a little checkbox which says "VLAN aware". This box was previously unchecked. I have now checked it, but it seems to want a reboot to make it effective.

View attachment 48391


The reboot will ahve to wait. I have too many other VM's running right now to want to reboot. This isnt a production box, but it might as well be given how many things in my house depend on it. Let's call it "home production".

Welp, simply making the bond VLAN aware did not do it. In fact it made the host unable to access the network. HAd to go back in on the local console and comment out the new option in /etc/network/interfaces to get it to come back up again.

Apparently it doesn't just use a default untagged VLAN1 like the switch does, once enabled.

I'll probbly have to go in and set the VLAN for every VM in order for it to work, but I'm not quite sure where I set the VLAN for the host itself...

Maybe it's better if I just give this one VM a NIC port of its own...
 

Cmustang87

[H]ardness Supreme
Joined
Oct 4, 2007
Messages
4,411
If you are making a bridged interface that is a trunk, you have to make sure you create a sub-interface for each VLAN on that bridged interface. The interface being bridged is simply a L1 construct, but the trunks and VLANs are L2.
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
Welp, simply making the bond VLAN aware did not do it. In fact it made the host unable to access the network. HAd to go back in on the local console and comment out the new option in /etc/network/interfaces to get it to come back up again.

Apparently it doesn't just use a default untagged VLAN1 like the switch does, once enabled.

I'll probbly have to go in and set the VLAN for every VM in order for it to work, but I'm not quite sure where I set the VLAN for the host itself...

Maybe it's better if I just give this one VM a NIC port of its own...

I mean if you're going to ignore my directions, the most obvious thing is to break the first cardinal rule and try to mix tagged and untagged traffic on the same interface. :p Yes the HP switch does allow you to do it, but to quote one of my instructors "Just because you can, doesn't mean you should"

I've tried mixing tagged and untagged traffic and PFSense just doesn't seem to like that at all. Often times it won't work how you want it to and you'll end up in a situation where one or the other traffic is being dropped before it leaves the adapter. It's a lot easier to get away with on the HP switch as it will just pass everything, but PFSense blocks traffic by default so you need to add rules to the firewall not only for the physical interface you bound to an OPTx connection, but also each sub interface needs specific rules as well. It's about 100x easier to never try to mix tagged and untagged traffic on the same connection, and I can't say as I've ever seen someone do it in an enterprise environment either. It's just a lot easier to put another NIC into whatever device you are using than to deal with the caveats associated with trying to mix the two. In your proxmox box either set it up so that one interface is trunked, or just use a couple of connections that are dedicated to each vlan on the switch.

A lot of times it's much easier to just let the switch handle things. If you only need 2 of your VLANs on a specific machine it's always going to be easier to just set the vlan on the switch and then the host doesn't even need to know that VLANs exist. I took at a peek at my proxmox box and that's how I'm doing it. I have vlans created for two networks for that system, but I just have 2 bridged interfaces which are not VLAN aware because the switch is handling that side of the traffic. The other upside to that is that if someone compromises that server, they can't try to add additional networks on to the system and escape into other parts of your network. (I'm sure everyone was good little boys and girls and made sure they set vlan allowed lists on all of their trunk ports /s)

So back to PFSense. STOP TRYING TO USE igb0 FOR BOTH TAGGED AND UNTAGGED TRAFFIC!!! It's not going to work! (I'm sure someone can figure it out, but I haven't yet) As soon as you create another vlan like VLAN 8, then click the little drop down for the LAN and make it say "VLAN 8 on igb0". Do that and everything that PFSense already had configured for the LAN will move over to VLAN 8 like magic. Hop on the switch, change your desktops switchport to vlan 8 and BOOM, stuff will start working again.

If I had to guess Proxmox doesn't work like you think it will. I'm pretty sure that setting that "VLAN Aware" setting on the bridge will only make it so that the proxmox network adapter will allow it to forward tagged traffic. Otherwise it likely is just dropping any traffic with tagging. So in order to actually make a VM talk on one of those interfaces, you'd actually need to go into the network properties of the adapter inside of the VM, and make sure it's tagging it's traffic with the VLAN you expect it to be talking on. Like Cmustang87 said, the bridge is just allowing it to take stuff coming into the adapter and mirror it to the other side, but about the only setting you have on that bridge is the ability to ignore or not ignore tagging.


For one of your other questions, "Do I need to put an IP onto OPT2"? No you do not. The biggest hurdle for people when moving from regular traffic to VLANs is realizing that traffic still passes on interfaces without any type of network configuration on them. L2 occurs without requiring anything fancier like IP addresses or gateways, as those are in higher layers. They rely on the VLANs to be correct, but VLANs don't rely on IPs to work. You can put IPs on these interfaces for troubleshooting, but generally you don't want to have IPs out there for no reason as you don't want your device to be able to interact directly with the traffic it's carrying. In the example of your HP switch, you can assign an IP for VLAN 7 on the switch, then from the HP switch it could talk to other clients on the VLAN. Great for troubleshooting, but later on if VLAN 7 is a malware ridden network, you wouldn't want anything to be able to access your switch from VLAN 7. Even without the IP on there, the HP switch will happily pass traffic from your PFSense box to the end client, allowing things to work without putting the switch at risk. The clients are not aware they are on a VLAN as they cannot see the vlan tags as these are stripped by the switch before the data is sent to the client. This is what makes VLANs a good, clean way to handle lots of traffic without as much physical overhead (cabling, ports, etc)
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I mean if you're going to ignore my directions, the most obvious thing is to break the first cardinal rule and try to mix tagged and untagged traffic on the same interface. :p Yes the HP switch does allow you to do it, but to quote one of my instructors "Just because you can, doesn't mean you should"

I've tried mixing tagged and untagged traffic and PFSense just doesn't seem to like that at all. Often times it won't work how you want it to and you'll end up in a situation where one or the other traffic is being dropped before it leaves the adapter. It's a lot easier to get away with on the HP switch as it will just pass everything, but PFSense blocks traffic by default so you need to add rules to the firewall not only for the physical interface you bound to an OPTx connection, but also each sub interface needs specific rules as well. It's about 100x easier to never try to mix tagged and untagged traffic on the same connection, and I can't say as I've ever seen someone do it in an enterprise environment either. It's just a lot easier to put another NIC into whatever device you are using than to deal with the caveats associated with trying to mix the two. In your proxmox box either set it up so that one interface is trunked, or just use a couple of connections that are dedicated to each vlan on the switch.

A lot of times it's much easier to just let the switch handle things. If you only need 2 of your VLANs on a specific machine it's always going to be easier to just set the vlan on the switch and then the host doesn't even need to know that VLANs exist. I took at a peek at my proxmox box and that's how I'm doing it. I have vlans created for two networks for that system, but I just have 2 bridged interfaces which are not VLAN aware because the switch is handling that side of the traffic. The other upside to that is that if someone compromises that server, they can't try to add additional networks on to the system and escape into other parts of your network. (I'm sure everyone was good little boys and girls and made sure they set vlan allowed lists on all of their trunk ports /s)

So back to PFSense. STOP TRYING TO USE igb0 FOR BOTH TAGGED AND UNTAGGED TRAFFIC!!! It's not going to work! (I'm sure someone can figure it out, but I haven't yet) As soon as you create another vlan like VLAN 8, then click the little drop down for the LAN and make it say "VLAN 8 on igb0". Do that and everything that PFSense already had configured for the LAN will move over to VLAN 8 like magic. Hop on the switch, change your desktops switchport to vlan 8 and BOOM, stuff will start working again.

If I had to guess Proxmox doesn't work like you think it will. I'm pretty sure that setting that "VLAN Aware" setting on the bridge will only make it so that the proxmox network adapter will allow it to forward tagged traffic. Otherwise it likely is just dropping any traffic with tagging. So in order to actually make a VM talk on one of those interfaces, you'd actually need to go into the network properties of the adapter inside of the VM, and make sure it's tagging it's traffic with the VLAN you expect it to be talking on. Like Cmustang87 said, the bridge is just allowing it to take stuff coming into the adapter and mirror it to the other side, but about the only setting you have on that bridge is the ability to ignore or not ignore tagging.


For one of your other questions, "Do I need to put an IP onto OPT2"? No you do not. The biggest hurdle for people when moving from regular traffic to VLANs is realizing that traffic still passes on interfaces without any type of network configuration on them. L2 occurs without requiring anything fancier like IP addresses or gateways, as those are in higher layers. They rely on the VLANs to be correct, but VLANs don't rely on IPs to work. You can put IPs on these interfaces for troubleshooting, but generally you don't want to have IPs out there for no reason as you don't want your device to be able to interact directly with the traffic it's carrying. In the example of your HP switch, you can assign an IP for VLAN 7 on the switch, then from the HP switch it could talk to other clients on the VLAN. Great for troubleshooting, but later on if VLAN 7 is a malware ridden network, you wouldn't want anything to be able to access your switch from VLAN 7. Even without the IP on there, the HP switch will happily pass traffic from your PFSense box to the end client, allowing things to work without putting the switch at risk. The clients are not aware they are on a VLAN as they cannot see the vlan tags as these are stripped by the switch before the data is sent to the client. This is what makes VLANs a good, clean way to handle lots of traffic without as much physical overhead (cabling, ports, etc)

You have been 100% correct thus far.

Pfsense does not appear to like running untagged and tagged at the same time.

I have been delaying making this change due to the pain in the ass it is to open up the web configuration to the wan, and then go in there if it stops responding.

I'm thing I can take advantage of recent pfSense versions automatic config backup function though. If I make the change, and I lose communication of th epfsense box, I can probably just hook up a VGA monitor and an old keyboard and restore the recent config backup that way.

Maybe I'll try it again tomorrow.

Thanks again for all your help!

--Matt
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
You have been 100% correct thus far.

Pfsense does not appear to like running untagged and tagged at the same time.

I have been delaying making this change due to the pain in the ass it is to open up the web configuration to the wan, and then go in there if it stops responding.

I'm thing I can take advantage of recent pfSense versions automatic config backup function though. If I make the change, and I lose communication of th epfsense box, I can probably just hook up a VGA monitor and an old keyboard and restore the recent config backup that way.

Maybe I'll try it again tomorrow.

Thanks again for all your help!

--Matt
If you have another interface, you can always use that for management. I think your board might have wifi, so you could always setup the wireless to allow access to the GUI if PFsense detects your wireless interface. Then you could leave that setup in place to make troubleshooting a bit easier. I'd even give a try at a USB interface, you might have a usb to Ethernet that might work as well.
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
If you have another interface, you can always use that for management. I think your board might have wifi, so you could always setup the wireless to allow access to the GUI if PFsense detects your wireless interface. Then you could leave that setup in place to make troubleshooting a bit easier. I'd even give a try at a USB interface, you might have a usb to Ethernet that might work as well.
I have a monitor and a keyboard sitting on a workbench around the corner from my ghetto "rack" where I keep my networking stuff.

As luck would have it, the wires are long enough, and I get local console output on the screen, so if I hose everything in my switch to tagged VLAN1 for the main network, restoring it to the previous configuration should be a piece of cake.

Modern versions of pfSense apparently automatically backup their config file every time a change is made, and keep the 30 or so most recent backups in history, with a nice little change log.

Code:
matt@Matt-Desktop ~ $ ssh -l root 10.0.1.1
Password for root@router.localdomain:
pfSense - Netgate Device ID: xxxxxxxxxxxxxxxxxxxxxxxxxxx

*** Welcome to pfSense 2.4.2-RELEASE-p1 (amd64) on router ***

 WAN (wan)       -> em0        -> v4/DHCP4: xxx.xxx.xxx.xxx/24
 LAN (lan)       -> igb0       -> v4: 10.0.1.1/24
 PIAVPNINTERFACE (opt1) -> ovpnc1     -> v4: xxx.xxx.xxx.xxx/32
 OPT2 (opt2)     -> igb0.7     -> 

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell + pfSense tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Disable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

Enter an option: 15


Restore Backup from Configuration History

1) List Backups
2) Restore Backup
Q) Quit


Please select an option to continue: 1
30. 9/3/17 16:12:18    v15.8    admin@10.0.1.116
    /firewall_rules_edit.php made unknown change

29. 9/3/17 16:13:45    v15.8    admin@10.0.1.116
    /firewall_rules_edit.php made unknown change

28. 9/3/17 16:16:45    v15.8    admin@10.0.1.116
    /firewall_rules_edit.php made unknown change

27. 9/22/17 21:25:49    v15.8    admin@10.0.1.118
    /firewall_shaper.php made unknown change

26. 9/22/17 21:25:55    v15.8    admin@10.0.1.118
    /firewall_shaper.php made unknown change

25. 9/22/17 21:25:59    v15.8    admin@10.0.1.118
    /firewall_shaper.php made unknown change

24. 10/14/17 16:20:04    v17.0    (system)
    Upgraded config version level from 15.8 to 17.0

23. 10/14/17 16:20:19    v17.0    (system)
    Intermediate config write during package install for nut.

22. 10/14/17 16:20:22    v17.0    (system)
    Overwrote previous installation of nut.

21. 10/14/17 16:20:23    v17.0    (system)
    Overwrote previous installation of Cron.

20. 10/14/17 16:22:02    v17.0    admin@10.0.1.116
    Widget configuration has been changed.

19. 10/15/17 23:22:33    v17.0    admin@10.0.1.116
    Creating restore point before package installation.

18. 10/15/17 23:22:44    v17.0    (system)
    Intermediate config write during package install for ntopng.

17. 10/15/17 23:22:47    v17.0    Unknown Page
    Unknown

16. 10/15/17 23:25:13    v17.0    Unknown Page
    Unknown

15. 10/15/17 23:25:16    v17.0    Unknown Page
    Unknown

14. 11/5/17 15:07:44    v17.3    (system)
    Upgraded config version level from 17.0 to 17.3

13. 11/5/17 15:08:22    v17.3    (system)
    Intermediate config write during package install for ntopng.

12. 11/5/17 15:09:18    v17.3    (system)
    Overwrote previous installation of ntopng.

11. 11/13/17 13:46:16    v17.3    admin@10.0.1.244
    Dynamic DNS client configured.

10. 11/13/17 13:47:28    v17.3    admin@10.0.1.244
    Dynamic DNS client configured.

09. 11/13/17 16:00:20    v17.3    admin@10.0.1.244
    Dynamic DNS client configured.

08. 11/13/17 16:06:33    v17.3    admin@10.0.1.244
    Dynamic DNS client configured.

07. 11/21/17 21:26:50    v17.3    admin@10.0.1.118
    Creating restore point before package installation.

06. 12/15/17 22:45:07    v17.3    admin@10.0.1.116
    Creating restore point before package installation.

05. 1/1/18 16:02:31    v17.3    admin@10.0.1.116
    /interfaces_vlan_edit.php made unknown change

04. 1/1/18 17:43:55    v17.3    admin@10.0.1.116
    /interfaces_assign.php made unknown change

03. 1/1/18 17:47:36    v17.3    admin@10.0.1.116
    /interfaces.php made unknown change

02. 1/1/18 17:49:38    v17.3    admin@10.0.1.116
    /interfaces.php made unknown change

01. 1/1/18 17:50:17    v17.3    admin@10.0.1.116
    /interfaces_vlan_edit.php made unknown change


Restore Backup from Configuration History

1) List Backups
2) Restore Backup
Q) Quit


Please select an option to continue:
As you can see, my last attempt of messing with the interfaces is listed there on January first, so if I wanted to undo it, I'd just choose to restore the appropriate backup.


Not sure when this feature was added. It certainly didn't use to do this, but I like it a lot. It already saved my ass once, when my config file somehow became corrupted.

So, I already have my secondary VLAN configured. Now I'm going to add VLAN 1 for th emain network, and set my router port on my switch to tagged. Wish me luck!

--Matt
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
And it worked!

Well, I'm not done yet, but I now have my main LAN up and running on VLAN2.

I did deviate slightly from your instructions. Rather than dedicate a port for management (I'm always running low on switch ports) in the switch network setup, I changed the management VLAN from 1 to 2, so I can now access it on the main lan.

So, my method for doing this was something like this:

1.) Do all the pfSense stuff, assign LAN interface as VLAN2, hit apply and lose connection with router as expected.

2.) Create VLAN2 on switch.

3.) Assign VLAN2 ports. Router port is tagged VLAN 2, all others are untagged, EXCEPT the port used for my desktop, which stays on the default VLAN1.

4.) Desktop will now have no networkj connectivity other than the switch management interface.

4.) Using desktop, switch management VLAN from 1 to 2. Desktop will now lose access to switch management interface.

5.) using phone via wifi (or other computer on network) log on to management interface on switch, and now change desktop port VLAN from being untagged on 1 to being untagged on 2.

6.) The desktop has now rejoined the world.

I did think I had failed for a second though, as after doing all this, I had no connectivity to the pfSense box or the outside world. It turns out, after changing the main LAN interface to the VLAN2 interface, it didn't work until the pfSense box got a reboot for some reason. After the reboot all of my activity LED's on my switch instantly came to life, telling me I had succeeded.
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
That's all and good now, but obviously I left out any mention of VLAN 3. We essentially did a bunch of work to accomplish exactly what we had before. All of the ports that are on VLAN 2 can talk just like they did before, except for port 3. At this point port 3 is not working, because there are more steps. On the switch side VLAN 3 is configured. You can verify this by putting another port on VLAN 3, and the two computers will talk to each other. We set the participation for your "trunk" to include this VLAN, and we even set the interface in PFSense for VLAN 3. What we need to do now, is actually enable that interface in PFSense, give it an IP, and then put some firewall rules in place to allow it to go places. All of that is taken care of by default for you when you setup your PFSense box, but we're going to have to do all of that now for the new interface.

So login to your PFSense box, and then go to Interfaces > OPT1. The first thing you'll need to do is click the checkbox to enable interface. This is important to know because PFSense will hide it on other screens if you don't have it enabled. I'm pretty sure it doesn't show up in the firewall rules until it's enabled. Let's give this an IP. For the IPv4 configuration type, choose static IP. Then you can set the static IPv4 configuration to something, I'll just pick 192.168.3.1 /24 for now. Make sure the block private IPs checkbox at the bottom is unchecked, and click save.

Now click on Firewall > Rules, and you should have an OPT1 tab that you didn't have before. It will say something about No rules defined at this point. To just make stuff work for testing, we're going to go with the wide open configuration, so you can make sure it work. So click either one of the add buttons, make sure the action is set to "pass" the protocol is set to "Any". The other stuff should be any by default, so make sure it is and click save. The basically turned off the firewalling on that port, so later we need to put rules in place, like a rule that blocks outbound traffic to your LAN subnet, so stuff on "New LAN" (OPT1) can't talk to stuff on the original LAN.

Chances are high that you want to hand out leases for this interface, so we need to go to Services > DHCP server. Once again you'll now have an OPT1 tab that didn't exist before. Click the box for enable, and setup your pool however you want. Click save and that should start the DHCP server for that interface.

So now we are ready to test VLAN 3. From port 3 on the switch, you should be getting a DHCP lease from the pool you just created. Since we opened up the firewall rules, you should be able to ping the internet and anything that is on the "LAN" which is VLAN 2. If that works then you're in business and you've successfully gotten it all to work!

At this point creating new VLANs is as simple as going through the same process, making sure you create both sides of it, setting the port as tagged for the connection that goes between PFSense and the switch, configuring a port to use it, and then doing the configuration on the PFSense side to enable it. There's definitely a few steps, but once you have stuff working the new interfaces is a lot easier. You'll probably want to setup firewall rules to start blocking things, and for reference the PFSense web GUI will be accessible from any IP it has. So you can get into it now from VLAN 3 if you type 192.168.3.1 into your browser. This means you can probably turn off access from the WAN again, but you might also want to block web access from VLAN 3 as well.

EDIT: Also, After you determine that VLAN 3 is working, that is the time to make another set of configuration backups for both the switch and the firewall. Cause when you break it after that you now have a point to revert back to where everything was working with VLANs in place.
Alright,

Unfortunately here is where it all fell apart.

VLAN2 is up and running perfectly as my main LAN. No problems there at all.

I decided it would be easier to not deal with Proxmox vlan tagging over its internal bridge interfaces. Since I had a spare port on the server, I just assigned that directly to the one VM I want on the separate VLAN, and am using that port untagged. Once I get this working, I may go back later and see if I can get tagging across to the VM server to work as a second step, but for now I figured it was easier to separate these two and get the first part working first.

The separated LAN on VLAN7 port can't ping anything at all inside or outside the network, not even pfsense. (Though I have confirmed that ir works, as if I plug it into a different port on the switch, it behaves like expected on the main LAN)


So, here is what it looks like:

VLAN2 (Main LAN, This works fine)

upload_2018-1-21_16-40-19.png


VLAN7 (Separated LAN)

upload_2018-1-21_16-42-19.png


So, the intent here is it gets VLAN7 tags on the same port as VLAN2, but only passes them to port 21, where the one machine I need to be separate resides.


Here are my interface assignments:

upload_2018-1-21_16-45-2.png



I created my "pass any" rule on the SeparateLan interface:

upload_2018-1-21_16-46-14.png


But when I go to the DHCP server, there is only one tab, and that is the LAN tab, no SeparateLAN Tab:

upload_2018-1-21_16-47-28.png



I'm not too worried about the DHCP itself, I only plan on using statically assigned IP's on the separate LAN, but it does seem like an indication that something is wrong.

Any ideas?


Could it have somethign to do with my outbound NAT rules set to manual, due to my use of a VPN?

upload_2018-1-21_16-49-30.png


At first, I thought this might be it, but this shouldn't be preventing me from pinging the router's LAN interface, so the problem appears to be on the LAN side.

The server that needs to be separated is currently using ammanual IP configuration with an ip address on the same subnet as the main LAN. Could this be the problem?


As always, I appreciate any help!

I need to send you a few beers or something!

--Matt
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
My SeparateLan interface still does not have an IP address like you suggest, but this old forum post suggests that maybe I need to add one, but on a different subnet?

My main net is 10.0.1.0/24

Maybe I should add an ip address to the secondary LAN interface with a different subnet, like 10.0.2.0/24?

Granted, that forum post is 7 years old, from a much earlier version of pfSense, and is not specifically for VLAN's but for using a separate physical interface (though in much the same waty as I am trying to accomplish).

As always, appreciate any suggestions.

--Matt
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Update:

By giving the Separate Lan (VLAN7) interface an ip address on a different subnet (I chose 10.0.7.0/24) and changing the static IP address on the server to be separated to the same subnet, I was able to ping the router.

At least now I know that the VLAN stuff is working, even if this is the incorrect way to set things up.

That is about all I got though. No routing through to the WAN.

Maybe since I have manual advanced WAN NAT enabled, I need to add some rules here for the separate subnet. I could try duplicating every rule I hvae for my main subnet, but changing it to the new one...
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
Maybe since I have manual advanced WAN NAT enabled, I need to add some rules here for the separate subnet. I could try duplicating every rule I have for my main subnet, but changing it to the new one...
Woo. That did the trick!


upload_2018-1-21_17-31-25.png


So I now have a separate subnet, able to reach the world outside the WAN, and not able to ping my main subnet, with or without switching the IP back to the original subnet.

I'd call that a success.

Now I need to figure out how to prevent this new subnet from reaching the web interface or SSH port on the router.

I mean, they are password protected, but I still don't want them exposed to the separate subnet.

The web interface has options to turn them off or on, but I cant find any on a per interface basis.

Can I just create a firewall rule to block ports 22, 80 and 443 on the 10.0.7.1 interface, or will this mess up anything on the routing side?
 

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
This seems to have done the trick.

upload_2018-1-21_17-53-29.png



Port 80 isn't open on pfSense, but I don't know how their port redirect to https works, so I blocked them both just to be on the safe side.

This does not appear to have disrupted anything else.

I'm going on the assumption here that blocking TCP is all I need to do. I don't think these ports will accept UDP connections. Network security is far from my strength though.
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
This seems to have done the trick.

View attachment 51014


Port 80 isn't open on pfSense, but I don't know how their port redirect to https works, so I blocked them both just to be on the safe side.

This does not appear to have disrupted anything else.

I'm going on the assumption here that blocking TCP is all I need to do. I don't think these ports will accept UDP connections. Network security is far from my strength though.
I'd probably just go ahead any block * protocol anyway. Basically look at the anti lockout rule, and copy that one, but then change it from allow to block.

Here's the kicker though, If your secondary network is 10.0.7.0/24, realize that the allow rule below it still allows that subnet to access your PFSense web page. How you say? Because it has access to the interface that is sitting on the LAN because of the ANY / ANY rule. You'll want another block rule just above the any rule that does something like this:

Protocol *, Source: Separate LAN Net, Port *, Destination: LAN Net, Port *, Gateway *.

That will basically block any traffic originating from the Separate LAN to reach any hosts on your LAN. The reverse will still work because on the LAN rule you have an any / any with no blocks. So the LAN can reach Separate LAN, but Separate LAN won't be able to reach LAN.

I would also switch your rule from explicitly stating 10.0.7.1 to say Separate LAN Address. That should point PFsense to whatever IP is currently assigned to that interface, allowing your rules to be dynamic whenever you make a change. Change the IP on that interface, the rules will instantly match.


You already figured it out, but yes, on the PFSense side you need to have any IP on any VLAN you create IF you want it to route traffic. Routing is in Layer 3, which requires a protocol to flow so traffic can be routed. PFSense can pass traffic through a VLAN interface to another interface if they are bridged without an IP, but as soon as you hit the routing layer you need a protocol to route with. Technically you could route using IPX, IPv4, IPv6 or probably even other protocols, but you have to have some type of routable protocol functioning for it to work. So in our case the most obvious is IPv4 being enabled and setup to flow.


As far as trying to block the Separate LAN interface 10.0.7.1 entirely, you'd have to play around and see what it does. I generally don't get too carried away with that type of stuff because on that particular interface you can have a lot of unintended consequences. Just by looking at the state table, I actually don't believe it would block traffic headed for the WAN if you blocked a host on that subnet from trying to access that IP directly. (I think it's only going to filter traffic which is directed at that interface, not all traffic passing through it). BUT, stuff will get goofy in a hurry because things like DHCP or even DNS require you to contact that IP interface directly, so you'd need to make sure if you're using those services you open up Port 67, and port 53 otherwise things will just stop working.

If you go to diagnostics > Sockets, you can see what services are listening. By default a lot of services are listening on *, which basically means they will respond on ANY interface. (Including WAN, but by default all inbound traffic heading to the WAN is blocked, so nothing can access it. If you put in allow rules you don't have to actually rebind the service to get it to work. Which you already found out with SSH. It's listening on anything, but it's up to you to control what interfaces are allowed or blocked)
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,461
I'd probably just go ahead any block * protocol anyway. Basically look at the anti lockout rule, and copy that one, but then change it from allow to block.

Here's the kicker though, If your secondary network is 10.0.7.0/24, realize that the allow rule below it still allows that subnet to access your PFSense web page. How you say? Because it has access to the interface that is sitting on the LAN because of the ANY / ANY rule. You'll want another block rule just above the any rule that does something like this:

Protocol *, Source: Separate LAN Net, Port *, Destination: LAN Net, Port *, Gateway *.

That will basically block any traffic originating from the Separate LAN to reach any hosts on your LAN. The reverse will still work because on the LAN rule you have an any / any with no blocks. So the LAN can reach Separate LAN, but Separate LAN won't be able to reach LAN.

I would also switch your rule from explicitly stating 10.0.7.1 to say Separate LAN Address. That should point PFsense to whatever IP is currently assigned to that interface, allowing your rules to be dynamic whenever you make a change. Change the IP on that interface, the rules will instantly match.


You already figured it out, but yes, on the PFSense side you need to have any IP on any VLAN you create IF you want it to route traffic. Routing is in Layer 3, which requires a protocol to flow so traffic can be routed. PFSense can pass traffic through a VLAN interface to another interface if they are bridged without an IP, but as soon as you hit the routing layer you need a protocol to route with. Technically you could route using IPX, IPv4, IPv6 or probably even other protocols, but you have to have some type of routable protocol functioning for it to work. So in our case the most obvious is IPv4 being enabled and setup to flow.


As far as trying to block the Separate LAN interface 10.0.7.1 entirely, you'd have to play around and see what it does. I generally don't get too carried away with that type of stuff because on that particular interface you can have a lot of unintended consequences. Just by looking at the state table, I actually don't believe it would block traffic headed for the WAN if you blocked a host on that subnet from trying to access that IP directly. (I think it's only going to filter traffic which is directed at that interface, not all traffic passing through it). BUT, stuff will get goofy in a hurry because things like DHCP or even DNS require you to contact that IP interface directly, so you'd need to make sure if you're using those services you open up Port 67, and port 53 otherwise things will just stop working.

If you go to diagnostics > Sockets, you can see what services are listening. By default a lot of services are listening on *, which basically means they will respond on ANY interface. (Including WAN, but by default all inbound traffic heading to the WAN is blocked, so nothing can access it. If you put in allow rules you don't have to actually rebind the service to get it to work. Which you already found out with SSH. It's listening on anything, but it's up to you to control what interfaces are allowed or blocked)
Thank you very much man.

This thread, and the process of figuring this whole thing out has helped grow my understanding of firewalls, rules and routing more than my 25 previous years of being on the internet, tinkering with PC's and servers combined. It's been a HUGE help.

If you ever find yourself in the greater Boston area, I owe you a beer (or 7)!

--Matt
 

bman212121

[H]ard|Gawd
Joined
Aug 18, 2011
Messages
1,558
Glad this thread has helped and you got it sorted out. Part of the reason why I joined this forum in the first place is because there seemed to be a lot more technical knowledge from the members, and you could actually find info that would help you solve an issue. Everyone has to start somewhere so it's great to document the process and learn from it. I'm glad I was able to put my knowledge that I've learned to help someone else get on their way, and hopefully just by having this thread out there someone else might come along and find that missing piece of the puzzle as to why their setup wasn't working.

Now that you have a handle on it, trying to do it again from scratch the next time will only be half as hard. I can tell you from experience you'll forget a step the next time you do it, so hopefully some of the notes in this thread will jog your memory and it won't take as long to get on the right track. After the 2nd or 3rd time it should all start clicking, then it will be easy enough to start putting several VLANs on your config. The biggest advantage is that once you understand how it works on one set of gear, you can pick up other vendors and figure out their nuances and get them working together.
 

jaxn

n00b
Joined
Oct 30, 2004
Messages
55
Hi All, Late to the party but I wanted to share for others later on. Mark Furneaux's (The Ubuntu Guy on Youtube) Comprehensive Guide To pfSense 2.3 was a huge help for me when learning about and setting up a pfSense box. I also wanted VLANs but for my gaming console and wifi traffic so a friend pointed me to his channel. Glad to see you were able to work out a resolution here!
 
Top