![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Hello all, I just got a chance to start configuring my new Point-to-point fiber network from comcast. I plan on migrating over to this soon-ish, but in my testing I already discovered a fairly massive issue (other than me having no experience with this sort of thing!). The general setup is as follows (actual VLANs and IPs are different, as usual):
P2P IP range - 5.5.5.5/30 (gateway to internet is 5.5.5.6 and my WAN port on the firewall is 5.5.5.7)
Site 1 - VLAN 1, IP Range 1.1.1.0/24 - Firewall, L3 switch, and main Ciena switch physically located here
Site 2 - VLAN 2, IP Range 2.2.2.0/24 - Regular ciena switch with a fully L2 internal network
Site 3 - VLAN 3, IP Range 3.3.3.0/24 - Regular ciena switch with a fully L2 internal network
Site 4 - VLAN 4, IP Range 4.4.4.0/24 - Regular ciena switch with a fully L2 internal network
Site 1 has the fortinet firewall acting as the main in/out for internet traffic, which is then connected to a Layer 3 aruba switch, which is then connected to the internal L2 network of Site 1. Port 1 of the ciena goes into the WAN port of the firewall, set the WAN port to be 5.5.5.7 with a static route of 0.0.0.0/0 5.5.5.6 gets the whole building right out to the internet (at record speeds too, im going from a 150/25 to synchronous gig!). I plug port 2 of the ciena (I was told this is my inter site port) into port 1 of the L3 switch (which is tagged to accept any VLAN from all of my sites). Site 1s LAN is connected locally to the L3 switch using a trunked pair of 1g fiber cables, the gateway address of each site (X.X.X.1) is located on the L3 switch, with a 0.0.0.0/0 static route to the internal port of the firewall. Site 1 is functioning perfectly fine on this config.
Sites 2 - 4 have no Layer 3 devices currently setup, the extremely limited help comcast would give me (apparently theyre not allowed to consult or configure customer equipment, the dude couldnt even tell me to run a ping to the new gateway....) led me to believe I can treat each site as a "layer 2" site. My understanding might be off, but if the sites are layer 2, then the Ciena switch would need my VLANs tagged on it in order to pass traffic, comcast scoffed at that notion, yet couldnt explain why. The material ive read, and the general murmorings of the internet would lead me to believe that while comcasts portion of the P2P network might be effectively layer 2, I should be able to treat it as layer 1. That is to say, a gigantic cable running between my sites with a single point of termination, to me this is the only way that I would be able to pass traffic through it without worrying about the physical switchports that my sites are plugged into on the ciena, the alternative would be that the ciena switchports come pre-tagged for VLANs 1-4094, either way this should allow me to consider this connection a layer 1 connection rather than layer 2, at least from a configuration standpoint. Would I be correct in my understanding of this, or am I completely off?
The issue I am having is that while I am able to get my site 1 moved over to the fiber connection just fine, I did a test run with site 2 where I disconnected it from its current firewall (used for the coax connection its on now) and plugged port 1 of that ciena switch into a switchport on site 2s L2 internal network, yet I cant ping any devices located at site 2 from site 1. I rebooted the stack thinking that maybe its still searching for its GW port (2.2.2.1) on the MAC address of the firewall, didnt help. I then rebooted the L3 switch at Site 1, which also didnt help. Inter VLAN routing is configured on the L3 switch, and I can ping the gateway for site 2 from the LAN on site 1. Do I need to do anything special to get site 2 to find its new gateway on the L3 switch at site 1? The way I am testing this is to run a ping to Site 2s core switch stack from site 1s LAN network, I also made a port on the L3 switch an untagged member of VLAN 2 trying to rule out any mis-tagging of the VLAN, but I still cant ping anything at site 2 from site 1. Any help would be greatly appreciated, and I can definitely provide any additional details that you need.
Thank you!
Smoblikat
P2P IP range - 5.5.5.5/30 (gateway to internet is 5.5.5.6 and my WAN port on the firewall is 5.5.5.7)
Site 1 - VLAN 1, IP Range 1.1.1.0/24 - Firewall, L3 switch, and main Ciena switch physically located here
Site 2 - VLAN 2, IP Range 2.2.2.0/24 - Regular ciena switch with a fully L2 internal network
Site 3 - VLAN 3, IP Range 3.3.3.0/24 - Regular ciena switch with a fully L2 internal network
Site 4 - VLAN 4, IP Range 4.4.4.0/24 - Regular ciena switch with a fully L2 internal network
Site 1 has the fortinet firewall acting as the main in/out for internet traffic, which is then connected to a Layer 3 aruba switch, which is then connected to the internal L2 network of Site 1. Port 1 of the ciena goes into the WAN port of the firewall, set the WAN port to be 5.5.5.7 with a static route of 0.0.0.0/0 5.5.5.6 gets the whole building right out to the internet (at record speeds too, im going from a 150/25 to synchronous gig!). I plug port 2 of the ciena (I was told this is my inter site port) into port 1 of the L3 switch (which is tagged to accept any VLAN from all of my sites). Site 1s LAN is connected locally to the L3 switch using a trunked pair of 1g fiber cables, the gateway address of each site (X.X.X.1) is located on the L3 switch, with a 0.0.0.0/0 static route to the internal port of the firewall. Site 1 is functioning perfectly fine on this config.
Sites 2 - 4 have no Layer 3 devices currently setup, the extremely limited help comcast would give me (apparently theyre not allowed to consult or configure customer equipment, the dude couldnt even tell me to run a ping to the new gateway....) led me to believe I can treat each site as a "layer 2" site. My understanding might be off, but if the sites are layer 2, then the Ciena switch would need my VLANs tagged on it in order to pass traffic, comcast scoffed at that notion, yet couldnt explain why. The material ive read, and the general murmorings of the internet would lead me to believe that while comcasts portion of the P2P network might be effectively layer 2, I should be able to treat it as layer 1. That is to say, a gigantic cable running between my sites with a single point of termination, to me this is the only way that I would be able to pass traffic through it without worrying about the physical switchports that my sites are plugged into on the ciena, the alternative would be that the ciena switchports come pre-tagged for VLANs 1-4094, either way this should allow me to consider this connection a layer 1 connection rather than layer 2, at least from a configuration standpoint. Would I be correct in my understanding of this, or am I completely off?
The issue I am having is that while I am able to get my site 1 moved over to the fiber connection just fine, I did a test run with site 2 where I disconnected it from its current firewall (used for the coax connection its on now) and plugged port 1 of that ciena switch into a switchport on site 2s L2 internal network, yet I cant ping any devices located at site 2 from site 1. I rebooted the stack thinking that maybe its still searching for its GW port (2.2.2.1) on the MAC address of the firewall, didnt help. I then rebooted the L3 switch at Site 1, which also didnt help. Inter VLAN routing is configured on the L3 switch, and I can ping the gateway for site 2 from the LAN on site 1. Do I need to do anything special to get site 2 to find its new gateway on the L3 switch at site 1? The way I am testing this is to run a ping to Site 2s core switch stack from site 1s LAN network, I also made a port on the L3 switch an untagged member of VLAN 2 trying to rule out any mis-tagging of the VLAN, but I still cant ping anything at site 2 from site 1. Any help would be greatly appreciated, and I can definitely provide any additional details that you need.
Thank you!
Smoblikat