Spanning L2 on an L3 network - physically separate links or not?

[hutchingsp]

I'm reasonably sure that the "right" way to redesign our currently massive flat network is to look to do OSPF between core and distribution - nothing unusual there.

I'm also reasonably sure that we will be doing L2 between distribution and access for our own VLANs such as data and voice - L3 would be nice but it's too pricey to do OSPF on our access layer and we still end up a lot better off than we are now, so nothing terribly unusual there.

We have an issue in that we do need to make some L2 VLANs available anywhere/everywhere across our site.

As we have sufficient SM/MM in the core and distribution locations would it be sensible to use dedicated physical paths between distribution and core layers just for the L2 trunks?

Doing so would mean the OSPF links would be pure point to point - no need for trunks.

It seems to make stuff like spanning tree less of an issue as we'd still need to run it but using dedicated physical interfaces for the L2 trunks feels like it should be simpler?

That could be due to lack of knowledge about exactly how spanning tree and OSPF could interact running on the same physical interface.

It's a compromise but the stuff that requires L2 is beyond our control.

Any suggestions on whether we're setting ourselves up for a massive fall here?

 

[Cmustang87]

Kind of hard to give you any advice OP without any documentation. It's a little tricky to understand what you are trying to do and how your infrastructure is currently set up.

Your language is a bit confusing, can you clean up the post a little bit and elaborate a little better with what you are trying to achieve? Any documentation would help, but please obfuscate any private information.

 

[hutchingsp]

Fair point and apologies, I know enough to get by but not enough "official" terminology

This is the best parallel I've found to the situation we have: http://packetlife.net/blog/2011/feb/9/hybrid-access-layer-design-revisited/

Right now we're literally flat network, no routing at all.

 

[Cmustang87]

Well let's clear up a few things here:

OSPF is a routing protocol
STP is a layer 2 protocol that prevents broadcast storms due to redundant switch links.

Running open shortest path first (OSPF) really has no implications in relation to running spanning tree protocol (STP). With that said, let's knock some of this out:

I'm reasonably sure that the "right" way to redesign our currently massive flat network is to look to do OSPF between core and distribution - nothing unusual there.

OSPF in your scenario would be implemented so your distribution switches can reach other networks that are routed through your core, generally. This will only be possible if your distribution and core switches are L3 operational.

I'm also reasonably sure that we will be doing L2 between distribution and access for our own VLANs such as data and voice - L3 would be nice but it's too pricey to do OSPF on our access layer and we still end up a lot better off than we are now, so nothing terribly unusual there.

You can do this. You would connect a simple L2 switch (access) to a distribution switch with a Layer 3 interface, or switch virtual interface (SVI). The default gateway for that VLAN would be the SVI IP address or the Layer 3 interface. SVIs are virtual interfaces that correlate to a VLAN. This gives the switch Layer 3 capabilities along with management, since you have an IP address to work with. Anyone on that corresponding VLAN can reach that SVI. A layer 3 interface on a switch would be a physical port that has an IP address assigned to it, identical to a real router.

We have an issue in that we do need to make some L2 VLANs available anywhere/everywhere across our site.

As we have sufficient SM/MM in the core and distribution locations would it be sensible to use dedicated physical paths between distribution and core layers just for the L2 trunks?

If you can afford private fiber between sites and to your core, then I don't see why not. This is where things will kind of break down a little bit depending on how it's really set up. What will generally happen is 1 of 2 situations:
ScenarioA) You have another network that you route through to the core. This would just be a simple L3 link.
ScenarioB) You need this to be a trunk.

Doing so would mean the OSPF links would be pure point to point - no need for trunks.

True, unless you had hosts connecting through the core that needed to be on the same VLAN as the other switch.

It seems to make stuff like spanning tree less of an issue as we'd still need to run it but using dedicated physical interfaces for the L2 trunks feels like it should be simpler?

That could be due to lack of knowledge about exactly how spanning tree and OSPF could interact running on the same physical interface.

They operate at different layers. STP will protect your network from broadcast storms by blocking ports that are redundant to the root bridge. OSPF is just a dynamic routing protocol. Let's say you have redundant "CoreSwA" and "CoreSwB". "DistSwA" has a link up to "CoreSwA" and "CoreSwB". "CoreSwA" is the root bridge, so the link connected to "CoreSwB" would go into a blocking state which will prevent frames from being flooded out that port. Sorry if I'm repeating something you already know.

 

[hutchingsp]

Things did move on a little since the original post - we now *think* that a hybrid model with a single physical link with SVI's for the routed links and L2 trunked where we need it would work using VSTP to handle blocking on a per VLAN basis.

There wouldn't ever be L3 loops on the RVIs so it *should* only ever block based on L2.

Also I'm using the word "sites" when we're a single large site geographically - all the SM/MM is private.

 

[Cmustang87]

Kind of lost me a little bit there, but... just make sure you document the ever-loving hell out of it, haha. Are you the main person behind it?

 

[hutchingsp]

It's a little political - I'm not responsible for it but I have enough of an interest (and likelihood of being expected to deal with any issues depending who's around) that it's important it's done correctly.

I'm pretty sure VSTP would do what we need by handling loops at a VLAN level - just after any kind of sanity checks