Using MPLS on the WAN is a great way for multi-customer WAN connectivity, but so far it’s all layer 3. Layer 2 technologies like Ethernet over MPLS (EoMPLS) exist, but I have little experience with it so far, so I’m not covering it (yet).
Instead, something else: layer 2 WAN services. Let’s assume you need to transport layer 2 VLANs from multiple customers over WAN links. Using a dedicated line per customer makes configuration easy, but it’s not cost-efficient. Instead, you want to use a mesh of switches that transport the customer data. The concept is simple enough, but it has some practical limitations: you can’t have overlapping VLAN numbers between customers.
This is where Metro Ethernet switches step into the picture. In particular, I had the chance to work with a ME-3600X-24CX-M switch, and it’s something entirely different from a Catalyst switch.
The difference is that this switch is made for MAN and WAN networks: it can run MPLS, supports VRF and has support for Ethernet Virtual Circuits (EVC).
EVC is a technology where you no longer use the 802.1q header of a frame only to specify to which network a frame belongs, but you use it for a number of different things, such as identifying a customer for example. It even allows multiple tags on a frame. Configuration of EVC is done on an interface (physical or port-channel), by making it a trunk, and the command ‘switchport trunk allowed vlan none’. Why this command? Because it makes the interface step out of its typical trunk mode and into EVC, after which service instances can be defined that match on 802.1q headers. An example configuration, where G0/1 and G0/2 are customer links and G0/3 is a shared WAN link:
Switch(config)#interface GigabitEthernet0/1
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 1 ethernet
Switch(config-if-srv)#encapsulation dot1q 100-200
Switch(config-if-srv)#bridge-domain 5010
Switch(config-if-srv)#exit
Switch(config-if)#exit
Switch(config)#interface GigabitEthernet0/2
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 1 ethernet
Switch(config-if-srv)#encapsulation default
Switch(config-if-srv)#bridge-domain 5020
Switch(config-if-srv)#exit
Switch(config-if)#exit
This seems complicated at first sight, but let’s look at the configuration lines one by one:
- As said before, ‘switchport trunk allowed vlan none’ activates EVC.
- ‘service instance number ethernet’ defines a policy, matching on 802.1q headers.
- The ‘encapsulation dot1q 100-200’ makes that this service instance will be applied to all incoming frames with an 802.1q header between 100 and 200.
- ‘bridge-domain 5010’ places these frames in a bridge domain. A bridge domain is like a VLAN, but it doesn’t use tags. All frames of the service instance are put in bridge domain 5010. All frames of the service instance on the next port are put in bridge-domain 5020. This way, even if there are overlapping VLANs, the differing bridge-domains keep them separated inside the switch.
- ‘encapsulation default’ means anything not defined before. If it is the only service instance on the link, as in this case, it means every incoming frame.
So the configuration of the customer links makes sure the frames stay separated (in bridge-domains) even if there is a VLAN overlap. The shared WAN uplink is now as following:
Switch(config)#interface GigabitEthernet0/3
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 1 ethernet
Switch(config-if-srv)#encapsulation dot1q 10
Switch(config-if-srv)#rewrite ingress tag pop 1 symmetric
Switch(config-if-srv)#bridge-domain 5010
Switch(config-if-srv)#exit
Switch(config-if)#service instance 2 ethernet
Switch(config-if-srv)#encapsulation dot1q 20
Switch(config-if-srv)#rewrite ingress tag pop 1 symmetric
Switch(config-if-srv)#bridge-domain 5020
Switch(config-if-srv)#exit
Switch(config-if)#exit
Note there are two service instances, one per customer in this case:
- Both instances refer to different bridge-domains, keeping the separation per customer.
- Both match one 802.1q tag on incoming frames: 10 for bridge-domain 5010, customer G0/1, 20 for bridge-domain 5020, customer G0/2
- ‘rewrite ingress tag pop 1’ means incoming frames will have their outer 802.1q tag removed.
- Finally, the ‘symmetric’ keyword means that outgoing frames will have an 802.1q tag added: the one defined in the service instance.
This means the following: a frame entering from G0/1 will have a second 802.1q applied, VLAN 10, if it goes over the WAN link. Same for G0/2 towards the WAN link, only with VLAN 20. This means that if both customers send a frame with VLAN 100, the G0/1 frame will be sent out of G0/3 with inner 802.1q header 100 and outer header 10. The G0/2 frame will have inner header 100 also, but outer header 20. Separation between both customers is ensured, even with overlapping VLANs! A remote Metro switch on the other end of the link can use similar service instances again to remove the outer header and place each frame in its own bridge-domain.
This is just one of the things possible with EVC. You can add 802.1q headers, remove them, but also translate them, by matching different headers on different interfaces in the same bridge-domain:
Switch(config)#interface GigabitEthernet0/4
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 1 ethernet
Switch(config-if-srv)#encapsulation dot1q 30
Switch(config-if-srv)#rewrite ingress tag pop 1 symmetric
Switch(config-if-srv)#bridge-domain 5030
Switch(config-if-srv)#exit
Switch(config-if)#exit
Switch(config)#interface GigabitEthernet0/5
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 1 ethernet
Switch(config-if-srv)#encapsulation dot1q 60
Switch(config-if-srv)#rewrite ingress tag pop 1 symmetric
Switch(config-if-srv)#bridge-domain 5030
Switch(config-if-srv)#exit
Switch(config-if)#exit
This configuration translates frames incoming on G0/4, VLAN 30, to frames tagged as VLAN 60 on G0/5. Again, the ‘symmetric’ keyword makes it work in both directions.
The switch does all of this in hardware forwarding, so gigabit bandwidth while forwarding and changing headers is possible. When headers are added, the MTU of the link may need to be increased though. And what if you want a layer 3 VLAN interface on the switch to manage it remotely? Can you enter through a link using EVC? Yes, you can: so far, I’ve used bridge domains above 4094, but if you define a bridge-domain below 4094, the VLAN of the same number will automatically be created on the switch, and its layer 3 interface is automatically part of that bridge domain. You do need to remove the 802.1q header(s) for incoming frames, as the frames need to reach the layer 3 interface ‘untagged’ internally. Assume you come in over the WAN link, outer header VLAN 30, inner header 10:
Switch(config)#interface GigabitEthernet0/3
Switch(config-if)#switchport trunk allowed vlan none
Switch(config-if)#switchport mode trunk
Switch(config-if)#service instance 3 ethernet
Switch(config-if-srv)#encapsulation dot1q 30 second-dot1q 10
Switch(config-if-srv)#rewrite ingress tag pop 2 symmetric
Switch(config-if-srv)#bridge-domain 50
Switch(config-if-srv)#exit
Switch(config-if)#exit
Switch(config)#interface Vlan50
Switch(config-if)#description Management
Switch(config-if)#ip address address subnetmask
Switch(config-if)#exit
As you can see, both 802.1q headers will be stripped off and the frame is placed in bridge-domain 50. This way it reaches VLAN interface 50. Other customers can’t reach this VLAN interface because they’re not part of the same bridge-domain.
Cisco support forums has a nice document explaining the concept, also together with MPLS. Be sure to check it out if you want to know more!
Reggle 