Sign in Subscribe
EVPN

EVPN-VXLAN Explainer 1 - Static VXLAN

joe neville

• 8 min read
EVPN-VXLAN Explainer 1 - Static VXLAN

In this series of blog posts I'm going to break down how to configure Aruba AOS-CX switches for VXLAN and EVPN, plus explain how to read the EVPN table and various 'show' commands.
In this first post I will look at VXLAN, its configuration and operation.

A note about EVPN-VXLAN

VXLAN: encapsulation type of 8 bytes, works at the data plane, concerned with the forwarding of packets.
Ethernet VPN (EVPN): Extension to BGP, works that the control plane, concerned with learning and advertising MAC/IP addresses.

Static VXLAN - Start Here

VXLAN configuration is one of the basic building blocks of a EVPN-VXLAN network, it is worth familiarising yourself with static VXLAN configuration, even if you are never going to use it.

Example Network

Figure 1 below shows a network comprised of three Aruba 6300 switches acting as VTEPs, with two customer VLANs, VLAN10 and VLAN 20, that are bound with VNI 1010 and VNI 1020 respectively across the VXLAN network.

VXLAN Configuration

Component parts of the configuration:

  1. Customer-side VLAN - carries the traffic to be encapsulated.
  2. An IP network between VTEPs (the underlay network)- there must be bi-directional connectivity between an IP address designated on the VTEP. This is the source/destination address for the VXLAN traffic encapsulation and decapsulation. This can be any interface that has said connectivity, but a loopback is often used to provide a more resilient target address.
  3. An IGP on the underlay network - If the VTEP source/destination address is not directly connected, a means of enabling connectivity between VTEPs is required. This can be static routes, but is often an IGP, such as OSPF.

VTEP Underlay Configuration

Here's the relevant configuration from VTEP 6300-1 for the underlay:

Config notes

  • Interface 1/1/10 connects to a transit L2 switch (not shown) that forms part of the underlay network.
  • VLAN 2010 and 2011 are the underlay VLANs, their L3 VLAN interfaces are configured as point-to-point links, with OSPF adjacencies formed via these links.
  • Interface 1/1/20 is the customer-facing interface. In my home lab it connects to a linux server, which is configured with multiple VLAN interfaces, hence why this port is a trunk.

VTEP VXLAN interface configuration

Each VTEP is configured with a single VXLAN interface. This is the source and destination of VXLAN encapsulated traffic for the VTEP, it is effectively the overlay interface.
In addition, the configuration linking VLAN to VNI, and the peers in that VNI, are configured on this interface.

Config notes

  • Only one VXLAN interface can be configured per Aruba 6300 switch.
  • The VXLAN interface holds the binding between the VNIs and VLANs.
  • VTEP peers are configured underneath each VNI, this effectively configures the peers as part of a VPN.
  • Peers must be configured under each VNI that they participate in.
  • The VXLAN interface is shut by default, alter this with the 'no shutdown' command.

VXLAN and Data Plane Learning

VXLAN is a Layer 2 technology that forwards data based upon MAC address.

Layer 2 switching - Flood and Learn

To explain the forwarding process of VXLAN, here is an overview of the standard switching process upon which it is based:

  • On initial boot, a switch has no knowledge of the MAC addresses of clients on a network.
  • A unicast frame received from a local client that has not been heard from before is treated as an 'unknown unicast' frame, these are forwarded out of all ports configured with the receiving VLAN.
  • For example, if Client A sends traffic bound for Client B, the source MAC address of the sender, Client A, is recorded in the L2 switching table against the receiving port.
  • If the frame reaches its destination, and Client B replies, the switch already has the MAC address of Client A against the relevant port, and forwards Client B's frame out of that port only.
  • The switch now has Client B's MAC address recorded against the port it received that client's traffic on, so any subsequent traffic from Client A to Client B is forwarded on that port.
  • As clients on the network send frames, the switch builds its L2 table; broadcasting new unknown unicast and sending known unicast out of the relevant port.
  • This process is known as 'flood and learn'.

Note: To view the Layer 2 table of an Aruba switch, enter the CLI command show mac-address-table

VXLAN forwarding

VXLAN is similar to standard L2 forwarding, I'll use my previous configuration as an example:

interface vxlan 1
    source ip 192.168.0.1
    no shutdown
    vni 1010
        vlan 10
        vtep-peer 192.168.0.2
  • As with standard switching, a VTEP on boot will have an empty L2 table. VXLAN, as an L2 technology, uses the same L2 table. Use show mac-address-table to view it.
  • When the VTEP receives traffic from local clients on VLANs that are not configured as part of VXLAN, standard L2 switching 'flood and learn' applies.
  • When an unknown unicast frame is received on a VLAN bound to a VNI; in this example, VLAN 10, the source MAC is recorded in the L2 table and the frame is forwarded out of all local ports configured in VLAN 10, plus it is encapsulated in VXLAN, with the VNI 1010 and unicast to the VTEP peer 192.168.0.2.
  • Note that, just as with L2 switching, the VTEP, at this stage, has no knowledge of whether the destination MAC sits behind the destination VTEP, the sender is merely flooding to the peers configured against that VNI-VLAN binding.
  • Upon receiving the VXLAN encapsulated frame, the VTEP 192.168.0.2 decapsulates the frame, records the source MAC and sending VTEP in its L2 table, checks its own VNI to VLAN binding and forwards the frame on all port configured with that VLAN.
  • One important aspect to bear in mind at this stage is that the 802.1Q tag is not encapsulated in the VXLAN packet, hence the packet itself provides no indication of which VLAN to forward the packet on. The VLAN that the packet is forwarded on depends upon the configuration of the receiving VTEP. It is especially important to ensure VTEPs VNI to VLAN configurations are correct as per the user requirements.
  • To finish this exchange, if the destination client does indeed sit local to 192.168.0.2, and replies, 192.168.0.2 will consult its L2 table, encapsulate and forward to 192.168.0.1, which will forward directly to the source client.

Here's an example L2 table from 6300-1, the 192.168.0.1 VTEP:

6300-1# show mac-address-table
MAC age-time            : 300 seconds
Number of MAC addresses : 7

MAC Address          VLAN     Type                    Port
------------------------------------------------------------
50:65:f3:2f:c9:a1    10       dynamic                 1/1/20
48:0f:cf:b9:59:a6    10       dynamic                 vxlan1(192.168.0.2)
50:65:f3:2f:c9:a1    20       dynamic                 1/1/20
48:0f:cf:b9:59:a6    20       dynamic                 vxlan1(192.168.0.2)
2c:59:e5:ba:90:01    20       dynamic                 vxlan1(192.168.0.3)
b8:d4:e7:05:ef:00    2010     dynamic                 1/1/10
b8:d4:e7:05:39:40    2011     dynamic                 1/1/10

In this table we can see that locally learnt, underlay, MAC addresses, such as b8:d4:e7:05:ef:00 in VLAN 2010 on port 1/1/0, share the same table as overlay VXLAN-learnt MACs, such as 48:0f:cf:b9:59:a6 via VTEP 192.168.0.2.

VXLAN forwarding packet walk-through

Now let's look at the other VNI configured on 6300-1 and step through an actual packet walk-through on my home lab.

Configuration snippet:

interface vxlan 1
    source ip 192.168.0.1
    no shutdown
    vni 1020
        vlan 20
        vtep-peer 192.168.0.2
        vtep-peer 192.168.0.3
  • Client 172.16.20.1 attached to 6300-1 wishes to send a packet to the client behind 6300-3, 172.16.20.3.
  • The client machine has no ARP entry for the destination, and thus transmits an ARP request with the broadcast destination MAC ff:ff:ff:ff:ff:ff on VLAN 20.
  • 6300-1 receives the ARP request on its trunk port configured for VLAN 20, this VLAN is bound to VXLAN VNI 1020.
  • 6300-1 broadcasts the ARP request on any local port configured with VLAN 20, plus encapsulates the packet and forwards it to all VTEP peers configured under that VNI, namely 192.168.0.2 & 192.168.0.3. This is known as ingress replication.
  • Both destination VTEPs are configured to bind VNI 1020 with VLAN 20, thus they both decapsulate the VXLAN packet and forward the broadcast ARP request on all ports configured with VLAN 20.
  • On 192.168.0.2's local LAN, the ARP packet is discarded.
  • But the destination client 172.16.20.3 resides on 192.168.0.3's LAN, and the client responds with an ARP reply, with the originating client's IPv4 address, 172.16.20.1, and MAC, 50:65:f3:2f:c9:a1, as the destination.
  • 6300-3 receives this ARP reply and consults its L2 table, seeing that 50:65:f3:2f:c9:a1 resides behind 192.168.0.1. 6300-1 encapsulates the ARP reply in a VXLAN packet, with an outer destination IPv4 address of 192.168.0.1 and VNI of 1020.
  • 6300-1 receives the VXLAN packet from 192.168.0.3, decapsulates and forwards directly to 50:65:f3:2f:c9:a1 after consulting its L2 table for the direct egress port.
  • Client 172.16.20.1 updates its ARP cache with 172.16.20.3's MAC address and now forwards its data packet to 2c:59:e5:ba:90:01.
  • 6300-1 receives the data packet, consults its L2 table and knows that 2c:59:e5:ba:90:01 sits behind 192.168.0.3. Rather than sending to both VTEP peers, 6300-1 encapsulates the data packet to be sent via the VXLAN interface, with an outer IPv4 address of 192.168.0.3.
  • Finally, 6300-3 will decapsulate the VXLAN packet and forwards directly to 2c:59:e5:ba:90:01.

Limitations of static VXLAN

While static VXLAN may suit some, especially smaller, networks; it is possible that when scaling up the size of a network, these operations prove to be less than optimal.
The static nature of the VXLAN configuration, especially the need to configure every VTEP under every VNI it partipates in can result in cumbersome config files.
In addition, the reliance on flood and learn to build the L2 table can result in inefficient traffic forwarding, as the network devices must learn a view of the network reactively as traffic flows in the data plane.
For such reasons, it may be prudent to employ a control plane protocol to advertise MAC addresses, that is where EVPN comes in, which is the subject of my next blog.

🐦 @joeneville_