EVPN

EVPN-VXLAN Explainer 3 - BGP UPDATE & Route Type 2

joe neville

May 18, 2022 • 8 min read

The story so far

In post 1 we looked at VXLAN, an encapsulation type that enables L2 traffic to traverse L3 networks, the use case being virtual overlay networks to provide connectivity to VMs and containers.
VXLAN alone uses data plane learning to build a view of the MAC addresses of the clients attached to the network. This can be suboptimal because traffic needs to hit a switch for it to learn a MAC, it is akin to basic L2 switching across the network.
EVPN is used as a control plane for VXLAN networks, allowing peers to learn about MACs by advertising them in an extended form of BGP UPDATE.
In post 2 we looked at the BGP EVPN session and configuration, now let's move on to advertising some routes.

Example Network - Two peers

The philosophy behind this series is start small and build. With that in mind, I'm going to start with just two EVPN peers, supporting a couple of customer VLANs.

EVPN Operations

In principle, L2VNI is very similar to the static VXLAN we saw in post 1 of this series. It consists of a Layer 2 segment that can be stretched across a Layer 3 network as an overlay. VXLAN is, of course, still the encapsulation type, but now, rather than learning remote MAC addresses from the data plane, as the traffic hits our devices, we use EVPN to share the MACs of local clients.

EVPN Route Advertisement

At its heart, EVPN is essentially BGP, but whereas BGP originally shared IPv4 prefixes in UPDATE messages, EVPN needs to share a different type of information, that being MAC-addresses. So how does it do this? With a number of special types of BGP UPDATE, these are the EVPN Route Types, and the first type we need to familiarize ourselves with is Route Type 2.*
*To be exact, these are a new BGP Network Layer Reachability Information (NLRI) called the EVPN NLRI. In the NLRI, the first octet is the Route Type field, which indicates the type of the EVPN NLRI, hence why these are commonly referred to as Route Types. See RFC 7432, Section 7 for more details.

EVPN Route Type 2

EVPN Route Type 2, or RT-2, is at the heart of our EVPN operations, it is the route that shares information about MAC addresses, but can also, optionally, share IP addresses. In RFC7432, it is referred to as 'MAC/IP Advertisement Route'. In our example, these updates only contain MAC addresses, not IP addresses.

EVPN Route Learning

EVPN learns of MAC addresses in two ways, locally and remotely.

EVPN Local Learning

With local learning, the EVPN speaker learns the MAC addresses of the locally connected clients via the data plane, just like VXLAN.
However, EVPN does not listen for MAC addresses by default, the EVPN device must be configured with the details of the VLAN, or VLANs, that are a part of the EVPN network. EVPN now 'listens' on these VLANs for MAC addresses.
Clients boot up and send frames, such as gratitious ARP, or various other basic operational traffic from their OS. This traffic hits the EVPN speaker, which records the source MAC in its Layer 2 table, just like normal switching operation.
If a MAC address is learnt on a VLAN configured for EVPN, the device will also create an entry in its EVPN table for that MAC address, in the form of a EVPN Route Type 2, detailing the MAC address of the local client.
Moreover, once installed in its local EVPN table, the device will advertise this route to its EVPN peers, which then use remote learning to populate their own EVPN tables.

EVPN Remote Learning

While local learning of MAC addresses employs data plane learning, remote learning uses EVPN in the control plane.
By receiving information about the available MAC addresses of clients attached to remote peers, EVPN speakers can build up their view of the network i.e. their EVPN tables, without traffic having to hit them.
Then, if traffic for a known destination MAC address does arrive at the EVPN device, the traffic will be forwarded to the address of the advertising EVPN peer, the next-hop address, rather than having to rely upon flood and learn.
The traffic is known unicast, rather than unknown.

EVPN Configuration

Before we go deep into the operations of the EVPN BGP UPDATE, let's have a look at the configuration, to try to fit the pieces of the puzzle together. EVPN configuration by itself doesn't do much, it needs BGP and VXLAN to build a working solution.

Config Notes

The EVPN specific configuration is only one piece of the required config, it works in conjuection with BGP and VXLAN, as shown.
The EVPN sub-commands activate EVPN local-learning of MAC addresses on the specified VLANs, along with the route distinguisher and route targets.

Route Distinguisher

Each VLAN active for EVPN also needs a route distinguisher and route targets for import and export of route advertisements.
The route distinguisher is carried in each EVPN route advertisement and ensures uniqueness of the update.
The RD can be manally configured or auto-derived.
If auto-derived, the RD is the VXLAN interface IP address of the device and the VLAN ID. For example, the RD for 6300-1's VLAN 10 in the example network is 192.168.0.1:10. For VLAN 20 it is 192.168.0.1:20.

Route Target

The route target is a BGP extended community, carried in each EVPN route advertisement. This value allows EVPN speakers to control the import of EVPN routes into their EVPN tables.
The route target can be manually configured or auto-derived.
If configured as 'auto', the RT is derived from the AS Number of the EVPN peer and the VNI associated with the VLAN. The conversion method from decimal to RT value for the VNI is based upon the conversion used to go from asplain to asdot notation used for AS numbers. It looks a little strange because an AS of 65001 and VNI of 1010 auto converts to an RT of 65001:268436466. If you take 268436466 and perform the conversion it results in 4096.1010, the lower part being the VNI.
You can use the convertor here to try for yourself. Or read up on the details of the conversion process here.

Here's an example of auto-derived RTs, shown in the show evpn evi command.

    L2VNI : 1010
        Route Distinguisher        : 192.168.0.1:10
        VLAN                       : 10
        Status                     : up
        RT Import                  : 65001:268436466
        RT Export                  : 65001:268436466
        Local MACs                 : 0
        Remote MACs                : 0
        Peer VTEPs                 : 1

    L2VNI : 1020
        Route Distinguisher        : 192.168.0.1:20
        VLAN                       : 20
        Status                     : up
        RT Import                  : 65001:268436476
        RT Export                  : 65001:268436476
        Local MACs                 : 0
        Remote MACs                : 0
        Peer VTEPs                 : 1
        ```

BGP UPDATE process for Route Type 2

Now let's have a closer look at the process EVPN uses to share information about the network, and subsequent packet forwarding.
For this I'm using the example network in Figure 1 and the config in Figure 2, both above.

At the start of the process, both EVPN peers have been configured and the BGP EVPN session is 'Established'.

6300-1 & 6300-2 are both configured with EVPN active on VLAN 10 & 20. At present, neither devices have heard from locally connected clients on those VLANs.
A client, ub1, boots up on 6300-1's VLAN 10. I'm using ubuntu Linux 22.04, and there's plenty of chatter from this OS, E.G. ICMPv6 and v4 GARP, to ensure 6300-1 'hears' from the client. The client MAC is 50:65:f3:2f:c9:a1.
6300-1 records 50:65:f3:2f:c9:a1 in its MAC address table. This is visible with the command show mac-address-table.

6300-1 records ub1's MAC address in its EVPN table, visible with the command show evpn mac-ip. Note that this table lists the EVI. This is the EVPN Instance, which is the same number as the VXLAN VNI. In this case it is 1010.

6300-1 will also install ub1's MAC address in its BGP EVPN table as a MAC only Route Type 2.
We can see this in the BGP EVPN table and with a route specific show command. The format for this is a little tricky, it is route distinguisher then a hypen then the full route, as follows:
sh bgp l2vpn evpn 192.168.0.1:10-[2]:[0]:[0]:[50:65:f3:2f:c9:a1]:[]
At the foot of the full show command, the BGP extended community that serves as the Route Target is visible: 65001:268436466. This is added to the BGP UPDATE when advertised to EVPN peers, which use it to check against their EVPN import statement and determine whether to accept the route into their EVPN table.

6300-1 will now advertise ub1's MAC address to its EVPN peers as a BGP UPDATE Route Type 2.
The BGP UPDATE contains a large amount of information:
- BGP Attributes such as AS_PATH & ORIGIN.
- BGP Extended Communities for the exported Rout Target and the data plane encapsulation type (VXLAN).
- The route Next-Hop IP address.
- The Route Type.
- The Route Distinguisher.
- The MAC address, which is tucked away at the bottom.
- Finally, the VNI.

6300-2 receives the BGP UPDATE from 6300-1 and checks the UPDATE to ensure it is a valid BGP message, then installs the route into its BGP table.

6300-2 uses the Route Target value to determine whether to import the route to its EVPN table. If the RT matches the EVPN import statement, the route is installed in the EVPN table.

Finally, the MAC address is installed in 6300-2's MAC-address table. Note the route has a 'Type' of 'evpn'.

At this stage, the update of the control planes for 6300-1 and 6300-2 are now complete. 6300-1 has learnt of ub1 via local learning, and 6300-2 has learnt via remote learning using EVPN.
If 6300-2 receives traffic destined for 50:65:f3:2f:c9:a1 it will be treated as known unicast and sent directly to 192.168.0.1 rather than broadcast as unknown unicast.

Closing words

With the BGP UPDATE complete, and with an eye on the length, I'll conclude this post.
Next up, I'll look at Route Type 3's and use a larger example network.
Thanks for reading.

🐦 @joeneville_