VXLAN Overview

Ambarish Kumar
Cisco Advanced Services
What Is A VXLAN?
• A VLAN with an X in the middle J
• A VXLAN provides the same service to End Systems as a VLAN
• The X stands for eXtensible
– Scale!
– More layer 2 segments than VLANs
– Wider stretch than VLANs
• VXLANs are an Overlay Network technology
– MAC Over UDP/IP

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Overlay Networks

DEL
BLR
Air Traffic Control System

Ethernet Frames

V V
M M
1 4
IP Addr IP Addr
1.1.1.1 2.2.2.2
V Virtual IP Network Virtual V
M M
2 Switch Switch 5

V V
M UDP/IP Packets M
3 6
Hypervisor Hypervisor

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Why VXLANs?
Pain Points in Scaling Cloud Networking
• Use of server virtualization and cloud computing is stressing the network
infrastructure in several ways:
– Server Virtualization increases demands on switch MAC address tables
– Multi-tenancy and vApps driving the need for more than 4K VLANs
– Static VLAN trunk provisioning doesn’t work well for Cloud Computing and VM mobility
– Limited reach of VLANs using STP constrains use of compute resources

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
VxLAN Tunnel Encapsulation (MAC-in-UDP)

Outer Outer VXLAN

UDP Header Original L2 Frame
Mac Header IP Header Header FCS FCS

8 Bytes 8 Bytes
10 or 14 Bytes
20 Bytes

RRRR1RRR
UDP Length
Ether Type
MAC Addr.

VLAN Type
MAC Addr.

Reserved
Checksum

Reserved
IP Header

Checksum
Misc Data

Src. Port
VLAN ID

0x0800

Dst Port

VXLAN
Protocol

0x0000
0x8100

Dst. IP
Header

VNID
Outer
Src. IP

UDP
UDP
Src.

Outer
Dst.

0x11
Tag

48 48 16 16 16 16 16 16 16 8 24 24 8
72 8 16 32 32

Allows for possible

For next-hop transport in the The well known VXLAN port
Source and Destination VTEP 4789. Indicates a VXLAN packet︎ 16M segments︎
underlay network︎
addresses, allowing transport
Hash of the internal L2/L3/L4 header of the
across the underlay IP network︎ original frame. Can be used as entropy for
better ECMP/LACP load sharing ︎

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
VXLAN VTEP
VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point).
Each VTEP has two interfaces, one is to provide bridging function for local hosts, the
other has an IP identification in the core network for VXLAN encapsulation/
decapsulation.

Underlay Network
(IP Routing)

VTEP VTEP
IP Interface IP Interface

Local LAN Segment Local LAN Segment

End System End System End System End System

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
VTEP Use Of IGMP
IGMP Used to Join Each VXLANs Assigned Multicast Group on Demand

Web DB DB Web
VM VM VM VM

Join Multicast Join Multicast

Group 239.1.1.1 L3 Core Group 239.2.2.2
Transport Join Multicast
Join Multicast
Group 239.2.2.2
Group 239.1.1.1

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
 VXLAN Packet Forwarding Flow
Outer S-MAC: MAC-3
Outer S-MAC: MAC-1 Outer D-MAC: MAC-4
Outer D-MAC: MAC-2
3 Outer S-IP: IP-1
Outer S-IP: IP-1 Outer D-IP: IP-4
Outer D-IP: IP-4
Routed Based on
UDP
UDP Outer IP header
VXLAN VNID: 10
VXLAN VNID: 10 IP Network S-MAC: MAC-A
S-MAC: MAC-A Router-1 Router-2
D-MAC: MAC-B
D-MAC: MAC-B S-IP: IP-A
S-IP: IP-A MAC-2 MAC-3 D-IP: IP-B
D-IP: IP-B 2 IP-2 IP-3 4
MAC-4
IP-4
MAC-1
S-MAC: MAC-A
VTEP-1 VTEP IP-1 S-MAC: MAC-A
D-MAC: MAC-B VTEP-2
D-MAC: MAC-B
S-IP: IP-A
D-IP: IP-B 1 S-IP: IP-A 5
D-IP: IP-B
Host-B
Host-A

MAC-A MAC-B
VXLAN VNID 10 (Tenant Blue)
IP-A IP-B

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Multicast-base VXLAN
Peer Discovery & Address Learning
End System End System

S-MAC: MAC-1 3 ARP Request for IP B

D-MAC:
00:01:5E:01:01:01 Src MAC: MAC-A
Dst MAC: FF:FF:FF:FF:FF:FF

IP-3
VTEP 3

VTEP-
Outer S-IP: IP-1
Outer D-IP: 239.1.1.1

3
MAC VXLAN ID Remote VTEP

UDP
Address

MAC-A 10 IP-1
4
VXLAN VNID: 10 ARP Response from IP B
Src MAC: MAC-B
ARP Request for IP B Dst MAC: MAC-A
Src MAC: MAC-A
7 Dst MAC:
2 VTEP 2
FF:FF:FF:FF:FF:FF
End System B
ARP Response from IP B Multicast Group IP-2
MAC-B
MAC-2
Src MAC: MAC-B 239.1.1.1 IP-B
Dst MAC: MAC-A VTEP-1 2 VTEP-2 3
S-MAC: MAC-2
D-MAC: MAC-1 ARP Request for IP B
VTEP 1 5 Src MAC: MAC-A
End System A
MAC-A
1 IP-1 6 Outer S-IP: IP-2 Dst MAC: FF:FF:FF:FF:FF:FF
MAC-1 Outer D-IP: IP-1
IP-A MAC VXLAN ID Remote VTEP
ARP Request for IP B Address
Src MAC: MAC-A UDP MAC VXLAN ID Remote VTEP
MAC-B 10 IP-2 Address
Dst MAC: FF:FF:FF:FF:FF:FF VXLAN VNID: 10 MAC-A 10 IP-1
ARP Response
from IP B
Src MAC: MAC-B
Dst MAC: MAC-A
BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
 Multiple VXLANs Can Share One Multicast Group
Blue & Red VXLANs Share the 239.1.1.1 Multicast Group

Web App DB App

VM VM VM VM

• Encapsulate with Blue VXLAN ID VTEP Discards Since No VM with

• Multicast to Servers Registered for Blue VXLAN ID
239.1.1.1 Multicast Group

VM Broadcast Frames Sent to More Servers

But Broadcast Domain Respected Within VXLAN Segment
BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Configuration
Enable the Feature

feature nv overlay :: Enables the VXLAN feature.

feature vn-segment-vlan-based :: Configures the global mode for all

VXLAN bridge domains.

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
VXLAN Configuration – Overlay Configuration
Point to Multi-point tunnel
with VxLAN encapsulation

interface nve1
no shutdown
source-interface loopback1
member vni 6000 mcast-group 235.1.1.1

Used to Derive
Local VTEP IP VxLAN Identifier IP Multicast Group for Multi-destination Traffic
address

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
 VXLAN Configuration – Mapping VLANs to VNIs
Layer 2 Gateway Map VNI to VLAN/BD

VLAN CLI Model

vlan 3002
vn-segment 6000

interface nve1
no shutdown
source-interface loopback1
member vni 6000 mcast-group 235.1.1.1

VXLAN tunnel interface

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Verifying the VXLAN Config

Show nve interface

Show nve peers
Show nve vi
Show nve vxlan-params

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
VXLAN Benefits
• On demand network segments without physical network reconfiguration
• Massive scale of layer 2 segments for multi-tenant environments
• Allows virtual layer 2 segments to stretch across physical layer 2 network
boundaries
– Provides operational flexibility for deploying VMs anywhere in the data center
• VXLANs work over existing deployed data center switches and routers
– Alleviates network scaling issues associated with server virtualization

BRKVIR-2014 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 17