Abstract image of a woman sitting on books and studying on a laptop

VXLAN Integration with CloudStack Advanced Zone

Download as PPTX, PDF
Yoshikazu Nojima, profile picture
Yoshikazu Nojima

VXLAN Integration with CloudStack was presented at the Advanced Zone CCCEU13 conference in Amsterdam on November 21, 2013. The presentation discussed integrating VXLAN to overcome the VLAN ID limitation in CloudStack and allow for more scalable network isolation. VXLAN was demonstrated working with CloudStack to provide isolated networks and inter-tier connectivity within VPCs while maintaining network isolation. Basic functions like VM connectivity, migration, and network availability were tested under VXLAN and found to work as expected. Feedback was welcomed on the VXLAN integration in CloudStack.

Technology
1 of 56
Downloaded 179 times
1
2
3
4
5
Most read
6
Most read
7
8
Most read
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
VXLAN Integration with CloudStack Advanced Zone CCCEU13 - Amsterdam Nov. 21, 2013 Shinya Adachi s.adachi@ntt.com Yoshikazu Nojima y.nojima@verio.net 1
Why we come to here? • Open source community growth is important -Because we want to be free from specific vendor products. • Contribute technology for cloud scalability to accelerate migration from customer onpremise to be on the cloud -By suggest one of possible solution to have massive scalability. 2
Cloudn • CloudStack based public cloud services(Compute) • Currently available in Japan and the US • 2 interface for customer, Customer portal GUI and APIs (APIs over 150 including AWS compatible) • VPC type “Coming Soon” in Japan 3 3
Problem: VLAN ID limitation • Advanced Zone o More functionality • • o NAT, FW, LB, VPN VPC Isolation required • • For each guest network For each VPC tier Virtual Router • Isolation Method: VLAN o Virtual Router Isolated VLAN IDs are limited • • o Advanced Zone Public Network Only 4096 Should be identical within a zone Guest Network VPC Tier VPC Tier # of Domains is limited by VLAN • Each domain requires at least one VLAN ID VM VM VM VM VM VM VPC 4
VXLAN Overview VXLAN [Virtual eXtensible Local Area Network] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelope type UDP packet (L4 packet) Standardization Status Under IETF standardization process Implementation Software-based : Cisco Nexus Series Switches, VMware vSphere Distributed Switch, Open vSwitch, and Linux kernel hardware based : Arista 7150, Brocade ADX series Characteristics • • • • 16M (2^24) isolated networks On top of UDP packet • Can utilize L4 port based ECMP load balancing solutions • Src UDP port is a hash of payload MAC addr Ethernet broadcast is mapped to IP multicast • L2: IGMP (or MLD) snooping, otherwise it floods a little • L3: If you want to communicate across L3 subnet Dynamic tunnel endpoint learning http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-06 5
How traffic flows with VXLAN? Underlay Network for VXLAN ethX ethX vxlanM vxlanM vxlanM brethY-M brethY-M brethY-M vnet vnet vnet VM1 VM2 VM3 Host1 Host2 ethX Host3 1. If Unicast and KVM host (Src) learned mapping between VM and KVM host (Dst)  VXLAN uses Unicast 2. If broadcast or Unicast but KVM host (Src) doesn’t know mapping  VXLAN uses Multicast 6
Host2 VTEP IP address resolution table ex. ping VNI Capsule Dst IP addr N Underlay Network for VXLAN Payload Dst MAC addr VM3 MAC addr Host3 IP addr Host3 VTEP IP address resolution table VNI Payload Dst MAC addr Capsule Dst IP addr N VM2 MAC addr Host2 IP addr VTEP : VXLAN Tunnel End Point ethX ethX vxlanN vxlanN vxlanN bridge bridge bridge vnet vnet vnet VM1 VM2 VM3 Host1 Host3 Host2 Payload # ethX VXLAN header Src MAC addr Dst MAC addr VNI Src IP addr Dst IP addr Src UDP port number 1 ARP request VM2 MAC addr Broadcast N Host2 IP addr Multicast IP addr Hash(VM2 MAC addr) 2 ARP reply VM3 MAC addr VM2 MAC addr N Host3 IP addr Host2 IP addr Hash(VM3 MAC addr) 3 ICMP Echo request VM2 MAC addr VM3 MAC addr N Host2 IP addr Host3 IP addr Hash(VM2 MAC addr) 4 ICMP Echo reply VM3 MAC addr VM2 MAC addr N Host3 IP addr Host2 IP addr Hash(VM3 MAC addr) 7
How VXLAN shrink broadcast domain size? Underlay Network for VXLAN Underlay network segment 1 Underlay network segment 2 Underlay network segment 3 ethY vxlanN vxlanM bridge bridge bridge vnet (no VM associated with VNI : N) vxlanN vnet vnet VM2 VM3 VM1 Host1 Host2 Host3 Host4 1. Host1 contains no VM belong to VXLAN segment N, Host 1 doesn’t join the multicast group N. 2. Since VM1 & VM2 belong to VXLAN segment N, Host 2 & 3 join same multicast group N. 3. Since Host 4 contains no VM belong to VXLAN segment N, the path to Host4 is excluded from multicast domain if the switch supports IGMP snooping 8
CloudStack Integration 9
CloudStack KVM VLAN – bridging Overview Internet Public Network ethX KVM ethX KVM cloudbrL cloudbrJ vnetG vnetE VR vnetF brethY-M brethY-M Underlay Network vnetG vnetH VR vnetG VM VM ethY Encapsulate / Decapsulate ethY Encapsulate / Decapsulate Guest Network vnetA VR vnetB brethY-N vnetC VM ethY.N ethY.N VLAN Tunnel (VLAN ID : N) brethY-N vnetD VM 10
CloudStack KVM VXLAN – bridging Overview Internet Public Network ethX KVM ethX KVM cloudbrL cloudbrJ vnetG vnetE VR vnetF brethY-M brethY-M Underlay Network vnetG vnetH VR vnetG VM VM ethY Encapsulate / Decapsulate ethY Encapsulate / Decapsulate Guest Network vnetA VR vnetB brvxN vnetC VM vxlan-N vxlan-N VXLAN Tunnel (VNI : N) brvxN vnetD VM 11
Demo video 12
13
Functional Tests 14
Functional test result overview We tested the basic functions directly affected by VXLAN support. (ex. VM start/stop, Internet connectivity, Inter-tier connectivity and live migration in Isolated Network and VPC tier ) Case # VR/VM location 1 VR&VM exist in a same hypervisor Network type isolated Test target function connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart 2 VR&VM exist in different hypervisors isolated connectivity to VR after VM restart connectivity to the internet after VM restart connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart connectivity to VR after VM restart connectivity to the internet after VM restart VM migration connectivity to VR after VM restart connectivity to the internet after VM restart 3 VM1&VM2 exist in different isolated network isolated inter isolated network isolation 4 VR&VM exist in different hypervisors VPC connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart connectivity to VR after VM restart connectivity to the internet after VM restart VM migration connectivity to VR after VM restart connectivity to the internet after VM restart 5 VM1&VM2 exist in different tier, and routing between two tier is allowed 6 VM1&VM2 exist in different tier, and routing between two tier is denied Step # Procedure Expected result Result 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 11 migrate VM to another hypervisor 12 ping to VR 13 ping to the host in the internet (ex. 8.8.8.8) ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully ping reaches to the destination ping reaches to the destination Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass 1 ping from VM1 in one tier to the internet(ex. 8.8.8.8) 2 tcpdump from VM2 in another tier 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 11 migrate VM to another hypervisor 12 ping to VR 13 ping to the host in the internet (ex. 8.8.8.8) ping reaches to the destination ping packet from VM1 cannot captured ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully ping reaches to the destination ping reaches to the destination Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass VPC inter-tier connectivity 1 ping from VM1 in one tier to VM2 in another tier ping reaches to the destination Pass VPC inter-tier isolation 1 ping from VM1 in one tier to the internet(ex. 8.8.8.8) 2 tcpdump from VM2 in another tier ping reaches to the destination ping packet from VM1 cannot captured Pass Pass 15
VXLAN plugin restriction • VXLAN is not available for Public Network, Storage Network, and Management Network • These networks do not consume many VLAN IDs. • KVM is the only supported hypervisor • Maybe we can add LXC support • Mapping between VNI and multicast address is hardcoded. multicastAddress= "239.$(( ($vni >> 16) % 256 )).$(( ($vni>> 8) % 256 )).$(( $vni % 256 ))" 16
Resources • CloudStack Plugin guide for VXLAN • http://jenkins.buildacloud.org/job/build-docs-vxlanmaster/lastSuccessfulBuild/artifact/Apache_CloudStack -4.3.0-CloudStack_VXLAN_Guide-en-US.pdf • Design Doc • https://cwiki.apache.org/confluence/display/CLOUDSTA CK/Linux+native+VXLAN+support+on+KVM+hypervisor • JIRA ticket • https://issues.apache.org/jira/browse/CLOUDSTACK2328 Bug report, suggestions, any feedbacks are welcome! 17
Wrap up • VXLAN integration for CloudStack we contributed is merged to CloudStack 4.3 branch. • We confirmed basic functions work in Isolated Network and VPC Tier. • Please evaluate VXLAN integration, any bug report, suggestions, feedbacks are welcome! Special Thanks: Toshiaki Hatano NTT Communications Corp. Junji Arakawa NTT Communications Corp. Chris Cameron Verio Inc. 18
Appendix 19
NVGRE Overview NVGRE [Network Virtualization using Generic Routing Encapsulation] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelop type Extended GRE packet (L3 packet) Standardization Status Under IETF standardization process Implementation Microsoft Hyper-V 2012 R2, Intel Ethernet Switch FM6000 Series Characteristics • • 16M (2^24) isolated networks Extended GRE packet • Utilize GRE packet’s key option field as VSID and flow-ID. • ECMP load balancing solutions must be aware of NVGRE flow-ID • Spec leaves Ethernet broadcast undefined. • Mapping to IP multicast is suggested. • Multicast network operation is required. http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03 20
STT Overview STT [Stateless Transport Tunnel] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelop type TCP like original L3 packet (protocol type is same as TCP. Pretends TCP packet.) Standardization Status Under IETF standardization process Implementation VMware NSX (formerly Nicira NVP) Characteristics • • 2^64 isolated networks TCP-like header + STT header • Can utilize NIC’s TSO feature • FW/router may drop STT packets by statefull inspection. • Spec leaves Ethernet broadcast undefined. • Mapping to IP multicast is suggested. • Multicast network operation is required. http://tools.ietf.org/html/draft-davie-stt-04 21
Solutions comparison VXLAN NVGRE STT Overhead Header Size ○ (50 bytes) ○ (42 bytes) △(76 bytes) NIC Offloading ○ (Special NIC is required) ○ (Special NIC is required) ◎ (able to utilize normal TSO) Existing Assets Fitness ◎ (MTU may need to be adjusted) ◎ (MTU may need to be adjusted) △ (FW/router may drop STT packets) Interoperability ○ (Spec left only minor undefined points) ×（Tunnel endpoint address resolution is undefined.） ×（Tunnel endpoint address resolution is undefined.) Ethernet Broadcast ◎ (Mapping to IP Multicast) △ (Mapping to IP Multicast (suggestion)) △ (Mapping to IP Multicast (suggestion)) ECMP ○ (Able to utilize L2 fabric's L4 port base balancing) △ (L2 fabric must aware NVGRE flow-ID to balance) ○ (Able to utilize L2 fabric’s L4 port base balancing) Multicast Operation Required Required (depends on implementation) Required (depends on implementation) Supporting Vendors VMware/Citrix/Red Hat/ Cisco/Intel/Broadcom/Arista Microsoft/Arista/Emulex/ Dell/HP VMware(formerly Nicira) Linux Integration ◎ (kernel 3.7 or later) × (no implementation exists) ○ (Nicira’s Open vSwitch is required) 22
VXLAN Terminology • • • • • VXLAN • Virtual eXtensible Local Area Network VXLAN Segment • VXLAN Layer 2 overlay network over which VMs communicate VTEP • VXLAN Tunnel End Point • an entity which originates and/or terminates VXLAN tunnels VNI • VXLAN Network Identifier (or VXLAN Segment ID) VXLAN Gateway • an entity which forwards traffic between VXLAN and non-VXLAN environments 23
VXLAN segment format 0 Outer Ethernet Header: - FCS is newly calculated, inner FCS is omitted. Outer IP header - If Inner dst MAC is unicast MAC and local VTEP knows remote VTEP for the MAC address, dst IP set to remote VTEP’s IP address. - If not, packet will be sent out to multicast group associated with VNI. - The VTEP will use (*,G) joins. Outer UDP header - Source port: It is recommended to be calculated from inner Ether Header, for ECMP purpose. - Destination port: 4789 - Checksum: SHOULD be 0. Or correct value VXLAN header - VNI has a 24-bit field From current draft: (IPv4 case) http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-06 2013-04-17: IANA assigned udp/4789 for VXLAN port http://www.iana.org/assignments/service-names-portnumbers/service-names-port-numbers.xml 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 Outer Ethernet Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination MAC Address | Outer Source MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Source MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |OptnlEthtype = C-Tag 802.1Q | Outer.VLAN Tag Information | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ethertype = 0x0800 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Outer IPv4 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live |Protocl=17(UDP)| Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Source IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Outer UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = xxxx | Dest Port = VXLAN Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ VXLAN Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R|R|R|R|I|R|R|R| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VXLAN Network Identifier (VNI) | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (continuing to Inner Ethernet header, abbrev.) 24
CloudStack Integration -NETWORK CONCEPTS IN CLOUDSTACK- 25
Network concepts in CloudStack • Guest Network • • • Virtual network VMs are connected Isolated each other There are two type of Guest networks • Isolated network – – Traffic from VMs go out to Public Network through Virtual Router. VR is created per Isolated network. • Shared network – Traffic from VM go out directly. • VPC • • In VPC, Virtual Router can have multiple Isolated Networks (It is called VPC tier). In VPC, routing between tier is configurable. 26
Network concepts in CloudStack (cont.) • Isolation method • • • Method to isolate Guest Networks each other. Typical isolation method is VLAN. VXLAN need to be implemented as isolation method. • Physical Network • • Underlay network of Guest network Isolation method of guest network is specified while defining physical network • createPhysicalNetwork API has isolationmethods parameter. – http://cloudstack.apache.org/docs/api/apidocs4.2/root_admin/createPhysicalNetwork.html 27
Advanced Zone Network Overview Internet Public Network KVM VR for VPC * VR for VPC will be created per VPC KVM VR VPC tier VM VM VM VM VM VM Guest Network VM • • User can create multiple VPCs (depending on settings, up to 20 by default) VPC can have multiple tiers (depending on settings, up to 3 by default) 28
Step by Step VXLAN base zone setup procedure 29
VXLAN base zone setup procedure(1) - login 30
VXLAN base zone setup procedure(2) - welcome wizard Click skip bottom 31
VXLAN base zone setup procedure(3) - infrastructure tab Click Infrastructure tab 32
VXLAN base zone setup procedure(4) 33
VXLAN base zone setup procedure(5) Select Advanced type 34
VXLAN base zone setup procedure(6) Fill zone wizard. KVM is the only supported hypervisor. 35
VXLAN base zone setup procedure(7) Management network/Public network/Storage network are not supported by VXLAN Select VLAN Select VXLAN Guest network is supported by VXLAN Set underlay network I/F name (ex. “eth0”) to traffic type 36
VXLAN base zone setup procedure(8) Fill zone wizard. There is no VXLAN specific concern. 37
VXLAN base zone setup procedure(9) Fill zone wizard. There is no VXLAN specific concern. 38
VXLAN base zone setup procedure(10) You can use 0-16777215 as VNI 39
VXLAN base zone setup procedure(11) Fill zone wizard. There is no VXLAN specific concern. 40
VXLAN base zone setup procedure(12) Fill zone wizard. There is no VXLAN specific concern. 41
VXLAN base zone setup procedure(13) Fill zone wizard. There is no VXLAN specific concern. 42
VXLAN base zone setup procedure(14) Fill zone wizard. There is no VXLAN specific concern. 43
VXLAN base zone setup procedure(15) Fill zone wizard. There is no VXLAN specific concern. 44
VXLAN base zone setup procedure(16) Click “Launch zone” button 45
VXLAN base zone setup procedure(17) 46
VXLAN base zone setup procedure(18) Click “Yes” button 47
VXLAN base zone setup procedure(19) Add Instance wizard. There is no VXLAN specific concern. 48
VXLAN base zone setup procedure(20) Add Instance wizard. There is no VXLAN specific concern. 49
VXLAN base zone setup procedure(21) Add Instance wizard. There is no VXLAN specific concern. 50
VXLAN base zone setup procedure(22) Add Instance wizard. There is no VXLAN specific concern. 51
VXLAN base zone setup procedure(23) Add Instance wizard. There is no VXLAN specific concern. 52
VXLAN base zone setup procedure(24) Add Instance wizard. There is no VXLAN specific concern. 53
VXLAN base zone setup procedure(25) Add Instance wizard. There is no VXLAN specific concern. 54
VXLAN base zone setup procedure(26) 55
VNI validation If your VNI range is invalid, This error will pop up. 56

More Related Content

PDF
Deploying CloudStack and Ceph with flexible VXLAN and BGP networking
ShapeBlue
 
PPTX
CloudStack networking
ShapeBlue
 
PDF
VXLAN BGP EVPN: Technology Building Blocks
APNIC
 
PPTX
OpenvSwitch Deep Dive
rajdeep
 
PDF
Volume Encryption In CloudStack
ShapeBlue
 
PDF
An Introduction to VMware NSX
Scott Lowe
 
PDF
FreeSWITCH on Docker
建澄 吳
 
PDF
The Best Storage Solution For CloudStack: LINSTOR
ShapeBlue
 
Deploying CloudStack and Ceph with flexible VXLAN and BGP networking
ShapeBlue
 
CloudStack networking
ShapeBlue
 
VXLAN BGP EVPN: Technology Building Blocks
APNIC
 
OpenvSwitch Deep Dive
rajdeep
 
Volume Encryption In CloudStack
ShapeBlue
 
An Introduction to VMware NSX
Scott Lowe
 
FreeSWITCH on Docker
建澄 吳
 
The Best Storage Solution For CloudStack: LINSTOR
ShapeBlue
 

What's hot (20)

PDF
CloudStack - Top 5 Technical Issues and Troubleshooting
ShapeBlue
 
PPTX
Pushing Packets - How do the ML2 Mechanism Drivers Stack Up
James Denton
 
PDF
Backroll: Production Grade KVM Backup Solution Integrated in CloudStack
ShapeBlue
 
PDF
VXLAN and FRRouting
Faisal Reza
 
PDF
EBPF and Linux Networking
PLUMgrid
 
PDF
Edge Zones In CloudStack
ShapeBlue
 
PDF
Openstack pour les nuls
Chris Cowley
 
PDF
Virtualisation
Tsubichi
 
PDF
Red Hat OpenStack 17 저자직강+스터디그룹_1주차
Nalee Jang
 
PDF
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
ShapeBlue
 
PPTX
MP BGP-EVPN 실전기술-1편(개념잡기)
JuHwan Lee
 
PDF
Monitoring in CloudStack
ShapeBlue
 
PPTX
Introduction to CNI (Container Network Interface)
HungWei Chiu
 
PPTX
NGINX: High Performance Load Balancing
NGINX, Inc.
 
PPTX
Using the KVMhypervisor in CloudStack
ShapeBlue
 
PDF
Open vSwitch 패킷 처리 구조
Seung-Hoon Baek
 
PDF
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
ShapeBlue
 
PDF
Alphorm.com Formation VirtualBox
Alphorm
 
PPTX
CloudStack Networking
CloudStack - Open Source Cloud Computing Project
 
PPTX
Overview of Distributed Virtual Router (DVR) in Openstack/Neutron
vivekkonnect
 
CloudStack - Top 5 Technical Issues and Troubleshooting
ShapeBlue
 
Pushing Packets - How do the ML2 Mechanism Drivers Stack Up
James Denton
 
Backroll: Production Grade KVM Backup Solution Integrated in CloudStack
ShapeBlue
 
VXLAN and FRRouting
Faisal Reza
 
EBPF and Linux Networking
PLUMgrid
 
Edge Zones In CloudStack
ShapeBlue
 
Openstack pour les nuls
Chris Cowley
 
Virtualisation
Tsubichi
 
Red Hat OpenStack 17 저자직강+스터디그룹_1주차
Nalee Jang
 
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
ShapeBlue
 
MP BGP-EVPN 실전기술-1편(개념잡기)
JuHwan Lee
 
Monitoring in CloudStack
ShapeBlue
 
Introduction to CNI (Container Network Interface)
HungWei Chiu
 
NGINX: High Performance Load Balancing
NGINX, Inc.
 
Using the KVMhypervisor in CloudStack
ShapeBlue
 
Open vSwitch 패킷 처리 구조
Seung-Hoon Baek
 
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
ShapeBlue
 
Alphorm.com Formation VirtualBox
Alphorm
 
CloudStack Networking
CloudStack - Open Source Cloud Computing Project
 
Overview of Distributed Virtual Router (DVR) in Openstack/Neutron
vivekkonnect
 
Ad

Viewers also liked (9)

ODP
Implementing CloudStack's VPC feature
Marcus L Sorensen
 
PDF
How VXLAN works on Linux
Etsuji Nakai
 
PDF
Cisco at v mworld 2015 vmworld_sf-2015-hyperconverged
ldangelo0772
 
PDF
Cloud OS「Apache CloudStack」をお手軽に使ってみる方法
samemoon
 
PPTX
Cloud stack networking shapeblue technical deep dive
ShapeBlue
 
PPTX
OSSのクラウド基盤 OpenStack / CloudStack
VirtualTech Japan Inc.
 
PPTX
Vxlan control plane and routing
Wilfredzeng
 
PPTX
Vxlan deep dive session rev0.5 final
KwonSun Bae
 
PPT
Vlan
sanss40
 
Implementing CloudStack's VPC feature
Marcus L Sorensen
 
How VXLAN works on Linux
Etsuji Nakai
 
Cisco at v mworld 2015 vmworld_sf-2015-hyperconverged
ldangelo0772
 
Cloud OS「Apache CloudStack」をお手軽に使ってみる方法
samemoon
 
Cloud stack networking shapeblue technical deep dive
ShapeBlue
 
OSSのクラウド基盤 OpenStack / CloudStack
VirtualTech Japan Inc.
 
Vxlan control plane and routing
Wilfredzeng
 
Vxlan deep dive session rev0.5 final
KwonSun Bae
 
Vlan
sanss40
 
Ad

Similar to VXLAN Integration with CloudStack Advanced Zone (20)

PDF
20 - IDNOG03 - Franki Lim (ARISTA) - Overlay Networking with VXLAN
Indonesia Network Operators Group
 
PDF
VMworld 2013: Troubleshooting VXLAN and Network Services in a Virtualized Env...
VMworld
 
PDF
Xpress path vxlan_bgp_evpn_appricot2019-v2_
Jide Akintola JNCIE-M&T/SP #496 CCIE-SP#28552
 
PDF
PLNOG15: Is there something less complicated than connecting two LAN networks...
PROIDEA
 
PDF
Hungary Usergroup - Midonet overlay programming
Marton Kiss
 
PPTX
Network and Service Virtualization tutorial at ONUG Spring 2015
SDN Hub
 
PDF
Openstack v4 0
sprdd
 
PDF
EYWA Presentation v0.1.27
JungIn Jung
 
PDF
Practical Design Patterns in Docker Networking
Docker, Inc.
 
PPTX
DragonFlow sdn based distributed virtual router for openstack neutron
Eran Gampel
 
PDF
Scalable Enterprise Ready Neutron Networking with Nuage Networks
Scott Sneddon
 
PDF
PLNOG 5: Piotr Szołkowski - Data Center i nie tylko...
PROIDEA
 
PDF
MidoNet 101
alexbikfalvi
 
PDF
SR-IOV, KVM and Emulex OneConnect 10Gbps cards on Debian/Stable
juet-y
 
PDF
Linux Tag 2014 OpenStack Networking
yfauser
 
PPTX
Network Design patters with Docker
Daniel Finneran
 
PPTX
Odl virtualization-20140520
NEC Corporation
 
PPTX
NET4933_vDS_Best_Practices_For_NSX_Francois_Tallet_Shahzad_Ali
shezy22
 
PDF
UNDOCUMENTED Vyatta vRouter: Unbreakable VPN Tunneling (MEMO)
Naoto MATSUMOTO
 
PDF
6. hands on - open mano demonstration in remote pool of servers
videos
 
20 - IDNOG03 - Franki Lim (ARISTA) - Overlay Networking with VXLAN
Indonesia Network Operators Group
 
VMworld 2013: Troubleshooting VXLAN and Network Services in a Virtualized Env...
VMworld
 
Xpress path vxlan_bgp_evpn_appricot2019-v2_
Jide Akintola JNCIE-M&T/SP #496 CCIE-SP#28552
 
PLNOG15: Is there something less complicated than connecting two LAN networks...
PROIDEA
 
Hungary Usergroup - Midonet overlay programming
Marton Kiss
 
Network and Service Virtualization tutorial at ONUG Spring 2015
SDN Hub
 
Openstack v4 0
sprdd
 
EYWA Presentation v0.1.27
JungIn Jung
 
Practical Design Patterns in Docker Networking
Docker, Inc.
 
DragonFlow sdn based distributed virtual router for openstack neutron
Eran Gampel
 
Scalable Enterprise Ready Neutron Networking with Nuage Networks
Scott Sneddon
 
PLNOG 5: Piotr Szołkowski - Data Center i nie tylko...
PROIDEA
 
MidoNet 101
alexbikfalvi
 
SR-IOV, KVM and Emulex OneConnect 10Gbps cards on Debian/Stable
juet-y
 
Linux Tag 2014 OpenStack Networking
yfauser
 
Network Design patters with Docker
Daniel Finneran
 
Odl virtualization-20140520
NEC Corporation
 
NET4933_vDS_Best_Practices_For_NSX_Francois_Tallet_Shahzad_Ali
shezy22
 
UNDOCUMENTED Vyatta vRouter: Unbreakable VPN Tunneling (MEMO)
Naoto MATSUMOTO
 
6. hands on - open mano demonstration in remote pool of servers
videos
 

Recently uploaded (20)

PDF
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
Michael Weaver
 
PDF
Internet of Things (IoT) – Definition, Types, and Uses
GrapesTech Solutions
 
PPT
Overviiew on Intellectual property right
Ritu Maity
 
PDF
CCUS-as-the-Missing-Link-to-Net-Zero_AksCurious.pdf
AkarshaSrivastava1
 
PDF
Ebook - The Future of AI A Comprehensive Guide.pdf
baljeethmata
 
PPT
Storage Area Network Best Practices from HP
MuraliShankar22
 
PPTX
Report in SIP_Distance_Learning_Technology_Impact.pptx
KahelCaponesPamitang
 
PPTX
Digital Convergence: How GIS, BIM, and CAD Revolutionize Asset Management
Richard Chappell, GISP
 
PDF
Chapter 1: computer maintenance and troubleshooting
BarnootaKomputeraaaf
 
PDF
Advancements in abstractive text summarization: a deep learning approach
IAESIJAI
 
PDF
Fitaura: AI & Machine Learning Powered Fitness Tracker
Riwaz Mahat
 
PPTX
Information-Technology-in-Human-Society (2).pptx
744tusharsingh094
 
PDF
State of AI in Business 2025 - MIT NANDA
Razin Mustafiz
 
PDF
Human Computer Interaction Miterm Lesson
JasperGarcia9
 
PPTX
maintenance powerrpoint for adaprive and preventive
aladortigabu
 
PDF
NewMind AI Journal Monthly Chronicles - August 2025
NewMind AI
 
PDF
Optimizing bioinformatics applications: a novel approach with human protein d...
IAESIJAI
 
PDF
ment.tech-Siri Delay Opens AI Startup Opportunity in 2025.pdf
thomasnova26
 
PDF
The AI Revolution in Customer Service - 2025
Body of Knowledge
 
PDF
Gestión Unificada de los Riegos Externos
Cristian Garcia G.
 
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
Michael Weaver
 
Internet of Things (IoT) – Definition, Types, and Uses
GrapesTech Solutions
 
Overviiew on Intellectual property right
Ritu Maity
 
CCUS-as-the-Missing-Link-to-Net-Zero_AksCurious.pdf
AkarshaSrivastava1
 
Ebook - The Future of AI A Comprehensive Guide.pdf
baljeethmata
 
Storage Area Network Best Practices from HP
MuraliShankar22
 
Report in SIP_Distance_Learning_Technology_Impact.pptx
KahelCaponesPamitang
 
Digital Convergence: How GIS, BIM, and CAD Revolutionize Asset Management
Richard Chappell, GISP
 
Chapter 1: computer maintenance and troubleshooting
BarnootaKomputeraaaf
 
Advancements in abstractive text summarization: a deep learning approach
IAESIJAI
 
Fitaura: AI & Machine Learning Powered Fitness Tracker
Riwaz Mahat
 
Information-Technology-in-Human-Society (2).pptx
744tusharsingh094
 
State of AI in Business 2025 - MIT NANDA
Razin Mustafiz
 
Human Computer Interaction Miterm Lesson
JasperGarcia9
 
maintenance powerrpoint for adaprive and preventive
aladortigabu
 
NewMind AI Journal Monthly Chronicles - August 2025
NewMind AI
 
Optimizing bioinformatics applications: a novel approach with human protein d...
IAESIJAI
 
ment.tech-Siri Delay Opens AI Startup Opportunity in 2025.pdf
thomasnova26
 
The AI Revolution in Customer Service - 2025
Body of Knowledge
 
Gestión Unificada de los Riegos Externos
Cristian Garcia G.
 

VXLAN Integration with CloudStack Advanced Zone

  • 1. VXLAN Integration with CloudStack Advanced Zone CCCEU13 - Amsterdam Nov. 21, 2013 Shinya Adachi [email protected] Yoshikazu Nojima [email protected] 1
  • 2. Why we come to here? • Open source community growth is important -Because we want to be free from specific vendor products. • Contribute technology for cloud scalability to accelerate migration from customer onpremise to be on the cloud -By suggest one of possible solution to have massive scalability. 2
  • 3. Cloudn • CloudStack based public cloud services(Compute) • Currently available in Japan and the US • 2 interface for customer, Customer portal GUI and APIs (APIs over 150 including AWS compatible) • VPC type “Coming Soon” in Japan 3 3
  • 4. Problem: VLAN ID limitation • Advanced Zone o More functionality • • o NAT, FW, LB, VPN VPC Isolation required • • For each guest network For each VPC tier Virtual Router • Isolation Method: VLAN o Virtual Router Isolated VLAN IDs are limited • • o Advanced Zone Public Network Only 4096 Should be identical within a zone Guest Network VPC Tier VPC Tier # of Domains is limited by VLAN • Each domain requires at least one VLAN ID VM VM VM VM VM VM VPC 4
  • 5. VXLAN Overview VXLAN [Virtual eXtensible Local Area Network] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelope type UDP packet (L4 packet) Standardization Status Under IETF standardization process Implementation Software-based : Cisco Nexus Series Switches, VMware vSphere Distributed Switch, Open vSwitch, and Linux kernel hardware based : Arista 7150, Brocade ADX series Characteristics • • • • 16M (2^24) isolated networks On top of UDP packet • Can utilize L4 port based ECMP load balancing solutions • Src UDP port is a hash of payload MAC addr Ethernet broadcast is mapped to IP multicast • L2: IGMP (or MLD) snooping, otherwise it floods a little • L3: If you want to communicate across L3 subnet Dynamic tunnel endpoint learning http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-06 5
  • 6. How traffic flows with VXLAN? Underlay Network for VXLAN ethX ethX vxlanM vxlanM vxlanM brethY-M brethY-M brethY-M vnet vnet vnet VM1 VM2 VM3 Host1 Host2 ethX Host3 1. If Unicast and KVM host (Src) learned mapping between VM and KVM host (Dst)  VXLAN uses Unicast 2. If broadcast or Unicast but KVM host (Src) doesn’t know mapping  VXLAN uses Multicast 6
  • 7. Host2 VTEP IP address resolution table ex. ping VNI Capsule Dst IP addr N Underlay Network for VXLAN Payload Dst MAC addr VM3 MAC addr Host3 IP addr Host3 VTEP IP address resolution table VNI Payload Dst MAC addr Capsule Dst IP addr N VM2 MAC addr Host2 IP addr VTEP : VXLAN Tunnel End Point ethX ethX vxlanN vxlanN vxlanN bridge bridge bridge vnet vnet vnet VM1 VM2 VM3 Host1 Host3 Host2 Payload # ethX VXLAN header Src MAC addr Dst MAC addr VNI Src IP addr Dst IP addr Src UDP port number 1 ARP request VM2 MAC addr Broadcast N Host2 IP addr Multicast IP addr Hash(VM2 MAC addr) 2 ARP reply VM3 MAC addr VM2 MAC addr N Host3 IP addr Host2 IP addr Hash(VM3 MAC addr) 3 ICMP Echo request VM2 MAC addr VM3 MAC addr N Host2 IP addr Host3 IP addr Hash(VM2 MAC addr) 4 ICMP Echo reply VM3 MAC addr VM2 MAC addr N Host3 IP addr Host2 IP addr Hash(VM3 MAC addr) 7
  • 8. How VXLAN shrink broadcast domain size? Underlay Network for VXLAN Underlay network segment 1 Underlay network segment 2 Underlay network segment 3 ethY vxlanN vxlanM bridge bridge bridge vnet (no VM associated with VNI : N) vxlanN vnet vnet VM2 VM3 VM1 Host1 Host2 Host3 Host4 1. Host1 contains no VM belong to VXLAN segment N, Host 1 doesn’t join the multicast group N. 2. Since VM1 & VM2 belong to VXLAN segment N, Host 2 & 3 join same multicast group N. 3. Since Host 4 contains no VM belong to VXLAN segment N, the path to Host4 is excluded from multicast domain if the switch supports IGMP snooping 8
  • 10. CloudStack KVM VLAN – bridging Overview Internet Public Network ethX KVM ethX KVM cloudbrL cloudbrJ vnetG vnetE VR vnetF brethY-M brethY-M Underlay Network vnetG vnetH VR vnetG VM VM ethY Encapsulate / Decapsulate ethY Encapsulate / Decapsulate Guest Network vnetA VR vnetB brethY-N vnetC VM ethY.N ethY.N VLAN Tunnel (VLAN ID : N) brethY-N vnetD VM 10
  • 11. CloudStack KVM VXLAN – bridging Overview Internet Public Network ethX KVM ethX KVM cloudbrL cloudbrJ vnetG vnetE VR vnetF brethY-M brethY-M Underlay Network vnetG vnetH VR vnetG VM VM ethY Encapsulate / Decapsulate ethY Encapsulate / Decapsulate Guest Network vnetA VR vnetB brvxN vnetC VM vxlan-N vxlan-N VXLAN Tunnel (VNI : N) brvxN vnetD VM 11
  • 13. 13
  • 15. Functional test result overview We tested the basic functions directly affected by VXLAN support. (ex. VM start/stop, Internet connectivity, Inter-tier connectivity and live migration in Isolated Network and VPC tier ) Case # VR/VM location 1 VR&VM exist in a same hypervisor Network type isolated Test target function connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart 2 VR&VM exist in different hypervisors isolated connectivity to VR after VM restart connectivity to the internet after VM restart connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart connectivity to VR after VM restart connectivity to the internet after VM restart VM migration connectivity to VR after VM restart connectivity to the internet after VM restart 3 VM1&VM2 exist in different isolated network isolated inter isolated network isolation 4 VR&VM exist in different hypervisors VPC connectivity to VR connectivity to the internet VR restart connectivity to VR after VR restart connectivity to the internet after VR restart VM restart connectivity to VR after VM restart connectivity to the internet after VM restart VM migration connectivity to VR after VM restart connectivity to the internet after VM restart 5 VM1&VM2 exist in different tier, and routing between two tier is allowed 6 VM1&VM2 exist in different tier, and routing between two tier is denied Step # Procedure Expected result Result 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 11 migrate VM to another hypervisor 12 ping to VR 13 ping to the host in the internet (ex. 8.8.8.8) ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully ping reaches to the destination ping reaches to the destination Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass 1 ping from VM1 in one tier to the internet(ex. 8.8.8.8) 2 tcpdump from VM2 in another tier 1 ping to VR 2 ping to the host in the internet (ex. 8.8.8.8) 3 stop VR 4 start VR 5 ping to VR 6 ping to the host in the internet (ex. 8.8.8.8) 7 stop VM 8 start VM 9 ping to VR 10 ping to the host in the internet (ex. 8.8.8.8) 11 migrate VM to another hypervisor 12 ping to VR 13 ping to the host in the internet (ex. 8.8.8.8) ping reaches to the destination ping packet from VM1 cannot captured ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully job finishes successfully ping reaches to the destination ping reaches to the destination job finishes successfully ping reaches to the destination ping reaches to the destination Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass VPC inter-tier connectivity 1 ping from VM1 in one tier to VM2 in another tier ping reaches to the destination Pass VPC inter-tier isolation 1 ping from VM1 in one tier to the internet(ex. 8.8.8.8) 2 tcpdump from VM2 in another tier ping reaches to the destination ping packet from VM1 cannot captured Pass Pass 15
  • 16. VXLAN plugin restriction • VXLAN is not available for Public Network, Storage Network, and Management Network • These networks do not consume many VLAN IDs. • KVM is the only supported hypervisor • Maybe we can add LXC support • Mapping between VNI and multicast address is hardcoded. multicastAddress= "239.$(( ($vni >> 16) % 256 )).$(( ($vni>> 8) % 256 )).$(( $vni % 256 ))" 16
  • 17. Resources • CloudStack Plugin guide for VXLAN • http://jenkins.buildacloud.org/job/build-docs-vxlanmaster/lastSuccessfulBuild/artifact/Apache_CloudStack -4.3.0-CloudStack_VXLAN_Guide-en-US.pdf • Design Doc • https://cwiki.apache.org/confluence/display/CLOUDSTA CK/Linux+native+VXLAN+support+on+KVM+hypervisor • JIRA ticket • https://issues.apache.org/jira/browse/CLOUDSTACK2328 Bug report, suggestions, any feedbacks are welcome! 17
  • 18. Wrap up • VXLAN integration for CloudStack we contributed is merged to CloudStack 4.3 branch. • We confirmed basic functions work in Isolated Network and VPC Tier. • Please evaluate VXLAN integration, any bug report, suggestions, feedbacks are welcome! Special Thanks: Toshiaki Hatano NTT Communications Corp. Junji Arakawa NTT Communications Corp. Chris Cameron Verio Inc. 18
  • 20. NVGRE Overview NVGRE [Network Virtualization using Generic Routing Encapsulation] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelop type Extended GRE packet (L3 packet) Standardization Status Under IETF standardization process Implementation Microsoft Hyper-V 2012 R2, Intel Ethernet Switch FM6000 Series Characteristics • • 16M (2^24) isolated networks Extended GRE packet • Utilize GRE packet’s key option field as VSID and flow-ID. • ECMP load balancing solutions must be aware of NVGRE flow-ID • Spec leaves Ethernet broadcast undefined. • Mapping to IP multicast is suggested. • Multicast network operation is required. http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03 20
  • 21. STT Overview STT [Stateless Transport Tunnel] Objective Overcome VLAN scalability limitation NW Type Overlay network Envelop type TCP like original L3 packet (protocol type is same as TCP. Pretends TCP packet.) Standardization Status Under IETF standardization process Implementation VMware NSX (formerly Nicira NVP) Characteristics • • 2^64 isolated networks TCP-like header + STT header • Can utilize NIC’s TSO feature • FW/router may drop STT packets by statefull inspection. • Spec leaves Ethernet broadcast undefined. • Mapping to IP multicast is suggested. • Multicast network operation is required. http://tools.ietf.org/html/draft-davie-stt-04 21
  • 22. Solutions comparison VXLAN NVGRE STT Overhead Header Size ○ (50 bytes) ○ (42 bytes) △(76 bytes) NIC Offloading ○ (Special NIC is required) ○ (Special NIC is required) ◎ (able to utilize normal TSO) Existing Assets Fitness ◎ (MTU may need to be adjusted) ◎ (MTU may need to be adjusted) △ (FW/router may drop STT packets) Interoperability ○ (Spec left only minor undefined points) ×（Tunnel endpoint address resolution is undefined.） ×（Tunnel endpoint address resolution is undefined.) Ethernet Broadcast ◎ (Mapping to IP Multicast) △ (Mapping to IP Multicast (suggestion)) △ (Mapping to IP Multicast (suggestion)) ECMP ○ (Able to utilize L2 fabric's L4 port base balancing) △ (L2 fabric must aware NVGRE flow-ID to balance) ○ (Able to utilize L2 fabric’s L4 port base balancing) Multicast Operation Required Required (depends on implementation) Required (depends on implementation) Supporting Vendors VMware/Citrix/Red Hat/ Cisco/Intel/Broadcom/Arista Microsoft/Arista/Emulex/ Dell/HP VMware(formerly Nicira) Linux Integration ◎ (kernel 3.7 or later) × (no implementation exists) ○ (Nicira’s Open vSwitch is required) 22
  • 23. VXLAN Terminology • • • • • VXLAN • Virtual eXtensible Local Area Network VXLAN Segment • VXLAN Layer 2 overlay network over which VMs communicate VTEP • VXLAN Tunnel End Point • an entity which originates and/or terminates VXLAN tunnels VNI • VXLAN Network Identifier (or VXLAN Segment ID) VXLAN Gateway • an entity which forwards traffic between VXLAN and non-VXLAN environments 23
  • 24. VXLAN segment format 0 Outer Ethernet Header: - FCS is newly calculated, inner FCS is omitted. Outer IP header - If Inner dst MAC is unicast MAC and local VTEP knows remote VTEP for the MAC address, dst IP set to remote VTEP’s IP address. - If not, packet will be sent out to multicast group associated with VNI. - The VTEP will use (*,G) joins. Outer UDP header - Source port: It is recommended to be calculated from inner Ether Header, for ECMP purpose. - Destination port: 4789 - Checksum: SHOULD be 0. Or correct value VXLAN header - VNI has a 24-bit field From current draft: (IPv4 case) http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-06 2013-04-17: IANA assigned udp/4789 for VXLAN port http://www.iana.org/assignments/service-names-portnumbers/service-names-port-numbers.xml 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 Outer Ethernet Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination MAC Address | Outer Source MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Source MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |OptnlEthtype = C-Tag 802.1Q | Outer.VLAN Tag Information | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ethertype = 0x0800 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Outer IPv4 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live |Protocl=17(UDP)| Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Source IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Destination IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Outer UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = xxxx | Dest Port = VXLAN Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ VXLAN Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R|R|R|R|I|R|R|R| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VXLAN Network Identifier (VNI) | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (continuing to Inner Ethernet header, abbrev.) 24
  • 26. Network concepts in CloudStack • Guest Network • • • Virtual network VMs are connected Isolated each other There are two type of Guest networks • Isolated network – – Traffic from VMs go out to Public Network through Virtual Router. VR is created per Isolated network. • Shared network – Traffic from VM go out directly. • VPC • • In VPC, Virtual Router can have multiple Isolated Networks (It is called VPC tier). In VPC, routing between tier is configurable. 26
  • 27. Network concepts in CloudStack (cont.) • Isolation method • • • Method to isolate Guest Networks each other. Typical isolation method is VLAN. VXLAN need to be implemented as isolation method. • Physical Network • • Underlay network of Guest network Isolation method of guest network is specified while defining physical network • createPhysicalNetwork API has isolationmethods parameter. – http://cloudstack.apache.org/docs/api/apidocs4.2/root_admin/createPhysicalNetwork.html 27
  • 28. Advanced Zone Network Overview Internet Public Network KVM VR for VPC * VR for VPC will be created per VPC KVM VR VPC tier VM VM VM VM VM VM Guest Network VM • • User can create multiple VPCs (depending on settings, up to 20 by default) VPC can have multiple tiers (depending on settings, up to 3 by default) 28
  • 29. Step by Step VXLAN base zone setup procedure 29
  • 30. VXLAN base zone setup procedure(1) - login 30
  • 31. VXLAN base zone setup procedure(2) - welcome wizard Click skip bottom 31
  • 32. VXLAN base zone setup procedure(3) - infrastructure tab Click Infrastructure tab 32
  • 33. VXLAN base zone setup procedure(4) 33
  • 34. VXLAN base zone setup procedure(5) Select Advanced type 34
  • 35. VXLAN base zone setup procedure(6) Fill zone wizard. KVM is the only supported hypervisor. 35
  • 36. VXLAN base zone setup procedure(7) Management network/Public network/Storage network are not supported by VXLAN Select VLAN Select VXLAN Guest network is supported by VXLAN Set underlay network I/F name (ex. “eth0”) to traffic type 36
  • 37. VXLAN base zone setup procedure(8) Fill zone wizard. There is no VXLAN specific concern. 37
  • 38. VXLAN base zone setup procedure(9) Fill zone wizard. There is no VXLAN specific concern. 38
  • 39. VXLAN base zone setup procedure(10) You can use 0-16777215 as VNI 39
  • 40. VXLAN base zone setup procedure(11) Fill zone wizard. There is no VXLAN specific concern. 40
  • 41. VXLAN base zone setup procedure(12) Fill zone wizard. There is no VXLAN specific concern. 41
  • 42. VXLAN base zone setup procedure(13) Fill zone wizard. There is no VXLAN specific concern. 42
  • 43. VXLAN base zone setup procedure(14) Fill zone wizard. There is no VXLAN specific concern. 43
  • 44. VXLAN base zone setup procedure(15) Fill zone wizard. There is no VXLAN specific concern. 44
  • 45. VXLAN base zone setup procedure(16) Click “Launch zone” button 45
  • 46. VXLAN base zone setup procedure(17) 46
  • 47. VXLAN base zone setup procedure(18) Click “Yes” button 47
  • 48. VXLAN base zone setup procedure(19) Add Instance wizard. There is no VXLAN specific concern. 48
  • 49. VXLAN base zone setup procedure(20) Add Instance wizard. There is no VXLAN specific concern. 49
  • 50. VXLAN base zone setup procedure(21) Add Instance wizard. There is no VXLAN specific concern. 50
  • 51. VXLAN base zone setup procedure(22) Add Instance wizard. There is no VXLAN specific concern. 51
  • 52. VXLAN base zone setup procedure(23) Add Instance wizard. There is no VXLAN specific concern. 52
  • 53. VXLAN base zone setup procedure(24) Add Instance wizard. There is no VXLAN specific concern. 53
  • 54. VXLAN base zone setup procedure(25) Add Instance wizard. There is no VXLAN specific concern. 54
  • 55. VXLAN base zone setup procedure(26) 55
  • 56. VNI validation If your VNI range is invalid, This error will pop up. 56