Usage of VXLAN and VXLAN traffic flow

Traditionally, VLANs / Virtual LANs have been used to segment broadcast domains.

Such segmentation has been beneficial due to numerous benefits:
a. Smaller broadcast domains means fewer hosts that will process broadcasts from other hosts.
b. This also saves CPU and memory of all involved devices in a broadcast domain.

These VLANs are nowadys being replaced by VXLAN in data centers due to some major benefits.

Why VXLAN based Data Centers?

1.    Multitenancy

Many VXLAN segments as compared to VLANs.
Virtual Network Identifiers VNIs are used to identify the overlay segments; these overlays are created over existing physical networks.

Virtual Network Identifier VNI is 24 bit thereby enabling many VXLAN based overlays as compared to VLANs (Max 4094)

This means that you are able to provision more customers in your VXLAN based Data Center as compared to traditional data centers.

2.    Layer 2 adjacency across different sites means extending layer 2 but at the same time extending Spanning Tree Protocol - STP domains across data centers.

Such layer 2 extensions with inherent STP have been known to cause instabilities in network.

Instability of Layer 2 only WAN link will cause a state change and Spanning Tree re-convergence.

VXLAN based logical switches do not extend STP domains, there is no STP in a VXLAN based network.

  

Traffic flow in a VXLAN based network.

This topic is very fundamental to the understanding of VXLAN encapsulation & it is worthwhile having a thorough understanding of this topic.

Just like the understanding of IP routing process, a good understanding of this topic will enable network engineers to design, implement and operate a VXLAN based network.

In the above figure, VXLAN encapsulation and de capsulation is done by VTEPs.

In case of VMWare NSX, hypervisor ESXi is the VTEP and VXLAN based tunnels/overlays are between these VTEPs.

For traffic that is coming into VTEP A, VXLAN header is introduced before the traffic gets into IPv4 network. IPv4 network provides necessary transport for VXLAN encapsulated frames to move between VTEPs.

Vxlan frame format sheds a lot of light upon the different fields in VXLAN header.

For traffic sent out from virtual machine A to virtual machine B in the other site, these headers will be:

a.   VNI

This field identifies the overlay virtual network where virtual machine resides.

b.   Outer source IP

IP address of VTEP A

c.   Outer destination IP

This IP will be either destination IP of VTEP B or  this destination IP will be a multicast IP address mapped to a particular VNI ( This VNI corresponds to overlay virtual network where virtual machine A is connected)

d.   Outer Source MAC address

This will MAC address of VTEP A

e.   Outer Destination MAC address

As per the topology above, this will be MAC address of intermediate router / next hop router.