Usage of VXLAN and VXLAN traffic flow

Traditionally, VLANs / Virtual LANs have been used to segment broadcast domains.

Such segmentation has been beneficial due to numerous benefits:
a. Smaller broadcast domains means fewer hosts that will process broadcasts from other hosts.
b. This also saves CPU and memory of all involved devices in a broadcast domain.

These VLANs are nowadys being replaced by VXLAN in data centers due to some major benefits.

Why VXLAN based Data Centers?

1.    Multitenancy

Many VXLAN segments as compared to VLANs.
Virtual Network Identifiers VNIs are used to identify the overlay segments; these overlays are created over existing physical networks.

Virtual Network Identifier VNI is 24 bit thereby enabling many VXLAN based overlays as compared to VLANs (Max 4094)

This means that you are able to provision more customers in your VXLAN based Data Center as compared to traditional data centers.

2.    Layer 2 adjacency across different sites means extending layer 2 but at the same time extending Spanning Tree Protocol - STP domains across data centers.

Such layer 2 extensions with inherent STP have been known to cause instabilities in network.

Instability of Layer 2 only WAN link will cause a state change and Spanning Tree re-convergence.

VXLAN based logical switches do not extend STP domains, there is no STP in a VXLAN based network.

  

Traffic flow in a VXLAN based network.

This topic is very fundamental to the understanding of VXLAN encapsulation & it is worthwhile having a thorough understanding of this topic.

Just like the understanding of IP routing process, a good understanding of this topic will enable network engineers to design, implement and operate a VXLAN based network.

In the above figure, VXLAN encapsulation and de capsulation is done by VTEPs.

In case of VMWare NSX, hypervisor ESXi is the VTEP and VXLAN based tunnels/overlays are between these VTEPs.

For traffic that is coming into VTEP A, VXLAN header is introduced before the traffic gets into IPv4 network. IPv4 network provides necessary transport for VXLAN encapsulated frames to move between VTEPs.

Vxlan frame format sheds a lot of light upon the different fields in VXLAN header.

For traffic sent out from virtual machine A to virtual machine B in the other site, these headers will be:

a.   VNI

This field identifies the overlay virtual network where virtual machine resides.

b.   Outer source IP

IP address of VTEP A

c.   Outer destination IP

This IP will be either destination IP of VTEP B or  this destination IP will be a multicast IP address mapped to a particular VNI ( This VNI corresponds to overlay virtual network where virtual machine A is connected)

d.   Outer Source MAC address

This will MAC address of VTEP A

e.   Outer Destination MAC address

As per the topology above, this will be MAC address of intermediate router / next hop router.

SDN

SDN - Basics

Getting Started:

The key idea of an SDN is to split the network forwarding function, performed by the data plane, from the network control function, performed by the control plane. This allows a simpler and more flexible network control and management, and also network virtualization. 

OpenFlow is the main SDN implementation.

The network controller communicates with OpenFlow switches using the OpenFlow protocol through a secure channel. Using this connection, the controller is able to configure the forwarding tables of the switch.

If the devices do not have a rule for the data packet that has arrived, the devices forward the packet to the controller for review. The controller determines what to do with the packet and, if necessary, sends a new rule to the device so that it can handle future data packets in the same manner.

The control plane generates the routing table, whereas the data plane, using the control plane tables, determines where the packets should be sent to.

Interesting, yeah?

======================================================================

SDN allows for virtualization of networks so that one can look at the network more abstractly and focus on the big picture rather than on all the gritty details.

Example being Vmware NSX.

In much the same way that server virtualization programmatically creates, snapshots, deletes and restores software-based virtual machines (VMs), VMware NSX network virtualization programmatically creates, snapshots, deletes, and restores software-based virtual networks. 

With network virtualization, the functional equivalent of a “network hypervisor” reproduces the complete set of layer 2 to layer 7 networking services (e.g., switching, routing, firewallingand load balancing) in software. As a result, these services can be programmatically assembled in any arbitrary combination, to produce unique, isolated virtual networks in a matter of seconds.

SDN Based Current Projects being worked on:

Currently, there are several projects that use OpenFlow, which includes Europe and Brazil. In Europe, eight islands are currently interconnected using OpenFlow, whereas in Brazil, there are plans to create a network that will work with that in Europe to create a more widespread testbed. The project in Brazil is particularly important because replacing the Internet is a global endeavor, and a network that will only work with landmasses clustered together is not a viable solution

Future capabilities of SDN:

Updating the Internet brings many challenges because it is constantly being used; it is difficult to test new ideas and strategies to solve the problems found in an existing network. SDN technologies provide a means of testing ideas for a future Internet without changing the current network

Rural areas will also benefit from SDN based solutions.

SDN allows companies to decrease startup costs in rural environments, thereby allowing them to gain more profit. As rural networks become more profitable, more companies will be willing to give access to more and more rural areas.

Another benefit seen is moving the workload to underused networks. If a network is busy at a certain time of the day, the workload might be completed sooner in a network of a different time zone or in a network that is more energy efficient.

References:

Books:

Network Innovation through Openflow and SDN -Fei Hu

URLs:

http://www.vmware.com/products/nsx

Nexus 1000v

Cisco Nexus 1000v


I am writing this blog to mention below key components/terms/definitions related to Nexus 1000v



What is a hypervisor?
Hypervisor is a program that allows multiple OS to share single hardware host



Basic Building Blocks

VSM
- It is the control/management plane of 1000v

VEM
- Data Plane


Traffic flow with VSM and VEM:
Traffic flow from VEM northbound does not go through VSM.

The physical NICs on the VEM server are uplinks to the external fabric. VEMs switch traffic between the local virtual Ethernet ports connected to VM vNICs, but do not switch traffic to other VEMs. Instead, a source VEM switches packets to uplinks that the external fabric then delivers to the target VEM. The VSM runs the control plane protocols and configures the state of each VEM, but it never takes part in the actual forwarding of packets.

A single VSM can control up to 64 VEMs

VSM Deployment

VSMs are ideally deployed as a pair with active and redundant VSM.

So even if the primary VSM fails, the backup VSM can take over.

Does disruption in VSM to VEM communication affect virtual machine traffic?

No

So how does Nexus 1000v compare to a modular Cisco switch?

Primary VSM-------------------Supervisor

Backup  VSM-------------------Redundant supervisor

VEM 1-----------------------------Line Card 1

VEM 2 ----------------------------Line Card 2

What are port profiles?

Port profile defines a set of attributes that can include the following:

a. VLAN

b, Port channels

c. Private VLAN

d. ACL

e. Port security

f. NetFlow

g. rate limiting

h. QoS Marking

The network administrator defines port profiles in the VSM. When the VSM connects to vCenter Server, it creates a distributed virtual switch (DVS) and each port profile is published as a port group on the DVS. The server administrator can then apply those port groups to specific uplinks, VM vNICs,  or management ports, such as virtual switch interfaces or VM kernel NICs.



VEM Virtual Ports

Three types of vNIC in VMWare
1. Virtual NIC

2. vEth port
Represent a port on Nexus 1000v virtual switch.
These ports are assigned to port group.

3. lvEth port
Dynamically selected for vEth ports on the host.


Further, there are 3 types of Virtual NICs.
1. vnic
Part of the VM and represents the physical port of the host which is plugged into the switch

2. vmknic
Used by the hypervisor for management,VMotion,iSCSI,NFS and other network access needed by kernel.

3. vswif
VMWare service console network interface, the Service Console network interface.
The vswif interface is used as VMWare management port; these interface types map to a veth port within Nexus 1000v


VSM to VEM Communication

1. Layer 3 Control Mode

VEMs can be in different subnet than the VSM and also from each other in Layer 3 control mode.

Each VEM needs a designated VMkernel NIC interface that is attached to the VEM that communicates with the VSM.
This interface which is called Layer 3 Control vmknic must have a system profile applied to it so the VEM can enable it before contacting the VSM.

2. Layer 2 Control Mode

VEM and VSM are in the same subnet in the Layer 2 control mode.