NSX-T Federation - Active Active Data Centers

 

 NSX-T Federation - Active Active Data Centers

 

This blog covers NSX-T Federation feature which allows L2 stretching between Data Centers as well as supports micro segmentation for workloads based on security tags.

Earlier blogs covered NSX-T Federation with a single Tier 0 stretched Gateway.

Here we explore how two Tier 0 Gateways can be utilized for workloads which are active in both data centers.

Logical Setup used in Lab

 

The above setup is used in the lab.

 

A brief about the setup:

1. Global Manager sits in Bangalore

2. Both sites have local manager

3. Hosts in each site have been prepared as hosts transport nodes.

4. Edges have been deployed in each site and configured as edge transport nodes.

5. Each site has site local uplink VLANs, edge TEP VLAN, host TEP VLAN & RTEP VLAN

RTEP interfaces are instantiated on edges to handle inter site traffic.

Every stretched segment will have local edges designated as Active and Standby for that specific segment.

6. Four edges correspond to Tier 0 Gateway Bangalore which has Bangalore as Primary location and Delhi as secondary location.

One edge cluster in Bangalore and another in Delhi

7. Four edges correspond to Tier 0 Gateway Delhi which has Delhi as Primary location and Bangalore as secondary location.

One edge cluster in Bangalore and another in Delhi

8. Transport zone configuration, edge node configuration and hosts transport node configuration is done from Local Manager.

9. Stretched Tier 0 Gateway, segments used for uplinks of stretched Tier 0 Gateway and stretched Tier 1 Gateway is created from Global Manager UI.

Segments connected to stretched Tier 1 Gateways are also created from Global Manager UI.

10. A total of eight edge nodes are configured in this lab setup.

IP Addressing and VLAN details

NSX-T Fabric for Bangalore Location

IP Address Pools for Compute TEP, Edge TEP and RTEPs

Local Transport Zones in Bangalore

Uplink Profiles in Bangalore

 

Compute Transport Node Profile

Edge Node Connectivity

In the above diagram, edge is connected to VDS which was earlier used to configure NSX on hosts.

VLAN backed trunk segments are created on hosts' VDS for uplink connectivity of edge.

Fast path interfaces of edge are connected to these trunk segments.

Edge Transport Node Configuration

 

 

Host Transport Nodes are configured

 

Once hosts are configured for NSX, tunnel endpoint interfaces are created on the hosts, NSX-T software is installed on the hosts.

 

 

Edge Transport Nodes are configured

 

Once edges are configured for NSX, tunnel endpoint interfaces are created on edges and the edge is connected to appropriate trunk segments on host VDS.

 

Edge Clusters in Bangalore

 

 

RTEP configuration on edge clusters of Bangalore

RTEP configuration is applied to both edge clusters in Bangalore

Before applying configurations on Global Manager, ensure that below configurations are also applied in other location:

a. Transport Zones

b. IP Pools

c. Uplink Profiles

d. Compute Transport Node Profiles

e. Edge Transport Nodes config

f. Hosts are configured as host transport nodes.

g. RTEP configs on edge clusters in Delhi

BGP Setup

 

BGP Setup for Tier 0 Gateway Bangalore

BGP AS 65000 is used on Tier 0 Gateway Bangalore

e BGP is used between Tier 0 Gateway and upstream routers.

Physical network is under AS 65001

Traffic ingress and egress to/from subnet connected to Tier 1 Gateway Bangalore goes through physical routers in Bangalore.

This gives deterministic traffic flow.

AS Path prepending is used on physical routers of Delhi Location to influence this traffic flow.

Physical routers are sending a default route on a per BGP peer basis.

Routes from NSX are redistributed into BGP

BGP Setup for Tier 0 Gateway Delhi

 

BGP AS 65002 is used on Tier 0 Gateway Bangalore

e BGP is used between Tier 0 Gateway and upstream routers.

Physical network is under AS 65001

Traffic ingress and egress to/from subnet connected to Tier 1 Gateway Delhi goes through physical routers in Delhi

This gives deterministic traffic flow.

AS Path prepending is used on physical routers of Bangalore Location to influence this traffic flow.

Physical routers are sending a default route on a per BGP peer basis.

Routes from NSX are redistributed into BGP

 

Global Manager Configuration

Locations are added to Global Manager

Segments are created on Global Manager for uplink connectivity of Tier 0 Gateway

While creating segments on Global Manager, specify location, local transport zone and the VLAN ID

Tier 0 Gateway Bangalore

 

While creating stretched Tier 0 Gateway, specify the edge cluster and the corresponding location as Primary or Secondary.

For Tier 0 Gateway Bangalore, primary location is Bangalore.

For Tier 0 Gateway Delhi, primary location is Delhi

L3 interfaces on Tier 0 Gateway Bangalore

Likewise Tier 0 Gateway is created with Delhi as primary location & Bangalore as secondary location.

Tier 1 Gateway config on Global Manager

Next create stretched Tier 1 Gateway and connect to the already defined Tier 0 Gateway.

Tier 1 Gateways on Global Manager

 

 

Segments on Global Manager connected to Tier 1 Gateway

Next deploy VMs and connect them to appropriate segments.

 

 

VM connected to Overlay Network

Validation

Trace from router in Delhi to VM behind Tier 1 Gateway Bangalore goes through physical router of Bangalore

Trace from router in Bangalore to VM behind Tier 1 Gateway Delhi goes through physical router of Delhi

 

Trace from loopback of second physical router in Delhi to VM behind Tier 1 Gateway Bangalore goes through physical router 1 of Bangalore location

Trace from VM behind Tier 1 Gateway Bangalore towards loopback of physical router 2 in Delhi goes through physical router 2 of Bangalore location

 

Trace from VM in Delhi to loopback of physical router 1 in Bangalore goes through physical router of Delhi

Trace from loopback of physical router 2 in Bangalore to VM behind Tier 1 Gateway Delhi goes through physical router in Delhi

RTEP to RTEP tunnel is established