NSX-T Routing Configuration

NSX-T Routing Configuration

Overall Topology used in the lab

Pre-requisites like NSX-T manager installation, preparing and configuring compute host transport nodes, preparing and configuring edge transport nodes are covered here.

As shown in the topology above, two Tier 0 gateways are configured in the lab.
One Tier 0 gateway is configured in Active-Active High Availability mode and the other Tier 0 gateway is configured in Active-Passive High Availability mode.
I will be referring to the two Tier 0 Gateways as Tier 0 Gateway Left and Tier 0 Gateway Right.

A total of four edge node VMs are utilized, two for each Tier 0 gateway.
Edge node clusters are created, two edge node clusters are defined. Each edge node cluster effectively utilizes two edge node VMs.

BGP peerings

The above topology shows the e BGP peerings between NSX-T Edge Nodes and the physical routers.
Two VLANs are utilized as shown above.

Edge Transport Nodes

Here you see that Edge Transport Nodes are ready.

Below you find the interface configs applied on the Tier 0 Gateways.

Interfaces belonging to Edge Nodes on Tier 0 Gateway Left

Interfaces belonging to Tier 0 Gateway Right

BGP Diagram along with IP addressing

This topology shows the BGP AS numbers used in the lab setup.
Also shown in the diagram is the IP addressing used.
Segment with subnet 172.16.10.0/24 is attached to a Tier 1 Gateway Left.
And a segment with subnet 10.0.0.0/24 is directly attached to Tier 0 Gateway Right.

========================================

BGP Configuration on Tier 0 Gateway

Add caption

For BGP configuration, edit the Tier 0 Gateway and configure first the Local BGP AS number and save the configuration.
Next set the BGP peers.

BGP peer configuration on Tier 0 Gateway

You will apply similar BGP configuration on the other Tier 0 Gateway Right but in that case the Local BGP AS number is 65002 as shown in the BGP topology above and the remote AS number will be same i.e. 65001


BGP configuration on the TOR Physical Routers
Make sure you have BGP configured on the physical routers and also ensure that the BGP peerings are up.

BGP peerings on TOR1

BGP Peerings on TOR2

============================================

Configure redistribution on Tier 0 Gateways and Tier 1 Gateway

As mentioned in my earlier post, there is no dynamic routing between Tier 0 gateway and Tier 1 gateway in NSX-T.

We just need to configure redistribution on Tier 0 Gateway and Tier 1 Gateway appropriately.

As shown above, we are redistributing connected networks on Tier 1 Gateway


Enable redistribution on Tier 0 Gateway.
Ensure you also redistribute connected subnet of Tier 1 gateway on Tier 0 gateway.

Follow the same steps to enable redistribution of connected subnet on Tier 0 Gateway Right.

===============================================

Validation

The above command is executed from the CLI of NSX-T Edge Node VM.

This is the first edge node VM corresponding to Tier 0 Gateway Left.

We know that the Tier 0 Logical Router consists of SR and DR; the SR sits atop the DR.

Notice from above output that VRF 5 corresponds to the SR with name as SR-T0-GW

Let's go to VRF 5

High Availability Mode on Tier 0 SR

Check the high availability status of this Tier 0 SR corresponding to Tier 0 Gateway Left and you will find that it is Active-Active

BGP Peerings on Tier 0 SR corresponding to Tier 0 Gateway Left

Notice the three BGP peerings:

a. BGP peering with TOR1

b. BGP peering with TOR2

c. Inter SR BGP peering using link local IP address 169.254.0.131

Notice the BGP best path advertisement.

Also notice the additional attributes like metric, Local Preference, Weight.

Check all the BGP routes on this SR, these BGP routes are learnt on the SR of Tier 0 Gateway Left.

Below are all the routes noticed on Tier 0 SR corresponding to Tier 0 Gateway Left. This is the routing table.

=======================================

Validation on the physical routers for subnet connected to Tier 0 Gateway Right

Let's check how the route for 10.0.0.0/24 (which is locally connected to Tier 0 Gateway Right) is learnt by physical routers 

Notice from the output above, that the route is learnt via the Active Edge Node VM corresponding to Tier 0 Gateway Right.

Standby Edge Node VM corresponding to Tier 0 Gateway Right is doing Auto AS Path Prepend.

There is no explicit configuration done on the Tier 0 Gateway to achieve this AS path prepend.

=========================

End to end connectivity between Windows VM attached to 10.0.0.0/24 and Ubuntu machine connected to 172.16.10.0/24


Use ping and trace to verify connectivity between source and destination VM.

Ensure that you are able to ping with default VM MTU of 1500 bytes.

Palo Alto service insertion for Cross Vcenter NSX-V

Palo Alto service insertion for Cross Vcenter NSX-V 

References:

Multi-site with Cross-VC NSX and Palo Alto Networks Security

https://blogs.vmware.com/networkvirtualization/2016/09/multi-site-cross-vc-nsx-palo-alto-networks-security.html/

Cross Vcenter NSX design guide

https://blogs.vmware.com/networkvirtualization/2016/07/nsx-v-multi-site-options-cross-vc-nsx-design-guide.html/

Palo Alto service insertion in a single vcenter hosted in single DC

A brief about the Software Defined Data Center topology above:

- A single vcenter.
- A single NSX manager
- A single Panorama
- Controller cluster consisting of three nodes.
- Two clusters under the single vcenter.
- Multiple Logical Switches to which VMs are connected.
- Logical switches are connected to Distributed Logical Router and the gateway for the VMs is the Distributed Logical Router. 
- VMs are protected by Palo Alto Networks VM series firewall applied at the vnic of each VM.
- Two NSX edges to peer with physical network using VLAN and to peer with Distributed Logical Router using VXLAN/Logical Switch.
- Physical firewall is operating in Active/Standby way and hence a single VLAN is used to peer the NSX edges with the firewall.
- A single transport zone to which both the clusters are connected. Logical switches will be defined using this single transport zone.
- NSX domain uses BGP Private AS 64513.
And the physical network uses BGP Private AS 64512. 
- Both the NSX edges are operating in ECMP mode and hence both are two different routers. ECMP mode allows more bandwidth for North-South traffic, potentially close to 20 Gbps for the case of two NSX edges.
- e BGP peering is used between NSX edges and physical firewall
- i BGP peering is used between NSX edges and Distributed Logical Router.

 
Keeping this SDDC topology in mind, we will now move on to the integration between Palo Alto Networks VM series firewall and VMware NSX-V.


Content filtering techniques of Palo Alto Networks firewall has been covered earlier.

Integration between NSX and Palo Alto Networks VM series firewall brings about benefits like Application ID, vulnerability protection, malware protection, sand boxing and protection from zero day threats, threat analysis, security policies based on user ID, SSL decryption.

Steps for such an integration are:
1. Integrate Panorama with NSX manager.
One Panorama can possibly integrate with multiple NSX managers. And there is no 1:1 relationship between Panorama and NSX Manager.
2. Once the Palo Alto Networks service is available in NSX, prepare the clusters for this service.
Once the clusters are prepared, each host will have a Palo Alto Networks service VM.
Palo Alto Networks service VMs require distributed port groups for networking and data stores for storage.
3. Create security groups in NSX based on static or dynamic inclusion criteria.
5. Leverage these security groups and create redirection policies in NSX. 
This is where you redirect traffic from the VM towards Palo Alto Networks service VM sitting on host where VM resides.

You can also possibly leverage these security groups in Panorama security policies.
There is 1:1 relationship between security groups of NSX and dynamic address groups in Panorama.
Members in Palo Alto Networks dynamic address groups are populated dynamically based on security group criteria of VMware NSX.

====================================

A bit about security groups in NSX-V

In NSX-V, you are able to create security groups based on static or dynamic criteria.
Security groups can be created from Service Composer in NSX-V
Static inclusion criteria for NSX 6.4.x can contain the following:

1. Other security groups to nest within the security group you are creating.
2. Cluster
3. Logical switch
4. Virtual App
5. Datacenter
6. IP sets
7. MAC Sets
8. Security tag
9. vNIC
10. Virtual Machine
11. Resource Pool
12. Distributed Virtual Port Group

Security group membership can change constantly.

Dynamic inclusion criteria while creating security group can be based on VM name, security tag, computer OS name.


=====================================================================

Next we will cover Cross vcenter NSX along with Palo Alto Networks VM series firewall.

Cross Vcenter NSX-V using BGP has been covered in this blog.
A cross vcenter NSX environment has:
- Two vcenters
- Two NSX managers
- Controller cluster is Universal Controller Cluster
- Primary and Secondary NSX manager.
- Universal Transport Zone 
- Universal Logical Switches
- Universal Distributed Logical Router
- Universal Distributed Firewall

VMs in a Cross vcenter NSX topology are connected to the Universal Logical Switches

In a cross vcenter NSX environment, it is possible to redirect traffic flows towards Palo Alto Networks VM series for workloads that are under the primary vcenter.
This is shown in figure below.

Note: The below topology covers a requirement where there are multiple vcenters in the same site.

Traffic flows for VMs in primary vcenter are supported by Palo Alto Networks service insertion

 
===============================================================

Cross Vcenter flows are not supported by this Palo Alto Networks service insertion and for such cross vcenter traffic flows you can possibly leverage the NSX Universal Distributed Firewall UDFW.

The key point here is that with UDFW you will create security policies on NSX using Universal IP sets and Universal MAC sets.
You can possibly create security groups with static inclusion of Universal Logical Switch using Service Composer but the effective membership will show VMs from a single vcenter only.
 

Cross Vcenter traffic flows are not supported by Palo Alto service insertion

You can possibly use Cross Vcenter NSX along with Palo Alto Networks VM series firewall for topology where there is Active Data Center with one vcenter and a Disaster Recovery Site with another vcenter.