Showing posts with label nsx-v. Show all posts
Showing posts with label nsx-v. Show all posts

Friday, June 26, 2020

NSX-V to NSX-T Migration using Layer 2 Bridging

NSX-V to NSX-T Migration using Layer 2 Bridging

This blog will explore how we can migrate workloads which are on hosts prepared for NSX-V to hosts prepared for NSX-T using NSX-T Layer 2 Bridging.


Cluster Setup

In the lab setup, four hosts ESXi 1 up to 4 are prepared for NSX-T and the remaining four hosts ESXi5 up to ESXi 8 are prepared for NSX-V

NSX-T edges used for Layer 2 bridging are on NSX-V prepared hosts.


Logical Setup

The above is the logical setup used in this lab.


IP Addressing

Above shows the IP addressing used in the lab.


BGP AS Numbering

The above picture shows the BGP AS numbering used.


BGP peerings

The above diagram shows the BGP peerings.

e-BGP peerings between NSX and the physical network.

i BGP between NSX-V edges and Distributed Logical Router.

There is no routing protocol between Tier 1 Gateway of NSX-T and Tier 0 Gateway upstream.

During migration, traffic flow will be through NSX-V edges which means that:

1. You can prefer not to advertise connected subnets on the Tier 1 Gateway

2. Or to keep BGP disabled on Tier 0 Gateway.


NSX-V Setup

NSX-V Prepared Cluster

Above picture shows the four hosts prepared for NSX-V


NSX-V Edges and DLR

The required NSX-V edges and DLR have been deployed.


Workloads hosted on both clusters



VM on VXLAN

One VM Windows 10-2 is hosted on this NSX-V prepared cluster.


We need to make sure that security settings of the port group (corresponding to the VXLAN being bridged) are set accordingly.

  • Set promiscuous mode on the portgroup.
  • Allow forged transmit on the portgroup.

https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.5/administration/GUID-F133B293-5DEA-4DC8-99DB-6EF004C8D8D7.html

Security settings of VXLAN backed port group


NSX-T Setup

Overlay Transport Zone defined using REST API Client

To ensure unique mac addresses are used on layer 3 interfaces of DLR and Tier 1 Gateway respectively, ensure overlay transport zone in NSX-T is defined as above with

"nested_nsx": true



Compute host transport nodes prepared for NSX-T

NSX-T Edges

nsx-edge-1 and nsx-edge-2 are NSX-T edges which are used for Layer 2 bridging.
These edges are placed on cluster prepared for NSX-V.

The remaining two edges nsx-edge-3 & nsx-edge-4 are used for Tier 0 Gateway.




Edge used for L2 Bridging

Fast-path interfaces fp-eth0 and fp-eth1 on the edges used for Layer 2 bridging are used for Geneve traffic, they are uplinked to a trunk port group on VDS used for NSX-V preparation. This way all NSX traffic stays on this VDS which is also used for NSX-V host preparation.


NSX-T Edge Clusters

Tier 0 Gateway

Tier 1 Gateway

NSX-T Segment connected to Tier 1 Gateway

Gateway set on NSX-T Segment

Validation of the setup

VM on NSX-T Segment




Reach ability between physical router loopback and both VMs

The above picture shows BGP peerings between the router and NSX-V edges.
At this stage, the BGP peerings between the physical routers and NSX-T edges are down/disabled.
Reason being that all workloads from NSX-V prepared hosts are not yet on NSX-T prepared hosts.

Reach ability between VM on VXLAN to loopback of physical router

Reach ability between VM on NSX-T segment to loopback of physical router

At this point, we know that Layer 2 bridging is working as intended and that the layer 2 bridge is forwarding traffic upstream.


Traffic flow from VM on NSX-T Segment to loopback interface of physical router

Migration

Now we will migrate the VM which is on NSX-V prepared cluster to NSX-T prepared cluster.

With this, both the workloads will then be on NSX-T prepared cluster.

At this point of time, we need to ensure that workloads have Tier 1 Gateway as their gateway.

We will ensure BGP peerings between physical routers and NSX-T edges are now all up.

And disable the BGP peerings between physical routers and NSX-V edges


Workloads migrated to NSX-T prepared cluster

Above picture shows that workloads have moved to NSX-T prepared cluster.


After migration, traffic flow from physical router to VMs on NSX-T segment

From the physical router, we validate that BGP peerings with NSX-T edges are now up and those with NSX-V edges are down.

Traffic now starts flowing through NSX-T edges.


VM on NSX-T Segment to loopback IP of physical router

VM on NSX-T segment to loopback IP of physical router

Traffic flow after migration

Monday, July 1, 2019

Palo Alto service insertion for Cross Vcenter NSX-V

Palo Alto service insertion for Cross Vcenter NSX-V 

References:
Multi-site with Cross-VC NSX and Palo Alto Networks Security
https://blogs.vmware.com/networkvirtualization/2016/09/multi-site-cross-vc-nsx-palo-alto-networks-security.html/

Cross Vcenter NSX design guide
https://blogs.vmware.com/networkvirtualization/2016/07/nsx-v-multi-site-options-cross-vc-nsx-design-guide.html/

Palo Alto service insertion in a single vcenter hosted in single DC
A brief about the Software Defined Data Center topology above:

- A single vcenter.
- A single NSX manager
- A single Panorama
- Controller cluster consisting of three nodes.
- Two clusters under the single vcenter.
- Multiple Logical Switches to which VMs are connected.
- Logical switches are connected to Distributed Logical Router and the gateway for the VMs is the Distributed Logical Router. 
- VMs are protected by Palo Alto Networks VM series firewall applied at the vnic of each VM.
- Two NSX edges to peer with physical network using VLAN and to peer with Distributed Logical Router using VXLAN/Logical Switch.
- Physical firewall is operating in Active/Standby way and hence a single VLAN is used to peer the NSX edges with the firewall.
- A single transport zone to which both the clusters are connected. Logical switches will be defined using this single transport zone.
- NSX domain uses BGP Private AS 64513.
And the physical network uses BGP Private AS 64512. 
- Both the NSX edges are operating in ECMP mode and hence both are two different routers. ECMP mode allows more bandwidth for North-South traffic, potentially close to 20 Gbps for the case of two NSX edges.
- e BGP peering is used between NSX edges and physical firewall
- i BGP peering is used between NSX edges and Distributed Logical Router.

 
Keeping this SDDC topology in mind, we will now move on to the integration between Palo Alto Networks VM series firewall and VMware NSX-V.


Content filtering techniques of Palo Alto Networks firewall has been covered earlier.
Integration between NSX and Palo Alto Networks VM series firewall brings about benefits like Application ID, vulnerability protection, malware protection, sand boxing and protection from zero day threats, threat analysis, security policies based on user ID, SSL decryption.

Steps for such an integration are:
1. Integrate Panorama with NSX manager.
One Panorama can possibly integrate with multiple NSX managers. And there is no 1:1 relationship between Panorama and NSX Manager.
2. Once the Palo Alto Networks service is available in NSX, prepare the clusters for this service.
Once the clusters are prepared, each host will have a Palo Alto Networks service VM.
Palo Alto Networks service VMs require distributed port groups for networking and data stores for storage.
3. Create security groups in NSX based on static or dynamic inclusion criteria.
5. Leverage these security groups and create redirection policies in NSX. 
This is where you redirect traffic from the VM towards Palo Alto Networks service VM sitting on host where VM resides.

You can also possibly leverage these security groups in Panorama security policies.
There is 1:1 relationship between security groups of NSX and dynamic address groups in Panorama.
Members in Palo Alto Networks dynamic address groups are populated dynamically based on security group criteria of VMware NSX.

====================================

A bit about security groups in NSX-V

In NSX-V, you are able to create security groups based on static or dynamic criteria.
Security groups can be created from Service Composer in NSX-V
Static inclusion criteria for NSX 6.4.x can contain the following:
  1. Other security groups to nest within the security group you are creating.
  2. Cluster
  3. Logical switch
  4. Virtual App
  5. Datacenter
  6. IP sets
  7. MAC Sets
  8. Security tag
  9. vNIC
  10. Virtual Machine
  11. Resource Pool
  12. Distributed Virtual Port Group
Security group membership can change constantly.

Dynamic inclusion criteria while creating security group can be based on VM name, security tag, computer OS name.


=====================================================================

Next we will cover Cross vcenter NSX along with Palo Alto Networks VM series firewall.

Cross Vcenter NSX-V using BGP has been covered in this blog.
A cross vcenter NSX environment has:
- Two vcenters
- Two NSX managers
- Controller cluster is Universal Controller Cluster
- Primary and Secondary NSX manager.
- Universal Transport Zone 
- Universal Logical Switches
- Universal Distributed Logical Router
- Universal Distributed Firewall

VMs in a Cross vcenter NSX topology are connected to the Universal Logical Switches

In a cross vcenter NSX environment, it is possible to redirect traffic flows towards Palo Alto Networks VM series for workloads that are under the primary vcenter.
This is shown in figure below.

Note: The below topology covers a requirement where there are multiple vcenters in the same site.
Traffic flows for VMs in primary vcenter are supported by Palo Alto Networks service insertion
 
===============================================================

Cross Vcenter flows are not supported by this Palo Alto Networks service insertion and for such cross vcenter traffic flows you can possibly leverage the NSX Universal Distributed Firewall UDFW.

The key point here is that with UDFW you will create security policies on NSX using Universal IP sets and Universal MAC sets.
You can possibly create security groups with static inclusion of Universal Logical Switch using Service Composer but the effective membership will show VMs from a single vcenter only.
 
Cross Vcenter traffic flows are not supported by Palo Alto service insertion
You can possibly use Cross Vcenter NSX along with Palo Alto Networks VM series firewall for topology where there is Active Data Center with one vcenter and a Disaster Recovery Site with another vcenter.

Monday, December 3, 2018

Cross Vcenter NSX with OSPF

References:

Cross Vcenter NSX Design Guide
 
Intra area routes versus inter area routes


 
In one of the earlier posts, we have covered running OSPF in NSX domain.

Here we try to cover running OSPF in a Cross Vcenter NSX setup.

In the above topology:
1. There are two NSX managers.
Primary NSX manager in site 1
And secondary NSX manager in site 2
This is a Active/Passive model.

2. One vcenter in site 1 and the other in site 2

3. Universal Distributed Logical Router UDLR is connected to edges in primary & secondary site using a single Transit logical switch.

4. There are two edges in site 1 and additional two edges in site 2

5. The edges are connected to physical routers as shown above.

6. The physical routers are OSPF Area Border Routers interfacing with OSPF backbone area 0.

7. Totally NSSA area configuration is applied in this setup.
The Area Border Routers are injecting a default route automatically into the NSX domain.

8. Only one Universal Transport Zone can be created to span both the sites.

9. Universal Distributed Logical Router UDLR spans both the sites and allows connecting workloads from both the sites.
The status of UDLR in primary site is 'deployed' while in the secondary site it is 'active'.

10. Universal Distributed Firewall spans both the sites.

11. Universal Controller Cluster is deployed in Site 1

12. Universal objects can be created only from the Primary NSX Manager.
If there is a need to deploy additional UDLR, then this has to be done from Primary NSX Manager only. In this case the UDLR will consume resources (compute, memory and storage) of Site 1

13. UDLR control VM will reside in Site 1 and will establish OSPF peering with NSX edges from both sites 1 and 2.

14. Site 1 is preferred for ingress/egress traffic because the OSPF cost is increased for links between Site 2 physical routers and Site 2 NSX edges.

More traffic flows are discussed in traffic flow section below.


 ===============================================

Let's map this NSX topology with Cisco routers and explore the routing tables on devices.




Subnet 192.168.1.0/24 is connected to router UDLR.

And we check the route lookup for 192.168.1.0 on routers:
1. Core2
2. Core1
3. Physical Router 1 of Site 2
4. Edges of site 1 and site 2

Note the route lookup for 192.168.1.0/24 on Site 2 router 1 prefers the route towards NSX edges in the same site, site 2.
This is despite increasing the OSPF cost on the links connecting Site 2 physical routers and Site 2 NSX edges.
This is because OSPF prefers intra area routes as compared to inter area routes.




Above output shows that OSPF metric is increased to 120 via Site 2 physical routers.





Route for 192.168.1.0/24 on Core2
 Next hop from Core 2 for subnet 192.168.1.0/24 is Core 1 because of lower OSPF cost between site 1 physical routers and Site 1 NSX edges.



Route for 192.168.1.0/24 on Core 1

Route for 192.168.1.0/24 on Site2 Router 1



Route for 192.168.1.0/24 on Site 1 Edge

Route for 192.168.1.0/24 on Site2 Edge

Default route on UDLR

==================================================

Traffic flows


UDLR follows the default route

If there are other services/customers connected to the physical routers of Site 2, then such traffic will not go via Site 1. Instead be routed towards local NSX edges in Site 2 because intra area routes are preferred as compared to inter area routes.



Ingress from Core WAN to App Tier LS