Hi Guys
In this blog post we will be looking a little more at Nexus 1000V
This blog will be a little disjointed as this is really more for me to be honest, Sorry Guys!
OK, the SYSTEM VLAN is key, if the system VLAN is not specified on your vethernet that is used by your VMKernel, and your ethernet uplinks from the VEM, your going to have a very bad time.
Second of all, if you try and install Nexus 1000V, make sure you have three NIC's, even if your planning on using L3 mode (which is the simplest and my preferred method)
Here is a sample config for a Nexus 1000V in L3 Mode
VSM# show run
banner motd #Nexus 1000v Switch#
switchname VSM
vem 3
host vmware id 00000000-0000-0000-000e-ee000fdd002f
This VEM has already associated with the server, Note that the VEM _WON'T_ associate with the server until you have actually assigned an uplink and a VMKernel to the host itself.
port-profile type ethernet uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 1
no shutdown
system vlan 1
state enabled
Here is our uplink port, note the key which is that switchport access vlan 1 which is our management VLAN is set to system VLAN, this allows it to forward instantly even after the host has been rebooted , this is needed so the VEM can talk to the VSM on bootup.
port-profile type vethernet VMKernel
capability l3control vmware port-group
switchport mode access
switchport access vlan 1
no shutdown
system vlan 1
state enabled
The capability L3 Control is key here.
interface mgmt0
ip address 192.168.198.108/24
Your management address is used for the VSM communication to the VEM's for capability L3 Control, although you can change this.
interface Vethernet1
inherit port-profile VMKernel
description VMware VMkernel, vmk0
vmware dvport 64 dvswitch uuid "fa c0 25 50 67 04 48 67-1d 00 b3 c1 fa e4 cb e
d"
vmware vm mac 0025.B500.082F
interface Ethernet3/5
inherit port-profile uplink
svs-domain
domain id 8
control vlan 1
packet vlan 1
svs mode L3 interface mgmt0
The domain-id is important to make sure the hosts don't mix, the control and packet VLAN values are completely ignored as our host uses Capability l3
svs connection VMWARE
protocol vmware-vim
remote ip address 192.168.198.244 port 80
vmware dvs uuid "fa c0 25 50 67 04 48 67-1d 00 b3 c1 fa e4 cb ed" datacenter-name CCIE
max-ports 8192
connect
!
The last bit is fairly straight forward.
The main reason I wanted to do all of this, was to see how uplink vPC works with Nexus 1000V.
So, Let's super quickly go over our topology, basically I have two interfaces configured in UCS, that will be used for the uplinks, I actually have more than that, but for what we are doing this will do, they are vnic's 1 and 2. One goes to Fabric A, one goes to Fabric B.
Here is the relevant config:
port-profile type ethernet vPC-HM
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 200
channel-group auto mode on mac-pinning no shutdown
state enabled
port-profile type vethernet Server
vmware port-group
switchport access vlan 200
switchport mode access
no shutdown
state enabled
Pretty simple right so far? Ok, obviously i have assigned the server profile:
port-profile Server
Vethernet2
switchport access vlan 200
switchport mode access
switchport trunk native vlan 198
no shutdown
Vethernet3
switchport access vlan 200
switchport mode access
switchport trunk native vlan 198
no shutdown
Vethernet4
switchport access vlan 200
switchport mode access
switchport trunk native vlan 198
no shutdown
And now this VLAN has some MAC addresses learnt from each of the virtual hosts:
VSM# show mac address-table vlan 200
VLAN MAC Address Type Age Port Mod
---------+-----------------+-------+---------+------------------------------+---
200 0050.569c.60bf static 0 Veth3 3
200 0050.569c.60c0 static 0 Veth2 3
200 0050.569c.60c1 static 0 Veth4 3
Now, we can see if we look at each fabric interconnect, that some hosts are load-balanced one way, some hosts the other:
UCS1-B(nxos)# show mac address-table vlan 200
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 200 0050.569c.60c1 dynamic 0 F F Veth1486
200 0100.5e7f.fffa igmp 0 F F drop
UCS1-B(nxos)# exit
UCS1-B# connect nxos a
UCS1-A(nxos)# show mac address-table vlan 200
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 200 0050.569c.60bf dynamic 340 F F Veth1469
* 200 0050.569c.60c0 dynamic 0 F F Veth1469
All very simple and straight forward, just how you would expect it to behave, if I shut down one of the ports I will see it all flick over to the other VEM.
UCS1-A(nxos)# show mac address-table vlan 200
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 200 0050.569c.60bf dynamic 40 F F Veth1469
* 200 0050.569c.60c0 dynamic 70 F F Veth1469
* 200 0050.569c.60c1 dynamic 0 F F Veth1469
If you re-enable connectivity, it flips back over
UCS1-A(nxos)# show mac address-table vlan 200
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 200 0050.569c.60c0 dynamic 0 F F Veth1469
* 200 0050.569c.60c1 dynamic 0 F F Veth1469
200 0100.5e7f.fffa igmp 0 F F drop
No comments:
Post a Comment