Many things scare a newbie CCIE DC candidate, and one of those things that scared me was VPC+, VPC+ and Enhanced VPC?? i said to myself, How could it possibly get any more complicated?
Fortunately VPC+ (as well as eVPC which I have covered previously) is quite simple, here is a sample configuration
So first of all, you configure your fabricpath interfaces and VLAN like you always have:
vlan 10
mode fabricpath
interface Ethernet1/1
switchport mode fabricpath
Configure your VPC as normal with a special diffirence:
vpc domain 1
peer-keepalive destination 10.0.8.212
fabricpath switch-id 100!
The fabricpath switch-id command manually specifies a switch-id to be used and shared amongst the devices, this MUST MATCH on both vPC Peers
next you configure your peer link, and you must configure it as a fabricpath port:
interface Ethernet1/9 -- Member interface
switchport mode fabricpath
channel-group 1 mode active
interface port-channel1
switchport mode fabricpath
spanning-tree port type network
speed 10000
vpc peer-link
Peer Link shown above.
IMPORTANT NOTE: This also means that you can't use non-FabricPath VLAN's for your vPC member ports, they will fail to come up
Finally here is the VPC Member port:
interface port-channel2
switchport access vlan 10
spanning-tree port type edge
speed 1000
vpc 2
With this config everything shows as it should
N5K-p6-1(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
vPC+ switch id : 100
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
vPC fabricpath status : peer is reachable through fabricpath
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 10
vPC status
---------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans vPC+ Attrib
-- ---------- ------ ----------- ------ ------------ -----------
2 Po2 up success success 10 DF: Partial
If you try and add a non-fabricpath VLAN, like in our example VLAN 1 below, like with an FCoE VLAN it will allow it but it won't be etherchanneled, if you don't have a FabricPath enabled VLAN at all...
interface port-channel2
switchport access vlan 1 - (Not enabled for FabricPath this VLAN)
spanning-tree port type edge trunk
speed 1000
vpc 2
!
The Port channel will come up on the primary vPC, but the VPC will not come up:
N5K-p6-1(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
vPC+ switch id : 100
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
vPC fabricpath status : peer is reachable through fabricpath
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 10
vPC status
---------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans vPC+ Attrib
-- ---------- ------ ----------- ------ ------------ -----------
2 Po2 down success success - DF: Partial
This is because no non-fabricpath VLAN will go over that peerlink, because the peerlink is now a fabricpath port! (and CE VLAN's DON'T go over fabricpath ports!)
Fabric Path + MTU:
Did you know that Fabric Path itself, if you looked at it with a wire capturing utility, is NOT ethernet, it's ethernet encapsulated inside fabricpath, fabricpath adds an overhead of approx 15 bytes, but takes into account all of that for you, what it does NOT take into account however is jumbo MTU's, you need to specifically configure them if you want them/need them.
Here's proof:
N5K-p6-1(config-if)# show fabricpath isis interface brief
Fabricpath IS-IS domain: default
Interface Type Idx State Circuit MTU Metric Priority Adjs/AdjsUp
--------------------------------------------------------------------------------
port-channel1 P2P 1 Up/Ready 0x01/L1 1500 20 64 1/1
Ethernet1/1 P2P 2 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/2 P2P 3 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/3 P2P 4 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/4 P2P 5 Up/Ready 0x01/L1 1500 40 64 1/1
More proof? Check this out:
Here is all my relevant config:
interface Vlan10
no shutdown
mtu 9216
ip address 10.1.1.1/24
The adapter on the server has been set correctly also.
But if i configure it correctly on both ends
N7K-6-1(config-if)# show fabricpath isis interface brief
Fabricpath IS-IS domain: default
Interface Type Idx State Circuit MTU Metric Priority Adjs/AdjsUp
--------------------------------------------------------------------------------
Ethernet1/21 P2P 1 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/22 P2P 7 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/23 P2P 3 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/24 P2P 8 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/25 P2P 4 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/26 P2P 5 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/27 P2P 2 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/28 P2P 6 Up/Ready 0x01/L1 1500 40 64 1/1
N7K-6-1(config-if)# int eth1/21 - 28
N7K-6-1(config-if-range)# mtu ?
1500-9216 Enter MTU
N7K-6-1(config-if-range)# mtu 9216
And:
N5K-p6-2(config-if)# show fabricpath isis interface brief
Fabricpath IS-IS domain: default
Interface Type Idx State Circuit MTU Metric Priority Adjs/AdjsUp
--------------------------------------------------------------------------------
port-channel1 P2P 1 Up/Ready 0x01/L1 1500 20 64 1/1
Ethernet1/5 P2P 2 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/6 P2P 3 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/7 P2P 4 Up/Ready 0x01/L1 1500 40 64 1/1
Ethernet1/8 P2P 5 Up/Ready 0x01/L1 1500 40 64 1/1
So i changed the QoS policy on the N5k:
N5K-p6-1(config-pmap-nq)# show run | sect policy-map
policy-map type network-qos jumbomtu
class type network-qos class-fcoe
pause no-drop
mtu 2158
class type network-qos class-default
mtu 9216
multicast-optimize
N5K-p6-1(config)# system qos
N5K-p6-1(config-sys-qos)# service-policy type network-qos jumbomtu
Yet this did not seem to affect the output from the show fabricpath isis interface, and i couldn't set the MTU directly on the interface
however after doing both these steps, i could ping across at the maximum MTU of 8972
C:\Documents and Settings\student>ping 10.1.1.1 -l 8972 -f
Pinging 10.1.1.1 with 8972 bytes of data:
Reply from 10.1.1.1: bytes=8972 time=1ms TTL=255
Reply from 10.1.1.1: bytes=8972 time=1ms TTL=255
Reply from 10.1.1.1: bytes=8972 time=1ms TTL=255
Ping statistics for 10.1.1.1:
So interestingly, you MUST set this on the fabricpath interface on your 7k:
Check out this ping when i took it off:
N7K-6-1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N7K-6-1(config)# int eth1/21 - 28
N7K-6-1(config-if-range)# no mtu
C:\Documents and Settings\student>ping 10.1.1.1 -l 8972 -f
Pinging 10.1.1.1 with 8972 bytes of data:
Request Timed out.
When i enabled it again:
C:\Documents and Settings\student>ping 10.1.1.1 -l 8972 -f
Pinging 10.1.1.1 with 8972 bytes of data:
Reply from 10.1.1.1: bytes=8972 time=1ms TTL=255
Reply from 10.1.1.1: bytes=8972 time=1ms TTL=255
So these appear to be the steps you need to take, but I will get into advanced MTU a bit later, just know that if you want to enable it with fabricpath, you gotta specify it on a per-interface basis on your 7k, but on a 5k it just needs to be in your class-default QoS Policy (or whatever traffic class your traffic is falling under)
Peter, I am wondeing why you use "spanning-tree port type network" int the fabricpath enabled vpc peer-link.
ReplyDeleteIt's there by default after configuring the Peer Link, but you could easily remove it and it would have no effect.
DeleteWhen you configure the port-channel with "vpc peer-link", it will automatically enable BA on the link ("spanning-tree port type network"), but once you configure "switchport mode fabricpath" on the Peer-Link to enable vPC+, BA is rendered useless as, like everyone knows, FabricPath is NOT Ethernet and does NOT us STP.
it's there by "default" but most people verify that it is there. It provides bridge assurance.
ReplyDelete