CCIE DC: FCoE NPV

Hi Guys!

Final blog post for tonight I have spent lots of time this weekend working super hard on my study!

So let's check out FCoE NPV, first a quick word of advice: Make sure when switching fibre channel modes (NPV vs FC Switching) make sure you completely write erase before you switch modes.

Without further adue, let's look at the config!

First thing's first, let's turn on the FCoE NPV Feature on the switch that is going to be on the actual switch performing NPV:

feature fcoe-npv

On the FC switch that will act as our core and WILL participate in the FC as a FC Forwarder, we need to turn on NPIV

feature npiv


As usual with FCoE, let's create our VSAN's and our VLAN's

vsan database
  vsan 10
!
vlan 10
  fcoe vsan 10
!

Next, we configure our ethernet interface that faces up towards the core, we can do this on both our core switch and our NPV Switch



interface Ethernet1/10
  switchport mode trunk
  switchport trunk allowed vlan 10,20
!

Next, let's configure our VFC Interface:


interface vfc1
  bind interface Ethernet1/10
  switchport mode NP
  switchport trunk mode on
  no shutdown


This is now ready on our NPV side, on our Core switch the config is only slightly diffirent:


interface vfc1
  bind interface Ethernet1/10
  switchport trunk allowed vsan 10
  no shutdown
As you can see on the upstream switch our core switch just has this configured as an F Port.


Now finally, we need to make each of them realise that the port is a member of the VSAN:

vsan database
  vsan 10 interface vfc1

Now we can check to see if the vfc interface is up on our NPV Switch:

switch# show int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is NP, trunk mode is on
    snmp link state traps are enabled
    Port mode is TNP
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)


Success! the next step is to get our server facing interface going, for the sake of brevity i will show all the configuration for this at once:

switch# show run int vfc2

interface vfc2
  bind interface Ethernet1/1
  switchport trunk mode on
  no shutdown
!

interface Ethernet1/1
  switchport mode trunk
  switchport trunk allowed vlan 10,20
  spanning-tree port type edge trunk
!

vsan database
  vsan 10 interface vfc2

Let's check the server facing interface:

switch# show int vfc2
vfc2 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/1
    Hardware is Ethernet
    Port WWN is 20:01:54:7f:ee:af:1c:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)
!

Success! Let's check the NPV flogi table which is very useful.


switch# show npv flogi-table
--------------------------------------------------------------------------------
SERVER                                                                  EXTERNAL INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFACE
--------------------------------------------------------------------------------
vfc2      10   0xd80002 20:00:a4:4c:11:13:8c:d1 10:00:a4:4c:11:13:8c:d1 vfc1

Very Useful output showing us that the port vfc2 is logged in and is using vfc1 for its external interface.


The show npv command is actually quite powerful:


switch# show npv ?
  external-interface-usage  Show external interface usage by server interfaces
  flogi-table               Show information about FLOGI sessions
  internal                  Show internal NPV information
  status                    Show NPV status
  traffic-map               Show information about Traffic Map


In particular, i found the following command VERY helpful:

in parti


switch# show npv internal events

1) Event:E_DEBUG, length:94, at 270505 usecs after Sun Apr 21 10:39:22 2013
    [538976288] E(10,vfc1) Received GMAL Response from core switch with Core Switch Inet Addr: 10.1.1.15



2) Event:E_DEBUG, length:252, at 270497 usecs after Sun Apr 21 10:39:22 2013
    [538976288] E(10,vfc1) npivp_get_ext_intf_fsm(1466): Is Core VF Capable: TRU
E, Is Phy Login Done: TRUE, Is Port Channel: FALSE RID: { Type: Ext-If-VSAN(8),
IfIndex: vfc1(0x1e000000), UCD: 0, VSAN: 10, PWWN: 00:00:00:00:00:00:00:00 }, St
ate NPIVP_EXT_IF_ST_UP
This, along with show npv internal errors is really helpful in troubleshooting upstream issues.


If for example the upstream switch was not enabled for NPIV, I would see the following output in this command:




11) Event:E_DEBUG, length:98, at 464269 usecs after Sun Apr 21 10:41:22 2013
    [112] E(10,vfc1) FC Upstream switch is not NPIV enabled. Bringing down the external interface: vfc1