Spent today on IPExpert's lab looking at Fabric Path in great detail, the great thing about fabricpath for end users is that it's a peice of cake to configure, the bad thing about Fabric Path for us CCIE DC candidates is that it's actually bloody complicated to do some of the more advanced config ;).
Most of what I am talking about in this article I learnt from reading:
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-687554.html#wp9000328
Which is a very good whitepaper on fabricpath.
Let's take a look at a few advanced config options
UPDATE UPDATE UPDATE UPDATE UPDATE (Mostly for you peter)
Here is a great way to tell the root of the fabric-path tree's
N5K-p1-1(config)# show fabricpath isis topology summary
Fabricpath IS-IS domain: default FabricPath IS-IS Topology
Summary
MT-0
Configured interfaces: Ethernet1/1
Ethernet1/2 Ethernet1/3 Ethernet1/4 Ethernet1/
5 Ethernet1/6 Ethernet1/7 Ethernet1/8
port-channel5
Number of trees: 2
Tree id: 1, ftag: 1
[transit-traffic-only], root system: 0024.98e8.01c2, 1709
Tree id: 2, ftag: 2, root system:
001b.54c2.67c2, 2040
Remember with ISIS there are two authentication methods, the actual hello adjacency authentication, and the LSP data-plane authentication, here is a sample config of both of these:
key chain cisco
key 0
key-string 7 070c705f4d59
!
interface Ethernet1/16
switchport mode fabricpath
fabricpath isis authentication-type md5
fabricpath isis authentication key-chain cisco
!
The config as you can see above is quite simple, don't forget with key chains you can specify a accept lifetime and send lifetime, but for our case we are not going to, when you don't specify this it is simply assumed to be infinite:
SW2# show key chain
Key-Chain cisco
Key 0 -- text 7 070c705f4d59
accept lifetime (always valid) [active]
send lifetime (always valid) [active]
You can verify your ISIS authentication:
SW2# show fabricpath isis interface eth1/16
Fabricpath IS-IS domain: default
Interface: Ethernet1/16
Status: protocol-up/link-up/admin-up
Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1
Authentication type MD5
Authentication keychain is cisco
Authentication check specified Extended Local Circuit ID: 0x1A00F000, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 1500
P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:06
Level Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID
1 1 1 40 60 00:00:35 ffff.ffff.ffff.ff-ff
Topologies enabled:
Topology Metric MetricConfig Forwarding
0 40 no UP
Next if you want to actually configure the LSP's to be authenticated:
fabricpath domain default
authentication-type md5
authentication key-chain cisco
You can then verify this is configured:
SW2# show fabricpath isis
Fabricpath IS-IS domain : default
System ID : 547f.eec2.7d01 IS-Type : L1
SAP : 432 Queue Handle : 10
Maximum LSP MTU: 1492
Graceful Restart enabled. State: Inactive
Last graceful restart status : none
Metric-style : advertise(wide), accept(wide)
Start-Mode: Complete [Start-type configuration]
Area address(es) :
00
Process is up and running
CIB ID: 3
Interfaces supported by Fabricpath IS-IS :
Ethernet1/5
Ethernet1/6
Ethernet1/7
Ethernet1/8
Ethernet1/16
Level 1
Authentication type: MD5
Authentication keychain: cisco Authentication check specified MT-0 Ref-Bw: 400000
Address family Swid unicast :
Number of interface : 5
Distance : 115
L1 Next SPF: Inactive
As per Ron Fuller's blog post, a big hint that your auth is working for hello but not for LSP is that the hostnames don't come up correctly in your isis adjacency
http://ccie5851.blogspot.com.au/2012/09/fabricpath-authentication-in-nx-os.html
So next let's look at another advanced aspect of fabricpath which is load balancing.
Ok first of all it helps if we establish a few items of terminology. The first thing to remember is that fabricpath supports multiple topologies so that you can actually break out particular FabPath enabled VLAN's to use a particular topology. However this is only available in certain versions of NXOS and is quite advanced, so we will be skipping this advanced configuration.
However, the concept of "Trees" in fabricpath also exists, tree's are used for the distribution of "multidestination" traffic, that is traffic that is not a single destination, so perfect examples of this would be multicast, unknown unicast and other flooding types.
The first multidestination tree, tree 1 is normally selected for unknown unicast and broadcast frames except when used in combination with vpc+, but the detail of that we will ignore for now.
Multicast traffic is load balanced based on a hashing function (which is based on the source and dest IP address) across both the trees, you can see what kind of tree the traffic is going to take on a nexus 7000 with the following command:
N7K1# show fabricpath load-balance multicast ftag-selected flow-type l3 src-ip 10.1.1.1 dst-ip 10.1.1.33 vlan 666 module 4
128b Hash Key generated : 00 00 02 9a 00 00 00 00 00 00 00 a0 10 12 10 00
0x1b
FTAG SELECTED IS : 2
N7K1# show fabricpath load-balance multicast ftag-selected flow-type l3 src-ip 10.1.1.1 dst-ip 10.1.1.34 vlan 666 module 4
128b Hash Key generated : 00 00 02 9a 00 00 00 00 00 00 00 a0 10 12 20 00
0xda
FTAG SELECTED IS : 1
The FTAG is an important key here, the FTAG will correlate to the "Tree". The FTAG is used as it's an available field in the FabricPath Header that can be used to identify the frame and tell the switches "use this tree to distribute the traffic".
Now the whole point of this option is for scalability, especially with large multicast traffic domains, using this option you can increase link utilization for multicast traffic by having the traffic load balance across two "root" trees (yes, this is fabric path, so we don't really have a root tree like we do in spanning-tree, but for multidestination traffic we kind of have to ;))
You can actually tell using the following command what port your switch is going to use for that particular FTAG/MTREE:
SW2# show fabricpath mroute ftag 2
(ftag/2, vlan/666, *, *), Flood, uptime: 04:56:33, isis
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:26:19, isis
Interface Ethernet1/8, uptime: 00:26:19, isis
(ftag/2, vlan/666, *, *), Router ports (OMF), uptime: 01:48:38, isis igmp
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:26:19, isis
Interface Vlan666, [SVI] uptime: 01:48:38, igmp
Found total 2 route(s)
SW2# show fabricpath mroute ftag 1
(ftag/1, vlan/666, *, *), Flood, uptime: 04:56:40, isis
Outgoing interface list: (count: 2)
Interface Ethernet1/8, uptime: 00:26:26, isis
Interface Ethernet1/8, uptime: 00:26:26, isis
(ftag/1, vlan/666, *, *), Router ports (OMF), uptime: 01:48:45, isis igmp
Outgoing interface list: (count: 2)
Interface Ethernet1/8, uptime: 00:26:26, isis
Interface Vlan666, [SVI] uptime: 01:48:45, igmp
Found total 2 route(s)
As you can see from the above, there are two seperate paths that the switch is taking for each of the
Trees based on where the root of the tree lies
So how is the root of each tree chosen? It's based on:
- root-Priority (highest wins, default is 64)
- Switch-id (highest wins, default is randomally chosen but can be manually assigned)
- System-id (Tie-breaker)
There will always be two seperate roots for each tree, but as you can imagine, your root tree might not be the most optimally chosen tree, so you can configure the root priority, the highest root priority will become the root for FTAG 1, and second place will become the root tree for FTAG 2:
N7K1(config)# fabricpath domain default
N7K1(config-fabricpath-isis)# root-priority 254
N71k is now the root for this tree, you can attempt to verify this in a few ways, the first is to look at the show fabricpath mroute ftag 1 command we used previously, let's just quickly get our topology clear:
SW3# show fabricpath isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
SW2 N/A 1 UP 00:00:23 Ethernet1/5
SW2 N/A 1 UP 00:00:27 Ethernet1/6
SW2 N/A 1 UP 00:00:30 Ethernet1/7
SW2 N/A 1 UP 00:00:30 Ethernet1/8
N7K1 N/A 1 UP 00:00:29 Ethernet1/17
As you can see from the above, we have multiple connections between SW3 to SW2, and then a single connection from SW2 and SW3 up to N7K1
SW2# show fabricpath isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
SW3 N/A 1 UP 00:00:30 Ethernet1/5
SW3 N/A 1 UP 00:00:21 Ethernet1/6
SW3 N/A 1 UP 00:00:29 Ethernet1/7
SW3 N/A 1 UP 00:00:27 Ethernet1/8
N7K1 N/A 1 UP 00:00:24 Ethernet1/16
Let's check out our mroute routing:
SW2# show fabricpath mroute ftag 1
(ftag/1, vlan/666, *, *), Flood, uptime: 05:12:43, isis
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:07:13, isis
Interface Ethernet1/16, uptime: 00:07:13, isis
(ftag/1, vlan/666, *, *), Router ports (OMF), uptime: 02:04:48, isis igmp
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:07:13, isis
Interface Vlan666, [SVI] uptime: 02:04:48, igmp
SW2# show fabricpath mroute ftag 2
(ftag/2, vlan/666, *, *), Flood, uptime: 05:13:35, isis
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:07:09, isis
Interface Ethernet1/8, uptime: 00:43:21, isis
(ftag/2, vlan/666, *, *), Router ports (OMF), uptime: 02:05:39, isis igmp
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:07:09, isis
Interface Vlan666, [SVI] uptime: 02:05:39, igmp
You can tell from the above, neither of the switches will ever send unknown unicast (which remember, is placed into FTAG 1) out to each other but will instead always forward it up to the tree, up to N71k, which is our root for this tree.
From N7k's Perspective:
N7K1# show fabricpath mroute ftag 1
(ftag/1, vlan/666, *, *), Flood, uptime: 04:01:00, isis
Outgoing interface list: (count: 2)
Interface Ethernet4/1, uptime: 00:01:51, isis
Interface Ethernet4/2, uptime: 01:06:04, isis
(ftag/1, vlan/666, *, *), Router ports (OMF), uptime: 04:01:01, isis igmp
Outgoing interface list: (count: 2)
Interface Vlan666, [SVI] uptime: 01:51:33, igmp
Interface Ethernet4/1, uptime: 00:01:51, isis
He is responsible for forwarding it back down, so if an unknown unicast or a multicast frame that was hashed to FTAG 1 comes from SW2, it will go up to N7k1 and then back down towards SW3 through N7K1.
Let's manually configure switch 2 to be the root for FTAG 2 by manually configuring SW3 to have a lower priority:
SW3# show run | sect fabricpath
fabricpath domain default
root-priority 63
Let's take a look at the FTAG distribution now:
SW2# show fabricpath mroute ftag 2
(ftag/2, vlan/666, *, *), Flood, uptime: 05:18:03, isis
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:11:38, isis
Interface Ethernet1/8, uptime: 00:47:50, isis
(ftag/2, vlan/666, *, *), Router ports (OMF), uptime: 02:10:08, isis igmp
Outgoing interface list: (count: 2)
Interface Ethernet1/16, uptime: 00:11:38, isis
Interface Vlan666, [SVI] uptime: 02:10:08, igmp
Found total 2 route(s)
Let's check it out on the n71k:
N7K1# show fabricpath mroute ftag 2
(ftag/2, vlan/666, *, *), Flood, uptime: 04:12:32, isis
Outgoing interface list: (count: 2)
Interface Ethernet4/1, uptime: 00:12:28, isis
Interface Ethernet4/1, uptime: 00:12:28, isis
(ftag/2, vlan/666, *, *), Router ports (OMF), uptime: 04:12:33, isis igmp
Outgoing interface list: (count: 2)
Interface Vlan666, [SVI] uptime: 02:03:05, igmp
Interface Ethernet4/1, uptime: 00:12:28, isis
Found total 2 route(s)
N7K1# show fabricpath isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
SW2 N/A 1 UP 00:00:27 Ethernet4/1
Ok so now SW2 is the root for FTAG 2 and any frames from N71k will come down to him first, and he in turn will distribute it to SW3, now there is one bit of that config that might make you say "What Gives?" and that is, I have four connections between SW2 and SW3, why is traffic not load balancing across those Equal Cost Links?
Fabric Path only ECMP's for KNOWN unicast frames.
OK, here's one more way you can use to determine the root of a MTREE:
N7K1# show fabricpath isis trees
Fabricpath IS-IS domain: default
Note: The metric mentioned for multidestination tree is from the root of that tr
ee to that switch-id
MT-0
Topology 0, Tree 1, Swid routing table
2, L1
via Ethernet4/1, metric 40
3, L1
via Ethernet4/2, metric 40
Topology 0, Tree 2, Swid routing table
2, L1
via Ethernet4/1, metric 0
3, L1
via Ethernet4/1, metric 40
SW2# show fabricpath isis trees
Fabricpath IS-IS domain: default
Note: The metric mentioned for multidestination tree is from the root of that tr
ee to that switch-id
MT-0
Topology 0, Tree 1, Swid routing table
1, L1
via Ethernet1/16, metric 0
3, L1
via Ethernet1/16, metric 40
Topology 0, Tree 2, Swid routing table
1, L1
via Ethernet1/16, metric 40
3, L1
via Ethernet1/8, metric 40
SW3# show fabricpath isis trees
Fabricpath IS-IS domain: default
Note: The metric mentioned for multidestination tree is from the root of that tr
ee to that switch-id
MT-0
Topology 0, Tree 1, Swid routing table
1, L1
via Ethernet1/17, metric 0
2, L1
via Ethernet1/17, metric 40
Topology 0, Tree 2, Swid routing table
1, L1
via Ethernet1/8, metric 40
2, L1
via Ethernet1/8, metric 0
So the key point in this output I Have highlighted:
"Note: The metric mentioned for multidestination tree is from the root of that tr
ee to that switch-id"
What this is saying is that when your looking at this output, your being told the values for the topology tree as if you where running the command on the root of each tree itself, So if we take a closer look at a switch, Switch 3, which is not the root for either FTAG:
SW3# show fabricpath isis trees
Fabricpath IS-IS domain: default
Note: The metric mentioned for multidestination tree is from the root of that tr
ee to that switch-id
MT-0
Topology 0, Tree 1, Swid routing table
1, L1
via Ethernet1/17, metric 0
2, L1
via Ethernet1/17, metric 40
Topology 0, Tree 2, Swid routing table
1, L1
via Ethernet1/8, metric 40
2, L1
via Ethernet1/8, metric 0
The Metric for reaching Switch-ID 1, which this switch reaches via Eth1/17, is metric 0... Because Switch 1 _is_ the root for this FTAG
Same again for Tree 2, the root of the tree is Switch-ID 2, which is out eth1/8, which has a metric of 0, because obviously for Switch-ID 2, it's metric to reach itself, would be 0.
Let's now look at unicast load balancing
So if we look at our default unicast load balancing table right now on our switches with multiple, equal cost links (Remember, fabricpath only supports load balancing across equal cost links)
SW3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/3/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 01:43:35, local
1/1/0, number of next-hops: 1
via Eth1/17, [115/40], 0 day/s 01:43:44, isis_fabricpath-default
1/2/0, number of next-hops: 4
via Eth1/5, [115/40], 0 day/s 01:07:59, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 01:08:06, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 01:08:05, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 01:07:55, isis_fabricpath-default
We can see that our links are being equally balanced, how are they balanced?
SW3# show fabricpath load-balance
ECMP load-balancing configuration:
L3/L4 Preference: Mixed
Hash Control: Symmetric
Use VLAN: TRUE
They are load balanced based on a combination of values as shown above, these include:
• layer-3: Include only Layer 3 input (source or destination IP address).
• layer-4: Include only Layer 4 input (source or destination TCP and UDP ports, if available).
• mixed: Include both Layer 3 and Layer 4 input (default).
• source: Use only source parameters (layer-3, layer-4, or mixed).
• destination: Use only destination parameters (layer-3, layer-4, or mixed).
• source-destination: Use both source and destination parameters (layer-3, layer-4, or mixed).
•
symmetric: Sort the source and destination tuples before entering them
in the hash function (source-to-destination and destination-to-source
flows hash identically) (default).
• xor: Perform an exclusive OR operation on the source and destination tuples before entering them in the hash function.
• include-vlan: Include the VLAN ID of the frame (default).
• rotate-amount: Specify the number of bytes to rotate the hash string before it is entered in the hash function.
Each of these values is relatively straight forward, you can specify if you want to look at the layer 3 or layer 4 source/dest info OR a mixture (which is the defualt), you can specify that you only want to look at the source or destination OR mixed, you can control if the hash function will produce the same value for both source-dest traffic and the return dest-source traffic. Finally the VLAN ID Can be included in your combinations, last but not least the rotate-amount controls some of the mathematics of the hash function that we will get into.
Let's use our favorite command to look at this closely
SW2# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 3 src-mac 0000.1111.2222 dst-mac 1111.2222.3333 l4-dst-port 80 dst-ip 198.18.66.1 src-ip 198.18.66.4 l4-src-port 8080 vlan 666
Missing params will be substituted by 0's.
crc8_hash: 134
This flow selects interface Eth1/7
Missing params will be substituted by 0's.
crc8_hash: 134
This flow selects interface Eth1/7
SW2# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 3 src-mac 0000.1111.2222 dst-mac 1111.2222.3333 l4-dst-port 80 dst-ip 198.18.66.1 src-ip 198.18.66.4 l4-src-port 8081 vlan 666
Missing params will be substituted by 0's.
crc8_hash: 29
This flow selects interface Eth1/6
Missing params will be substituted by 0's.
crc8_hash: 29
This flow selects interface Eth1/6
We can see that we only changed one tiny param the port number and all of a sudden the traffic will load balance across another link, great! Looks pretty good so far right?
Let's check out what that symetric command does for us, check this out:
SW2# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 3 src-mac 1111.2222.3333 dst-mac 0000.1111.2222 l4-dst-port 8080 dst-ip 198.18.66.4 src-ip 198.18.66.1 l4-src-port 80 vlan 666
Missing params will be substituted by 0's.
crc8_hash: 134
This flow selects interface Eth1/7
Missing params will be substituted by 0's.
crc8_hash: 134
This flow selects interface Eth1/7
Here we have changed the source and destination ports and ip addressing etc around, and we are provided with exactly the same CRC hash, which leads us to exactly the same output interface!
Let's see if that is also true on the N7k:
N7K1# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 80 dst-ip 198.18.66.1 src-ip 198.18.66.4 l4-src-port 8080 vlan 666 module 4
128b Hash Key generated : 01f9029a00000c6124201c6124204005
This flow selects interface Eth4/1
N7K1#
N7K1# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 8080 dst-ip 198.18.66.4 src-ip 198.18.66.1 l4-src-port 80 vlan 666 module 4
128b Hash Key generated : 01f9029a00000c6124201c6124204005
This flow selects interface Eth4/1
128b Hash Key generated : 01f9029a00000c6124201c6124204005
This flow selects interface Eth4/1
N7K1#
N7K1# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 8080 dst-ip 198.18.66.4 src-ip 198.18.66.1 l4-src-port 80 vlan 666 module 4
128b Hash Key generated : 01f9029a00000c6124201c6124204005
This flow selects interface Eth4/1
If we change the length that the hash key is based on, the rotate amount, our hash key will change:
N7K1# show fabricpath load-balance
ECMP load-balancing configuration:
L3/L4 Preference: Mixed
Hash Control: Symmetric
Rotate amount: 15 bytes
ECMP load-balancing configuration:
L3/L4 Preference: Mixed
Hash Control: Symmetric
Rotate amount: 15 bytes
N7K1# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 80 dst-ip 198.18.66.1 src-ip 198.18.66.4 l4-src-port 8080 vlan 666 module 4
128b Hash Key generated : 0000000c6124201c612420400501f900
So now we have a diffirent hash key generated, based on a longer rotate-amount.
This is apparently simply used to make sure that identical or near identical traffic flows using VDC's disripute the traffic diffirently to each other, it simply adds a longer hash value (in this case, it takes a number of bytes from the VDC Mac address) to increase the likelyhood that the hash will differ between the VDC's,
Check this out for size:
Two totally separate VDC's are shown here, and what we do is change the rotate-amount on each of them to 0 (nothing), then ask us to show it what it thinks the hash key is:
sw1-2(config)# fabricpath load-balance unicast rotate-amount 0x0
sw1-2# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 8080 dst-ip 198.18.66.4 src-ip 198.18.66.1 l4-src-port 80 vlan 666 module 4
128b Hash Key generated : 00000c6124201c612420400501f90000
N7K1# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 2 flow-type l4 l4-dst-port 8080 dst-ip 198.18.66.4 src-ip 198.18.66.1 l4-src-port 80 vlan 666 module 4
128b Hash Key generated : 00000c6124201c612420400501f90000
128b Hash Key generated : 00000c6124201c612420400501f90000
As you can see, the hash is identical, which means our traffic would flow over the same paths between these VDC's which we may not want, so we can use the rotate-amount to increase how much of the VDC-MAC address is used in the hashing function.
Now let's talk about one more thing, which is: Just because FabricPath only supports equal cost load balancing, doesn't mean that we can't go through intermediate switches and still have load balancing, here is an example of this:
SW2# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
1/3/0, number of next-hops: 5
via Eth1/5, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/16, [115/40], 0 day/s 00:00:26, isis_fabricpath-default
via Eth1/5, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 00:00:02, isis_fabricpath-default
via Eth1/16, [115/40], 0 day/s 00:00:26, isis_fabricpath-default
In the above example, we have modified the metric on N71k so that SW1 and SW2, which have interfaces eth1/5 - 8 to each ohter, also see the route via N71k as a valid path between each other two, we did this by modifying the metrics like so:
SW2# show run int eth1/16
interface Ethernet1/16
switchport mode fabricpath
fabricpath isis metric 25
interface Ethernet1/16
switchport mode fabricpath
fabricpath isis metric 25
N7K1(config)# int eth4/1
N7K1(config-if)# fabricpath isis metric 15
N7K1(config-if)# int eth4/2
N7K1(config-if)# fabricpath isis metric 15
N7K1(config-if)# fabricpath isis metric 15
N7K1(config-if)# int eth4/2
N7K1(config-if)# fabricpath isis metric 15
Notice that the total cost of these links is now 40 (25 + 15) for SW2, which means SW2 now considers it an alternative Path
Over on SW3, since we have not modified the default metric, it will still load balance via the 4 links, not 5.
SW3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/3/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 02:30:59, local
1/1/0, number of next-hops: 1
via Eth1/17, [115/40], 0 day/s 02:31:08, isis_fabricpath-default
1/2/0, number of next-hops: 4
via Eth1/5, [115/40], 0 day/s 01:55:23, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 01:55:30, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 01:55:29, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 01:55:19, isis_fabricpath-default
That is, until we change the metric:
SW3(config)# int eth1/17
SW3(config-if)# fabricpath isis metric 25
SW3(config-if)# end
SW3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/3/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 02:31:34, local
1/1/0, number of next-hops: 1
via Eth1/17, [115/25], 0 day/s 02:31:43, isis_fabricpath-default
1/2/0, number of next-hops: 5
via Eth1/5, [115/40], 0 day/s 01:55:58, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 01:56:05, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 01:56:04, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 01:55:54, isis_fabricpath-default
via Eth1/17, [115/40], 0 day/s 00:00:03, isis_fabricpath-default
SW3(config-if)# fabricpath isis metric 25
SW3(config-if)# end
SW3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/3/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 02:31:34, local
1/1/0, number of next-hops: 1
via Eth1/17, [115/25], 0 day/s 02:31:43, isis_fabricpath-default
1/2/0, number of next-hops: 5
via Eth1/5, [115/40], 0 day/s 01:55:58, isis_fabricpath-default
via Eth1/6, [115/40], 0 day/s 01:56:05, isis_fabricpath-default
via Eth1/7, [115/40], 0 day/s 01:56:04, isis_fabricpath-default
via Eth1/8, [115/40], 0 day/s 01:55:54, isis_fabricpath-default
via Eth1/17, [115/40], 0 day/s 00:00:03, isis_fabricpath-default
I hope you found this post useful, I have tried very hard to be extremely thorough, this was quite a long day of study on FabricPath!
Great post Peter. Keep them coming
ReplyDeleteVery helpful, as usual, Peter!
ReplyDeleteThank you!
Another very useful and well explained post, thanks a lot!
ReplyDeleteI don't think I have gone through all of those looong show fabricpath load-balance commands myself. Great job.
ReplyDeleteRegarding "The first multidestination tree, tree 1 is normally selected for unknown unicast and broadcast frames except when used in combination with vpc+, but the detail of that we will ignore for now.",
ReplyDeleteWould you point any document which explains how MDT relates to vPC+?
Hello,
ReplyDeleteYou have mistake in the FP root selection order. The correct one is:
1. Highest root priority – 8-bit value between 0-255 (Default is 64)
2. Highest System-ID – 48-bit VDC MAC address
3. Highest Switch-ID – 12-bit SID
The switch id is taken into consideration after the system-id
Peter, your blog is excellent and helped me greatly on my CCIE DC journey two years ago. I'm studying to recert and I came across some interesting information regarding your note about "The first multidestination tree, tree 1 is normally selected for unknown unicast and broadcast frames". It turns out this behavior is no longer valid, and may have only ever been valid on F1 modules. Per https://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-687554.html:
ReplyDeleteIn brief, FabricPath edge switches use a hash function to select a multidestination tree to forward unknown unicast frames (note that F1 I/O modules always use Tree 1 for unknown unicast, except in the case of VPC+).
In brief, FabricPath edge switches use a hash function to select a multidestination tree to forward broadcast frames (note that F1 I/O modules always use Tree 1 for broadcast, except in the case of VPC+).
This was news to me and wanted to share for all those who use your blog for DC studies. All the best,
Jeff
Trung tâm đào tạo kế toán thực hành Tại cầu giấy
ReplyDeleteTrung tâm đào tạo kế toán thực hành Tại từ liêm
Trung tâm đào tạo kế toán thực hành Tại thanh xuân
Trung tâm đào tạo kế toán thực hành Tại hà đông
Trung tâm đào tạo kế toán thực hành Tại long biên
Trung tâm đào tạo kế toán thực hành Tại nguyễn chính thanh đống đa
Trung tâm đào tạo kế toán thực hành Tại minh khai hai bà trưng
Trung tâm đào tạo kế toán thực hành Tại bắc ninh
Trung tâm đào tạo kế toán thực hành Tại hải phòng
Trung tâm đào tạo kế toán thực hành Tại tphcm
Trung tâm đào tạo kế toán thực hành Tại quận 3
Trung tâm đào tạo kế toán thực hành Tại thủ đức
Trung tâm đào tạo kế toán thực hành Tại đà nẵng
Trung tâm đào tạo kế toán thực hành Tại biên hòa
Trung tâm đào tạo kế toán thực hành Tại đồng nai
Trung tâm đào tạo kế toán thực hành Tại nam định
Trung tâm đào tạo kế toán thực hành Tại thái bình
Trung tâm đào tạo kế toán thực hành Tại bắc giang
Trung tâm đào tạo kế toán thực hành Tại vĩnh phúc
Trung tâm đào tạo kế toán thực hành Tại thái nguyên
Trung tâm đào tạo kế toán thực hành Tại quảng ninh
Trung tâm đào tạo kế toán thực hành Tại hải dương
Trung tâm đào tạo kế toán thực hành Tại hưng yên
Trung tâm đào tạo kế toán thực hành Tại hà nam
Trung tâm đào tạo kế toán thực hành Tại ninh bình
Trung tâm đào tạo kế toán thực hành Tại nghệ an
Trung tâm đào tạo kế toán thực hành Tại vũng tàu
dịch vụ thành lập doanh nghiệp