Not all ports are created equal on Nexus5000

With Nexus 5000, it is very common to select the first few available ports for basic infrastructure such as peer links and uplinks. But that may not be best practice for resiliency, due to shared use of ASIC on the switch.

In short, there are 5 ASICs. Port e1/1-4 share the first ASIC, for example. If the first 4 physical port on Nexus 5000 are selected for all peer link and uplink to Nexus 7000, then loss of ASIC0 will bring down all these links at the same time. Even with the protection of VPC which can minimize loss, still not a desirable scenario.

Use “show hardware internal gatos all-ports” to see port to ASIC mapping. Below shows "gat" 0 which is the ASIC number for e1/1-4. It will also show the next four ports show the next ASIC, so on so forth.

Nexus-5010-sw1# show hardware internal gatos all-ports
process_cli_req:
Calling Hander:
Gatos Port Info:
name |log|gat|mac|flag|adm|opr|c:m:s:l|ipt|fab|xgat|xpt|if_index|diag
-------+---+---+---+----+---+---+-------+---+---+----+---+--------+----
xgb1/4 |3 |0 |0 |b7 |en |up |0:0:0:f|0 |55 |0 |2 |1a003000|pass
xgb1/3 |2 |0 |1 |b7 |en |up |0:1:1:f|1 |54 |0 |0 |1a002000|pass
xgb1/1 |0 |0 |2 |b7 |en |up |1:2:2:f|2 |56 |0 |4 |1a000000|pass
xgb1/2 |1 |0 |3 |b7 |en |up |1:3:3:f|3 |57 |0 |6 |1a001000|pass
xgb1/8 |7 |1 |0 |b7 |en |up |0:0:0:f|0 |50 |1 |2 |1a007000|pass
xgb1/7 |6 |1 |1 |b7 |en |up |0:1:1:f|1 |51 |1 |0 |1a006000|pass
xgb1/5 |4 |1 |2 |b7 |en |up |1:2:2:f|2 |53 |1 |4 |1a004000|pass
xgb1/6 |5 |1 |3 |b7 |en |up |1:3:3:f|3 |52 |1 |6 |1a005000|pass
xgb1/19|18 |2 |0 |b7 |en |up |0:0:0:f|0 |45 |4 |0 |1a012000|pass
xgb1/11|10 |2 |1 |b7 |dis|dn |0:1:1:f|1 |44 |2 |0 |1a00a000|pass
xgb1/20|19 |2 |2 |b7 |en |up |1:2:2:f|2 |49 |4 |2 |1a013000|pass
xgb1/12|11 |2 |3 |b7 |dis|dn |1:3:3:f|3 |48 |2 |2 |1a00b000|pass
xgb1/18|17 |3 |0 |b7 |en |up |0:0:0:f|0 |41 |4 |6 |1a011000|pass
xgb1/10|9 |3 |1 |b7 |en |up |0:1:1:f|1 |40 |2 |6 |1a009000|pass
xgb1/17|16 |3 |2 |b7 |en |up |1:2:2:f|2 |46 |4 |4 |1a010000|pass
xgb1/9 |8 |3 |3 |b7 |en |up |1:3:3:f|3 |47 |2 |4 |1a008000|pass
xgb1/15|14 |4 |1 |b7 |en |up |0:1:1:f|1 |36 |3 |0 |1a00e000|pass
xgb1/16|15 |4 |3 |b7 |en |up |1:3:3:f|3 |42 |3 |2 |1a00f000|pass
sup0 |20 |5 |0 |b7 |en |dn |0:0:0:0|0 |33 |0 |0 |15020000|pass
sup1 |21 |5 |1 |b7 |en |dn |0:0:0:1|0 |33 |0 |0 |15010000|pass
xgb1/14|13 |5 |1 |b7 |en |up |0:1:1:f|1 |32 |3 |6 |1a00d000|pass
xgb1/13|12 |5 |3 |b7 |en |up |1:3:3:f|3 |38 |3 |4 |1a00c000|pass
mfc2/5 |4 |6 |0 |b7 |dis|dn |0:0:0:0|0 |31 |6 |0 |01084000|pass
mfc2/6 |5 |6 |1 |b7 |dis|dn |0:0:0:1|0 |31 |6 |1 |01085000|pass
mfc2/7 |6 |6 |2 |b7 |dis|dn |0:1:0:2|1 |30 |6 |2 |01086000|pass
mfc2/8 |7 |6 |3 |b7 |dis|dn |0:1:0:3|1 |30 |6 |3 |01087000|pass
mfc2/4 |3 |6 |4 |b7 |dis|dn |1:2:2:0|2 |35 |5 |6 |01083000|pass
mfc2/3 |2 |6 |5 |b7 |dis|dn |1:2:2:1|2 |35 |5 |4 |01082000|pass
mfc2/2 |1 |6 |6 |b7 |dis|dn |1:3:2:2|3 |34 |5 |2 |01081000|pass
mfc2/1 |0 |6 |7 |b7 |dis|dn |1:3:2:3|3 |34 |5 |0 |01080000|pass

If you use dual physical ports for peer link, then definitely use two ports that are different ASICs.

Why “gatos”? Here is a little humor uncovered by Colin McNamara you might enjoy.