Nexus 5500 routing anomaly (OSPF stuck in EXSTART)

If you are using Nexus 5500 as a router, watch out for a routing anomaly. Otherwise, you may spend a lot of time debugging, troubleshooting becomes even more challenging when the other router is another vendor’s product.

In a typical scenario when a router is attached to a pair of Nexus 5500, and OSPF adjacency is to be established, the best practice is to run layer 3 Non VPC VLANs on a separate link from VPC trunk, and enable peer-gateway. However, the standard practice will result in OSPF adjacency established for directly connected Nexus 5k only, while the remote Nexu 5k remain in EXSTART.

Turns out, this is due to a little known Cisco bug. As a result, Cisco clearly states “separate link for nonvpc vlans on n5k is not supported”.

Currently the fix release is still pending. So a workaround is required, which is to run both VPC and non-VPC VLANs on VPC peer link.

Before change:

5548-sw1# sh ip ospf nei

OSPF Process ID 1 VRF default

Total number of neighbors: 2

Neighbor ID     Pri State            Up Time  Address         Interface

10.147.187.20  128 EXSTART/DR       00:10:14 10.147.187.20  Vlan110

10.147.254.161 129 FULL/BDR         00:10:12 10.147.187.19  Vlan110

Making change (both Nexus 5k):

5548-sw1# conf t

Enter configuration commands, one per line.  End with CNTL/Z.

5548-sw1(config)# int po10

5548-sw1(config-if)# switchport trunk allowed vlan 110

Must reset OSPF on interface for change to take effect:

5548-sw1(config)# int vlan 110

5548-sw1(config-if)# ip ospf shut

5548-sw1(config-if)# no ip ospf shut

5548-sw1# sh ip ospf nei

OSPF Process ID 1 VRF default

Total number of neighbors: 2

Neighbor ID     Pri State            Up Time  Address         Interface

10.147.187.20  128 FULL/DR          00:00:02 10.147.187.20  Vlan110

10.147.254.161 129 FULL/BDR         00:00:02 10.147.187.19  Vlan110