BGP timer and Cisco/Juniper negotiation

Juniper SRX integrates firewall features with full routing capabilities. In a network architecture where there are increasing demand for security segmentation and stateful firewall inspection within the network, multi-vender routing interaction may become necessary.

Risks and complexity exist with a mixed vendor environment, even when both sides support standard protocols. Here is such an example.

To establish BGP neighbor relationship, both sides need to agree on timer values. RFC4271 defines the characteristics of the timer, but does not specify value for keepalive and hold time.

BGP uses keepalive timers to monitor established connections. If for a period of time that exceeds hold time, BGP considers the neighbor connection down. Note the hold time is always three times the keepalive time.

By default, vendors set timers differently:
vendor Keepalive (second) Hold time (second)
Cisco Nexus (4.2) 60 180
Juniper SRX (10.0) 30 90

The timers should match between vendors. This is based on configuration, or negotiated behavior. Note with Juniper SRX, BGP on the local routing device uses the smaller of either the local hold-time value or the peer’s hold-time value received in the open message as the hold time for the BGP connection between the two peers. Therefore, by setting timer on Cisco Nexus to a smaller value (10/30), it is used by SRX.

To reduce convergence, the test sets Nexus keepalive to 10 seconds. This shows after successful negotiation, SRX (with a local hold time of 90) uses Nexus’s keepalive of 10 seconds, and active hold time of 30 seconds.

On Juniper SRX:
Peer: 10.88.15.14+179 AS 65000 Local: 10.88.14.4+54104 AS 65000
Type: Internal State: Established Flags:
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Export: [ bgp_outbound_policy ]
Options:
Local Address: 10.88.14.4 Holdtime: 90 Preference: 170
Number of flaps: 0
Peer ID: 10.88.120.130 Local ID: 10.88.14.4 Active Holdtime: 30
Keepalive Interval: 10 Peer index: 0
…

On Cisco Nexus:
Nexus7k# sh ip bgp nei vrf test
BGP neighbor is 10.88.14.4, remote AS 65000, ibgp link, Peer index 1
BGP version 4, remote router ID 10.88.14.4
BGP state = Established, up for 00:53:15
Last read 00:00:05, hold time = 30, keepalive interval is 10 seconds
Last written 0.956168, keepalive timer expiry due 00:00:09
Received 20433 messages, 1 notifications, 0 bytes in queue
Sent 18392 messages, 0 notifications, 0 bytes in queue
Connections established 2, dropped 1
Last reset by peer 02:05:25, due to peer deconfigured
Last reset by us never, due to process restart

Neighbor capabilities:
Dynamic capability: advertised (mp, refresh, gr)
Dynamic capability (old): advertised
Route refresh capability (new): advertised received
Route refresh capability (old): advertised received
4-Byte AS capability: advertised received
Address family IPv4 Unicast: advertised received
Graceful Restart capability: advertised received
…

Enable BGP load sharing with Multipath-relax

If your network is like most large enterprise’s, chances are you have Internet connection from dual ISPs to your data center.

It used to be a popular tactic to have a primary provider, and a backup provider which you pay lesser bandwidth for. In recent years, usage based circuits has given way to flat rate, often larger bandwidth circuits from both ISPs. After all, users demand more bandwidth, even in failure scenarios.

Why would you pay for expensive Internet access and only letting it sitting idle most of the time? Therefore, load sharing on redundant ISP connections has become highly desirable for data centers.

By default, BGP only installs one route into the routing table. To accomplish load sharing, BGP multipath needs to be enabled. To further complicate the matter, BGP requires all paths to have the following equal characteristics in order to become equal cost path:
• Weight
• Local preference
• AS-PATH length
• Origin
• MED
• AS-PATH

Since neighboring AS or AS-PATH must be equal, that means BGP can only load balance to a single AS or single ISP. Since different ISP will have different AS and AS-PATH, BGP will only pick one best route.

So how does Cisco support load balancing to two ISPs? There is a hidden command bgp bestpath as-path multipath-relax

I don’t know why it is hidden command, since it appears to be a very useful feature, and it is supported. But do use with caution, as the number of routing entries will increase, often overwhelming the limited memory you have on the Internet router.

In this year’s Networker, an alternative method was also shown, essentially separating out the Internet routing table by even and odd digits, for outbound traffic.