traceroute through MPLS

traceroute is often used as an effective analysis and troubleshooting tool. It is easily interpreted in a hop by hop routing network. Tracing packets through an MPLS network, however, requires more in-depth understanding of the internetworking between routing and tag switching.

The best place to start is the MPLS PE router. On the PE router, each customer’s VPN is represented by a vrf, in this case vrf “bigco”. Examining routing table for customer’s remote destination network (172.18.0.0), notice its “next hop” is the remote PE (BGP RR address). This may be counter-intuitive that a customer VPN has a next hop in the global routing table (effectively leaping from one vrf to another), but this is precisely where MPLS does its magic.

A_PE1#sho ip route vrf bigco 172.18.0.0

Routing entry for 172.18.0.0/16

…

  Last update from 10.8.0.1 5d18h ago

  Routing Descriptor Blocks:

  * 10.8.0.1 (Default-IP-Routing-Table), from 172.18.127.141, 5d18h ago

…

Note in the above display, customer VPN has a routing next hop 10.8.0.1 which exists only in the global routing table.  “Under the hood”, when customer VPN traffic arrives at PE, it is tag switched (not routed) through the MPLS network.

• Customer VPN destinations are learned from BGP peers (in this case 10.8.0.1 is BGP RR)
• Note 10.8.0.1 is not in vrf “bigco”, rather it is global “Default-IP-Routing-Table”
• How can a VPN route’s next hop to be global? On PE it is necessary, it is a special internal hook to make the linkage between routing and tag switching. All VPN route next hops are PE peers at layer 3

Here is a command that clearly illustrates the linkage between PE next hop and tag switching of VPN routes: “show bgp vpnv4 unicast vrf … tag”. The “tag” option is hidden. Here it shows that the next hop for VPN traffic is a remote PE.

A_PE1#sh bgp vpnv4 uni vrf bigco tag

   Network          Next Hop      In tag/Out tag

   172.18.0.0      10.8.0.1      notag/15

In order to reach the remote PE, PE looks up its tag switching table. In this case, tag switching identifies 10.8.0.1 with a local tag of 78, and out tag of 34. Tag switching continues through the MPLS network, until it reaches the remote PE.

A_PE1#sh mpls forward

78     34          10.8.0.1/32     0             Gi0/1      10.8.0.162

The topology represents the simplest form of an MPLS network which consists of P and PE. The sample VPN has a customer destination of 172.18.0.1. When packet to that destination arrives at A_PE1, routing table indicates its “next hop” as the remote B_PE1. To reach remote next hop, packet is tag switched through the MPLS network. The core routers (P) have no concept of VPN destinations, they are simply tag switching between PE destinations.

Traceroute ,when interpreted correctly, provides a nice end to end view. Here it shows tag switching from PE on. Note the inside tag identifies the VPN destination which does not change during transport. The outside tags (34 and 64) are tagging switching through the MPLS network (P and PE). Once it gets to the remote PE, the inside tag (15) is popped and regular routing applies to next hop (CE).

A_PE1#traceroute vrf bigco 172.18.0.1

…

  1 10.8.0.130 [MPLS: Labels 48/15 Exp 0] 128 msec 184 msec 216 msec

  2 10.9.32.226 [MPLS: Labels 43/15 Exp 0] 196 msec 232 msec 152 msec

  3 10.8.33.18 [MPLS: Label 15 Exp 0] 152 msec 88 msec 168 msec

  4 10.8.33.17 656 msec 704 msec 644 msec