Network based TCP MSS adjustment

Maximum Segment Size (MSS) is set by end points during initial TCP handshake. In special circumstances, router can step in to alter MSS.

Let’s look at such a scenario when two hosts communicate through an SSL tunnel. End points sees a path MTU of 1500 byte, and set MSS to be 1500. However, SSL adds extra overhead. Therefore, when a 1500 byte packet arrives at tunnel end points, it becomes a little larger. Furthermore, SSL often sets DF (Do not Fragment). Since the packet is now larger than 1500 byte, with DF set, the router drops it. This results in communication failure between hosts (while ping and traceroute appears to be working). An extended ping with varying packet size will verify this exact behavior.

How to get around this issue? Increase MTU? Reduce MSS set by host and application? There is an easier method available in IOS 12.2(4)T and higher. Configured under interface, router can intervene and “adjust” TCP MSS with “ip tcp adjust-mss” command.

With the TCP adjustment option, router examines TCP SYN coming through the interface, and adjust it if necessary to ensure that it is lower than the set value. In other words, the router can lower MSS to account for the extra tunnel overhead. All this happens transparent to applications. The end result is TCP session is set up with a slightly lower MSS than application originally intended. Now packets with DF set will remain within MTU 1500 even with tunnel overhead, and thus transmitted across instead of being dropped.