In the quest for ultimate performance, IT folks tweak TCP stacks, strip out unnecessary services, and manage traffic flows. But where the WAN is concerned, some things are in the hands of the ISP rather than IT. That’s a sad fact of life: As link speeds and round trip time increase (latency), overall throughput degrades tremendously if router queues aren’t sized accordingly.
Work by researchers Curtis Villamizar and Cheng Song has found that TCP performance does not depend on link speed alone, but on many factors such as the product of the link speed and round trip time, also known as the BDP (bandwidth delay product). An interesting application of BDP is in determining the proper queue size (in packets) for a backbone router. This value is found by dividing the BDP by the product of the MTU (maximum transmission unit) size converted to bits.
Many routers come preset with buffer levels that have not been set for optimum efficiency. If the buffer size is too small, the risk of congestion and TCP retransmission arises — a major cause of performance drop off. Too large a buffer and latency is increased. Both situations cause TCP to slow down, greatly reducing a WAN link’s efficiency and throughput. (Like Goldilocks said: not too big, not too small, but just right.)
Ethernet’s default MTU of 1500 Bytes is fine for speeds of 10 Mb/s or even 100 Mb/s, but orders of magnitude too small when you reach Gigabit and greater speeds. One way to help improve your WAN performance is to use a larger MTU size. But again, since you have no control over the MTU settings used by your provider, if you use jumbo frames and your ISP uses standard sizing, your packets will be fragmented and repackaged at the smaller value as determined by the router, increasing the load on the router further complicating performance.
Problem is, unless you maintain your own backbone, there’s little chance that your provider will tweak their routers to suit your link speed and latency. While you could beg and plead with your provider to modify their queue size for your deployment, because your traffic is mixed in with other traffic they manage, optimal settings for your installation may adversely affect another user. So you end up with a semi-arbitrary queue size that falls somewhere in the middle.