How to Adjust the TCP Window Size Limit on Linux

Iperf is a bandwidth testing tool that network engineers use for stress testing their networks. As we will this in this article, the maximum TCP bandwidth that iperf can achieve depends on many factors. Some users that need to generate more than 1 Gbps may need to use some iperf advanced options. In some cases, it may also be necessary to change some TCP/IP settings within their operating systems.

Factors impacting iperf TCP throughput

There are three main factors that impact iperf maximum TCP throughput:

  1. Interface: if you are using a Raspberry Pi that has a 10/100 interface, then the upper limit is 100 Mbps.
  2. CPU capabilities: pushing several Gbps of traffic needs quite a bit of CPU performance.
  3. Latency: the greater the latency between source and destination, the larger the TCP window size needs to be.

The interface and the CPU put hard limits on the maximum bandwidth you can achieve. You can’t bypass them unless you get new hardware.

However, given your link’s latency, you can adjust the TCP window size in order to achieve the desired bandwidth. But it can be tricky …

Calculating the TCP window size

The maximum achievable bandwidth is equal to:

[TCP Window Size in bits] / [Latency in seconds] = [Throughput in bits per second]

Here is a calculator for getting the TCP window size.

The default window size that iPerf uses varies by machine type and operating system. If it’s 64 KBytes, with a modest latency of 10 ms, the maximum bandwidth you can get is 52.43 Mbits/sec. For 50 ms the maximum is 10.49 Mbits/sec.

The obvious option would be to increase the window size to a larger value and get up to, let’s say, 500 Mbps. For a 50 ms latency, the window size needs to be 3.1 Mbytes.

Indeed, iPerf gives you the option to set the window size, but if you try to set it to 3.1 or 4.0 Mbytes you may get the following warning from iPerf:

“TCP window size:  416 KByte (WARNING: requested 4.00 MByte)”

Oops… Why didn’t iPerf follow your command and used only a 416 Kbyte window?

The devil is in the details!

In this case, the devil is the operating system that has a hard limit on the TCP window size that an application can use. It looks like in this case the limit is 416 Kbytes. These limits exist for good reasons. Larger TCP windows take more system memory and if you have multiple applications running, using large windows they may bog down the system. Or the system may deny TCP connections if it is running out of memory.

Changing the TCP window size on Linux

In your case, if you have full control of the system and you know what you are doing, you may want to increase the OS limits. As an example, to increase those limits to 4 MBytes, run the following commands as root on a Debian Linux machine:

echo 'net.core.wmem_max=4194304' >> /etc/sysctl.conf
echo 'net.core.rmem_max=12582912' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_rmem = 4096 87380 4194304' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_wmem = 4096 87380 4194304' >> /etc/sysctl.conf
sysctl -p

Now if you request iPerf to use a 4 Mbyte TCP window size, you will see that your request will get fulfilled like it’s supposed to.

Conclusion

For some iperf basics, take a look at some of these past posts:

decoration image

Get your free trial now

Monitor your network from the user perspective

You can share

Twitter Linkedin Facebook

Let's keep in touch

decoration image