Linux for Network Engineers: How to monitor packet drops with dropwatch

By July 29, 2020Linux

linux_packet_drops_dropwatch_remote_netbeez

We’ve talked about Linux tools that enable you to manipulate kernel networking parameters such as TCP window size and packet loss or latency. These kinds of tools are used when trying to optimize or troubleshoot network traffic either at the application or kernel level.

When it comes to measuring the impact of your changes, one of the tools you can use is dropwatch. As the name suggests, dropwatch enables you to monitor where packets are dropped on the Linux networking stack.

Dropwatch Installation

It doesn’t look like dropwatch is readily available within Linux repositories, so you will most likely need to install it from source by downloading it from here. Once downloaded, you have to run the following commands to install it:

If you get any errors, you may be missing one or more libraries. Running the following command should fix this problem:

Once these libraries are downloaded, re-run the installation commands.

Dropwatch Usage

Dropwatch doesn’t have many options, as you can see from its documentation

If you try to run dropwatch and get the following error, it means that the kernel of your system doesn’t support dropwatch’s functionality.

The most common way is to start it with the  -lkas and when prompted type “Start.”

Then you will see the following output.

And this will keep going until you hit  Ctrl-C to stop it.

As you can see there are some dropped packages related to software, and dropwatch gives us the kernel symbol or function related to each packet drop. If you don’t use the  -lkas option you will just get the address of the symbol (e.g. 0xffffffff9d5123d0) which is much more cryptic.

Testing Packet Loss

To test a bit more dropwatch we can try to force some packet drops on the host, and one way to do this is by running an iPert test while dropwatch is running and monitoring for dropped packets. iPerf is a utility that tests network bandwidth by pushing as much traffic as possible between two hosts. As you know, when the OS hits the limit of the interface bandwidth limit some packets will be dropped.

Here is how this looks like:

In the first few iterations we see a handful of packet drops reported. Once iPerf kicks in, the packet drop increases and then it dies off again once it’s done.