How to troubleshoot and validate QoS

Importance of QoS

TL;DR: In this article I want to provide a simple way to troubleshoot and validate QoS.

All networks are not created the same. Every business has a different set of needs and requirements. How do you ensure your critical services and applications are receiving the highest priority over everything else happening on your network? Historically the answer to this question was to add bandwidth. This creates a challenge in a network where some of your clientele, such as students will take every bit of bandwidth you throw at them. Quality of service, or QoS helps to provide a certain level of service across your network. A full end-to-end QoS solution provides a higher level of service to your critical applications and also provides value by keeping your circuits more affordable.

Our entire WAN is serviced by private MPLS services, provided by two carriers. Since we service both employees such as support staff and faculty, as well as a general student population, we looked for a solution that would help us validate and test our end-to-end QoS.

QoS Configuration

Our QoS configuration is pretty straightforward. We provide a priority class for Voice, and then define additional classes that we map traffic to and assign a bandwidth percent, or remaining percent depending on the carrier. Both MPLS carriers honor our markings to ensure consistent level of service.

On the other end, we make sure to mark and trust our L2 switches to ensure consistent marking across our network.

Validate QoS

This QoS configuration has served us well but the question was always “How do we validate our QoS?” Especially when someone places a call to the help desk complaining about a particular application such as Citrix, CIFS and HTTP. Usually, we would begin by taking a look at the graphs from another tool, which would show us bandwidth utilization per class or DSCP marking. This would help us determine causes such as too many Citrix sessions/terminals at the campus, which would provide sub-standard service to Citrix and starve out non-Citrix traffic. A situation such as this only happens a few times a year and results in a bandwidth upgrade for the campus.

What if that is not the issue? Next, we tried to do a file transfer test between two machines. This is never a good test as you must account for server CPU cycles, disk latency, etc. We have also used Iperf in the past, but this required installation on a client machine and coordination with our local campus techs to get the machine online and connected. This is where NetBeez became an attractive option.

Troubleshooting QoS with NetBeez

Each NetBeez agent is a dedicated Iperf client and server.

Iperf 1

We have several agents installed at our datacenter, as well as several deployed at our campuses. This is crucial as we can test the full end-to-end QoS between campuses as well as between campus and datacenter. By default if you just enter your source agent and your destination agent, a standard Iperf test will be launched.

Iperf 2

Where the real power comes in is by specifying the ‘TOS DSCP/PHB Class’ option.

Iperf 3

With this option, we can test a specific DSCP marking/class! We can use the NetBeez output, in conjunction with the CLI output to see exactly how our QoS and our circuits are performing. For this, we specify the bandwidth, the DSCP/PHB, Class and Test Duration.

Iperf 4

As you can see, we have validated 15Mbps via marking af21 in our COS3-DATA class with no drops. We can repeat this process by hitting a DSCP marking in each class to fully validate not only our own end-to-end QoS, but also the carrier QoS.

As a bonus option – Iperf can also be used to test multicast. We utilize a suite of software for software/package deployment to our client machines that depends on multicast. When there is a problem, this has historically required many hours on our part to work with the server administrators to find/fix the problem that is almost always the server configuration. With two NetBeez agents at the campus, we can validate that multicast routing is working as expected and provide this output to the server administrators to investigate their server/client multicast configuration.

On the same Iperf tab, select the checkbox for ‘Multicast IPerf’ and specify a group address. Again, we can use the NetBeez in conjunction with the CLI output to validate multicast routing.

Iperf 4

I hope this post shows the real power of having dedicated NetBeez agents at all of your locations to validate bandwidth, QoS and multicast.


About the Author

Matthew D. Smith (CCIE #26439 R&S) is a network engineer with over 14 years of experience including Fortune 500 companies, Government, ISP, consulting and private sector.

NetBeez Tech Overview and Benefits