Network Testing: Tools and Best Practices

Network testing is the process of verifying the correct functioning and performance of a network infrastructure. The goal of network testing is to ensure that a network is efficient, secure, and reliable. There are several tools and methodologies for network testing. Before picking a solution, it’s important to understand what are the main use cases, what are your requirements, and what goals you want to achieve. In this article, we will review when makes sense to test a network, and what commonly available tools you can use. We will also mention NetBeez network testing capabilities.

Why Network Testing Is Important

In today’s interconnected world, the smooth flow of data is essential for virtually every aspect of today’s businesses. Network testing is not just a way to assess network performance. Network testing acts as quality assurance of business processes that are delivered over a digital infrastructure. 

Here are five reasons why organizations need to implement network testing:

1. Identifying performance bottlenecks: There are several factors that impact the performance of a network, such as the specifications of its routers and switches, the way it is designed and configured, the type of connectivity is provisioned, etc. Performance testing helps identify areas where the throughput is not as expected.
2. Risk assessments: Network testing is used to assess the risks of a partial or total loss of network service and connectivity. For instance, when doing a redundancy test by turning off a primary switch or router, or shutting down the primary utility line to test if the backup generator kicks in. These controlled outages help verify that risk mitigation measures are correctly implemented and that are functioning as designed.
3. Compliance requirements: For example, financial and healthcare organizations require a way to validate that their network complies with PCI or HIPAA standards. Network testing provides a method to actively monitor and enforce adherence to these and other standards.
4. Assessing network controls: All organizations must implement security measures to protect and prevent network threats. Network testing can be used to verify that proper network controls such as access lists and firewalls are configured as expected.
5. Continuous improvement: Continuous network testing provides more depth of network insights as it generates an historical trace of network performance. These baselines will help analyze long-term trends and estimate in advance when upgrades will be necessary. Continuous improvement should also consider digital experience monitoring.

Network Testing Use Cases

You can perform network testing ad-hoc, for example after a network change, or permanently, to keep a constant monitoring of the network. There are different use cases for network testing. The most common ones are:

• Validate configuration changes –  Testing services and connectivity after a network configuration change to verify that it was successful and that it didn’t cause unplanned outages. Testing could be as easy as running a ping test, or more complicated checks in the case of routing policies or QoS reconfigurations.
• Circuits turn-up or sites go-live –  When turning up a new network site or link, a connectivity and throughput test will help verify that outside networks are reachable and that enough bandwidth is provisioned. This includes ping tests for outside reachability along with an iperf test for throughput. Should the link be more than 1 Gbps, specialized hardware may be used to gather accurate measurements. 
• Troubleshooting – When troubleshooting a single user complaint or an outage, network testing helps isolate the problem and collect all the evidence required to identify the root cause. Check out two articles that we wrote about how to troubleshoot network performance issues and how to troubleshoot end-user problems.
• Service assurance –  Whether you are offering a SaaS solution to customers, connectivity to residential subscribers, or WiFi to employees, network testing helps monitoring and enforcing Service Level Agreements. A network monitoring solution like NetBeez for example can trigger alerts and notification in the case where a performance metric doesn’t match a certain rule.
• Proactive monitoring – By running continuous network tests, you collect and monitor key performance metrics, such as latency and packet loss, that help you be proactive on issues that could degrade network performance and the end-users’ experience.

Network Testing Tools

The results of network testing provide key insights on network performance and highlight issues and potential risks associated with a network. In this section, we will mention commonly available tools that everyone can use. At the end of the section we will also introduce NetBeez, as it automates these tests. We also link relevant articles we published in the past, and that you can read to further understand a specific network testing tool

Ping

Ping is a command-line tool to test network connectivity. By default, ping works by sending one packet (echo_request) every second and waiting for a reply (echo_reply). With each reply ping reports the round-trip time (RTT), which is the time that it took for one exchange of packets. Round-trip time is expressed in milliseconds (ms). Ping can run to send a certain amount of requests (option -c count) or continuously until the user interrupts it. Once the command exits, ping reports the packet loss (% of packets not replied), and the min, max, and average of the round-trip time. Read this article on ping options if you want to learn some extra options and testing use cases you can achieve.

The following image shows a ping test where all 10 packets sent are replied: In this case we have no packet loss (0%).

In this other example, we run the same ping test, but one packet is lost: It could be that the echo_request was lost or damaged, or the echo_reply. We can’t tell that, but we still detect a 10% packet loss rate.

NetBeez supports ping tests to monitor latency, packet loss, estimate jitter and MOS, and get alerts when reachability to a remote host is lost or undergoing performance issues. Associated with each test, the user can define the alerting conditions that will trigger a notification. Below is a screenshot of a continuous ping test to google.com.

Traceroute

Traceroute is a command that discovers all the intermediate routers between a source host where the command runs, and a destination one. The command ends when it reaches the destination. Each intermediate router is called hop. For each router, traceroute reports the round-trip time (RTT) and its hostname. This article on traceroute explains in detail how it works. 

In network testing, traceroute allows you to verify that the traffic from one host is following the correct path. It also helps you find out where traffic is interrupted when troubleshooting. One important thing to be aware of is that there are many flavors of traceroute. Each one of them has its pros and cons:

• Traceroute: is the most basic and available tool that is installed in operating systems like Windows (called tracert), Linux, and Mac. Easy to use and gets the job done.
• MTR: This tool works as a traceroute test, but it also reports the packet loss per hop. This article on MTR provides more information about it. You will need to download and install this if you want to gather packet loss information per hop.

Below, we’ll present the example of a traceroute execution on Mac to netbeez.net. For each hop, traceroute reports its IP address, hostname (if available), and three RTT measurements.

% traceroute netbeez.net
traceroute to netbeez.net (141.193.213.10), 64 hops max, 52 byte packets
 1  172.29.0.1 (172.29.0.1)  2.805 ms  1.736 ms  1.724 ms
 2  164.52.244.85 (164.52.244.85)  4.029 ms  2.939 ms  4.314 ms
 3  64.58.254.226 (64.58.254.226)  4.000 ms  4.032 ms  2.993 ms
 4  * * *
 5  ae1.3510.level3.net (4.69.219.110)  35.903 ms *  14.297 ms
 6  4.14.100.230 (4.14.100.230)  15.174 ms * *
 7  172.71.188.4 (172.71.188.4)  46.029 ms
    172.70.32.4 (172.70.32.4)  15.888 ms
    173.245.63.243 (173.245.63.243)  14.852 ms
 8  netbeez.net (141.193.213.10)  14.897 ms  14.363 ms  14.116 ms

When you see * on a hop, that means that we don’t have information about that hop. The node most probably was configured to disregard traceroute probes. Unfortunately it’s a limit that can’t be hardly overcome in most cases. In some cases, you can try to change the protocol used by traceroute, which is ICMP in Windows and UDP in Linux and Mac.

Path Analysis

The two above commands, traceroute and MTR, only discover and report one path at the time. As a result, when ECMP is present, those tools may report an incorrect sequence of hops. So if discovering alternate paths is key for you, consider this option:

• Dublin-traceroute: This command supports Equal-Cost MultiPath (ECMP), meaning that it can discover redundant links and map them. By default it runs 20 concurrent traceroute tests to identify all possible routes to a destination. Dublin traceroute also provides information about the Autonomous System Number (ASN) of a node, NAT detection, and more. You can read more about dublin-traceroute in a previous article we wrote.

NetBeez supports both traceroute and dublin-traceroute commands. We call dublin traceroute Path Analysis. In our implementation, we enrich the node data with geolocation. Below is a screenshot of a path analysis test against netbeez.net. Note the display of ECMP routes, and compare to the traceroute test run up above. Path analysis is a must for network testing SaaS and other Internet applications.

DNS

DNS is a vital network service whose function is to translate hostnames such as those contained in web URLs to IP addresses. When a large company suffers a DNS outage, you’ll read about it in the news. Users wouldn’t be able to access websites and other applications, and hosted services would stop functioning. A nightmare for network engineers (just search for “DNS meme” and have a laugh). Let’s see how you can verify the correct functioning of a DNS server.

Most operating systems support the command nslookup (dig is an alternative in Linux and Mac). Nslookup allows you to perform a DNS query with a specific server. To run this utility, you type nslookup on the command prompt. That will get you into the nslookup console. Once in, you type the command server followed by the IP address of the DNS server that you want to test. Once selected the nameserver, you type the hostname that you want to resolve. Here’s an example with nameserver 192.168.1.1 and hostname netbeez.net.

nslookup
> server 192.168.1.1
Default server: 192.168.1.1
Address: 192.168.1.1#53
> netbeez.net
Server: 192.168.1.1
Address: 192.168.1.1#53

Non-authoritative answer:
Name: netbeez.net
Address: 141.193.213.10
>

The above nslookup output successfully reports the IP address of netbeez.net, which is 141.193.213.10. If you want to know why the server 192.168.1.1 replies with a “non-authoritative answer”, read this DNS article we wrote. 

Let’s see what happens when a DNS test fails:

> netbeez.net
;; connection timed out; no servers could be reached
>

In this case,  we don’t have a connection with the DNS server because of the timeout. First, validate that connectivity to the DNS server is available. Then, verify that the DNS service is running.

NetBeez provides DNS testing with query timing. You can select the DNS server to test, and the query to execute. If the server is unspecified, each network monitoring agent will use whatever DNS server has configured. NetBeez can then trigger alerts if there are no replies, or if the resolution time is above a certain threshold.

Iperf

Iperf is a bandwidth testing tool that generates TCP or UDP traffic between a source and a destination. Once the commands exits, it reports different statistics such as throughput, packet loss (UDP), jitter (UDP) and retransmissions (TCP). It’s a valuable open source utility that everyone can use. When considering iperf for bandwidth testing, make sure that the hardware supports the speed that needs to be tested. In cases where you want to push several Gbps of traffic, you may need to change the TCP window size or use multiple streams. We wrote extensively about iperf, so check out that link as a starting point.

To run iperf, execute first the server on the receiving host with iperf -s, then the client with iperf -c followed by the server’s IP address. Here’s one iperf command between two hosts.

------------------------------------------------------------
Client connecting to 10.0.20.5, UDP port 5001
Sending 1470 byte datagrams, IPG target: 1121.52 us (kalman adjust)
UDP buffer size: 0.16 MByte (default)
------------------------------------------------------------

[ 3] local 10.0.10.5 port 60603 connected with 10.0.20.5 port 5001

[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 1.22 MBytes 10.2 Mbits/sec
[ 3] 1.0- 2.0 sec 1.28 MBytes 10.8 Mbits/sec
[ 3] 2.0- 3.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 3] 3.0- 4.0 sec 1.24 MBytes 10.4 Mbits/sec
[ 3] 4.0- 5.0 sec 1.26 MBytes 10.5 Mbits/sec
[ 3] 5.0- 6.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 3] 6.0- 7.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 3] 7.0- 8.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 3] 8.0- 9.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 3] 0.0-10.0 sec 12.5 MBytes 10.5 Mbits/sec
[ 3] Sent 8917 datagrams

[ 3] Server Report:
[ 3] 0.0-10.0 sec 12.5 MBytes 10.5 Mbits/sec 0.009 ms 0/ 8917 (0%)

One thing to note about iperf is that it’s a unidirectional test. To run a test in both directions, you need to switch the roles of the two hosts. In alternative you can use the reverse option, that runs a transfer from the server to the client. This is useful in the case where the receiver (server) is behind a NAT or firewall.

NetBeez supports iperf for testing network bandwidth. When creating a new iperf test, the user selects the source and destination agents, the protocol, iperf version, and options. Then, it picks the schedule when the test should run and the alerting conditions. For instance, if the resulting bandwidth is less than 10 Mbps.

Internet bandwidth testing

There are several tools available for bandwidth testing. Two popular options are:

• Speedtest by Ookla is proprietary, has a CLI version that is multi-platform.
• NDT by M-LAB is open source, has a CLI version, supports Linux, Windows, and Mac.

Here, I’ll focus on the open source one. NDT (Network Diagnostic Tool) is a bandwidth testing tool that runs a download and upload test with a server located on the Internet. When the test starts, it selects the server to use from a list of available ones, generally the one with lowest latency. The organization that offers this service (M-LAB) is a non-profit whose goal is to collect performance measurements for the Internet. Many organizations participate in this initiative, including Internet2. You could also host your own NDT server. 

To conclude, NDT is a good alternative, or addition to Ookla speed test. NDT can also run on a web browser. If you search for speedtest on google, Google will give you the option to run a browser based speed test. In the following image you can see a NDT test run from a Windows machine.

NetBeez can run Internet bandwidth tests with NDT. Similarly to iperf, the user picks the agents, the schedule, and the warning conditions. The following screenshot displays the aggregate results of NDT tests from several agents. It is then possible to break down the results and sort by download, upload speed, or latency.

SaaS and web applications testing

One simple way to test SaaS and web applications is to use a web browser and verify that you can access them. However, this method doesn’t perform continuous testing. As a result, it doesn’t scale when you have many applications to check and/or if your network is complex. Think about when you have to push a routing change, or a firewall update. 

When working in a large and mission critical environment, automate the application tests. This way, you can receive alerts as soon as a problem arises. In that scenario, you can either revert back your changes, or quickly troubleshoot the problem. To conclude, automating application testing will help you to be proactive and quick at detecting issues.

Two network testing tools for SaaS and web applications available are:

• Curl is a powerful command line tool for testing HTTPS pages and web services in general. Curl supports timing breakdown, and other options that may be useful for network testing. I invite you to read an article we previously wrote about curl timing.
• Hping is a command line utility that tests if a TCP or UDP port is open and responding. This is one of the most common tools to test firewalls and check if a specific service on a host is responding. You can read a previous article we wrote about hping options.

NetBeez: Network testing solution

NetBeez is a powerful network testing and monitoring solution that stands out for its effectiveness in ensuring the reliability and performance of network infrastructures. What makes NetBeez particularly beneficial for network testing is its proactive and distributed approach. Continuous performance testing against the network and application ensures a proactive approach to accident management and quick troubleshooting.

By deploying hardware or software agents throughout your network, NetBeez continuously assesses network health from multiple vantage points, providing real-time insights into network latency, packet loss, and other key performance metrics. The following table lists the types of network monitoring tests run by NetBeez.

This distributed monitoring approach allows for early detection of network issues, enabling IT teams to troubleshoot and resolve problems before end-users even notice a dip in performance. NetBeez supports the following deployment options thanks to a vast range of agent types:

Additionally, NetBeez offers a user-friendly interface and comprehensive reporting, making it a valuable tool for network administrators and engineers seeking to optimize their network’s efficiency and ensure a seamless user experience. Overall, NetBeez empowers organizations with the tools they need to proactively manage and maintain their networks, reducing downtime and enhancing overall network performance.

Conclusion

Regular testing and proactive troubleshooting are key to a successful network management strategy. To meet the demands of the end-users and ensure business resiliency, it’s important to understand what tests to run, use the right tools, and follow best practices. Remember that network testing is a vast field, and this article is just a starting point. If you wish to learn more about NetBeez, request a demo and give it a try!