How to Use the NetBeez API to Retrieve Ping Results

The NetBeez API

NetBeez gathers performance data from multiple points in your networks, by deploying network monitoring sensors or agents (known as Beez), which conduct continuous end-to-end tests against multiple targets in your network using a multitude of tools (ping, DNS, http, traceroute, speed test, iperf etc). The data is available to be viewed on the web-based dashboard, but sometimes your specific needs may require some extra data analysis to understand exactly the performance of your network from the end-user’s point of view.

One use-case could be that you want to view some data in a way that the dashboard doesn’t readily provide at the moment. Then this information can be used to generate reports that could help you make decisions on future expansions, or report the status of your network to upper management.

In this article we will go through the steps of setting up the necessary variables, to make REST API requests to a BeezKeeper instance.

More documentation about the NetBeez API can be found here: https://api.netbeez.net (and for the legacy API which will eventually be fully replaced by the one just mentioned: https://demo.netbeezcloud.net/swagger/)

How to get started with the NetBeez API

Before you can start using the NetBeez API to get network performance data, you’ll need to perform two actions:

  1. Setup your NetBeez BeezKeeper details
  2. Setup the headers

Let’s review each one of these steps in more detail.

Setup your NetBeez BeezKeeper details

First, you need to identify your server’s FQDN and generate an API key to use in these examples.

If the URL you use to access NetBeez looks like this https://demo.netbeecloud.net then demo.netbeecloud.net is the fqdn.

In [1]:
beezkeeper_fqdn="demo.netbeezcloud.net"
base_url='https://' + beezkeeper_fqdn

In [2]:
from IPython.display import Markdown as md
md(f"To generate an API key head over to https://{beezkeeper_fqdn}/#settings-tab/api-keys-settings")
md(f"For documentation go to https://api.netbeez.net or https://{beezkeeper_fqdn}/swagger/")

Out[2]:
For documentation go to https://api.netbeez.net or https://demo.netbeezcloud.net/swagger/

In [3]:
api_key="7g5_hKXw_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

Set Up the Headers 

We define the content-type format as JSON and provide the API key that you generated in the previous step as part of the headers.

In [4]:
import requests
import urllib3

urllib3.disable_warnings()


headers = {
            'Cache-Control': 'no-store',
            'Content-type': 'application/json',
            'Authorization': 'Bearer ' + api_key
          }

legacy_api_headers = {
    'Authorization': api_key,
    'API-VERSION': 'v1',
    'Content-type': 'application/json'
}

Note that when the legacy API headers are used it means that we are using the legacy API which is subject to be replaced soon by the new JSON API (documentation here: https://api.netbeez.net)

How to Retrieve Ping Test Results

In this example, we will be searching for an agent by name, then selecting the ping tests that are currently running, and then retrieving the raw measurement results for those tests from the past 60 minutes.

Retrieve the agent

First, we set ‘retrieve the agent’ by search by name using this API endpoint: https://api.netbeez.net/#f3d48709-5344-45b7-af9b-7b92feddd4ec.

In [5]:
search_name = "Virtual Agent"
url = f"{base_url}/agents?filter[name]={search_name}&type=beta"

In [6]:
response = requests.request("GET", url, headers=headers, verify=False)

import json
data = json.loads(response.text)
agent = data['data'][0]
print(agent)

{'id': '341', 'type': 'agent', 'attributes': {'name': 'Pittsburgh - Virtual Agent', 'reached_target_test_limit': False, 'reached_scheduled_test_limit': None, 'nb_target_ids': [463, 464, 465, 437, 32], 'nb_test_ids': [1667068, 1667069, 1667070, 1667065, 1667066, 1667067, 1667071, 1688201, 1688302, 1688303, 1688304, 1688305, 1688306, 1688391, 1688392, 1688393, 1688394, 1688395, 1688474, 1688475, 1688476, 1688477, 1688478], 'agent_type_name': 'virtual-model-ova', 'os_version': 'Debian GNU/Linux 8.11 (jessie)', 'kernel_version': '#1 SMP Debian 3.16.51-3 (2017-12-13)', 'uuid': '00:0c:29:09:fa:ee', 'agent_class': 'virtual', 'active': True, 'active_ts': 1624631782781, 'description': None, 'category': 'network_agent', 'software_version': '8.0.6', 'open_incident_id': None, 'test_aggregate_counts_by_type': {'1': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '2': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '3': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '4': {'success': 4, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}}}, 'relationships': {'network_interfaces': {'data': [{'id': '387', 'type': 'network_interface'}]}, 'agent_groups': {'data': [{'id': '5', 'type': 'agent_group'}]}}}

Retrieve ping tests

Then we iterate through the tests and select the ping tests.

In [7]:
nb_tests = []
for nb_test_id in agent['attributes']['nb_test_ids']:
    url = f"{base_url}/nb_tests/{nb_test_id}.json"
    response = requests.request("GET", url, headers=legacy_api_headers, verify=False)
    data = json.loads(response.text)
    if data['heir_type'] == 'PingTest':
        print(data)
        nb_tests.append(data)


{'id': 1667068, 'target': 'www.google.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-04-01T15:39:11.736-04:00', 'created_at': '2021-04-01T19:39:11.000Z', 'updated_at': '2021-06-03T15:34:27.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11374, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': None, 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 99, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 32, 'nb_test_template_id': 99, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': True}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1667065, 'target': 'demo-app.netbeezcloud.net', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-04-01T15:39:11.604-04:00', 'created_at': '2021-04-01T19:39:11.000Z', 'updated_at': '2021-06-03T15:34:27.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11373, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 1830, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 437, 'nb_test_template_id': 1830, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': False}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688303, 'target': 'baidu.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-17T23:56:56.125-04:00', 'created_at': '2021-05-18T03:56:56.000Z', 'updated_at': '2021-07-05T15:02:15.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11441, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2074, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 463, 'nb_test_template_id': 2074, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688391, 'target': 'in.gr', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-17T23:57:16.957-04:00', 'created_at': '2021-05-18T03:57:16.000Z', 'updated_at': '2021-07-05T18:12:00.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11461, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2078, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 464, 'nb_test_template_id': 2078, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688474, 'target': 'amazon.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-18T00:00:42.578-04:00', 'created_at': '2021-05-18T04:00:42.000Z', 'updated_at': '2021-06-11T06:27:10.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11477, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2083, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 465, 'nb_test_template_id': 2083, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}

Retrieve the results

Then we retrieve the results for the past 60 minutes.

In [8]:
import time
import datetime
to_ts = int(time.time() * 1000)
from_ts = to_ts - (60 * 60 * 1000)

print(datetime.datetime.fromtimestamp(from_ts/1000.0))
print(datetime.datetime.fromtimestamp(to_ts/1000.0))

2021-07-06 02:18:17.003000
2021-07-06 03:18:17.003000

In [9]:
results = []
import pandas as pd
for nb_test in nb_tests:
    url = f"{base_url}/results.json?nb_test_id={nb_test['id']}&from={from_ts}&to={to_ts}"
    response = requests.request("GET", url, headers=legacy_api_headers, verify=False)
    df = pd.json_normalize(json.loads(response.text), 'index')
    if df.empty == False :
        df['timestamp'] = df['timestamp'].transform(lambda x: datetime.datetime.fromtimestamp(x/1000.0))
        df.set_index('id', inplace=True)
        results.append(df)
    
results = pd.concat(results)

…and here is a printout of the results:

In [10]:
print(results)

           value               timestamp  sequence_number  nb_test_id
id                                                                    
9905711541  12.10 2021-07-06 02:18:18.637          1643884     1667068
9905711772  20.40 2021-07-06 02:18:23.852          1643885     1667068
9905711878  20.40 2021-07-06 02:18:28.856          1643886     1667068
9905712156  20.10 2021-07-06 02:18:33.858          1643887     1667068
9905712268  38.40 2021-07-06 02:18:38.865          1643888     1667068
...           ...                     ...              ...         ...
9905840070   9.75 2021-07-06 03:17:53.055           844807     1688474
9905840241   9.57 2021-07-06 03:17:58.073           844808     1688474
9905840449   9.75 2021-07-06 03:18:03.077           844809     1688474
9905840669   9.74 2021-07-06 03:18:08.082           844810     1688474
9905840809   9.67 2021-07-06 03:18:13.083           844811     1688474

[3595 rows x 4 columns]
In [11]:
results_pivot = results.pivot(index='timestamp', columns='nb_test_id', values='value')
pf = results_pivot.interpolate()
print(pf)

nb_test_id               1667065  1667068  1688303  1688391  1688474
timestamp                                                           
2021-07-06 02:18:18.194      NaN      NaN      NaN      NaN    9.340
2021-07-06 02:18:18.637      NaN    12.10      NaN      NaN    9.376
2021-07-06 02:18:18.871    10.60    13.76      NaN      NaN    9.412
2021-07-06 02:18:18.881    10.58    15.42      NaN    105.0    9.448
2021-07-06 02:18:19.188    10.56    17.08    214.0    104.5    9.484
...                          ...      ...      ...      ...      ...
2021-07-06 03:18:13.083    10.42    20.90    223.8    104.0    9.670
2021-07-06 03:18:13.394    10.40    21.10    220.6    104.0    9.670
2021-07-06 03:18:13.465    10.40    21.30    217.4    104.0    9.670
2021-07-06 03:18:13.587    10.40    21.30    214.2    104.0    9.670
2021-07-06 03:18:13.904    10.40    21.30    211.0    104.0    9.670

[3415 rows x 5 columns]
In [12]:
pf.plot()

Out[12]:

retrieve ping API

Conclusion

You have the knowledge of a simple example on how to get started with getting data from NetBeez, doing some transformations, and finally visualizing them. It was pretty easy, right? Before getting into writing this example I have never used Python, but with the help of Jupyter Notebooks, I was able to easily write code and immediately see the results at each step of the way.

All this code is available as a Jupyter Notebook on the NetBeez GitHub account and for a live instance of the notebook, you can check out the nb-api binder link.