The NetBeez API and PING results
NetBeez gathers performance data from multiple points in your networks, by deploying network monitoring sensors or agents (known as Beez), which conduct continuous end-to-end tests against multiple targets in your network using a multitude of tools (ping, DNS, http, traceroute, speed test, iperf etc). The data is available to be viewed on the web-based dashboard, but sometimes your specific needs may require some extra data analysis to understand exactly the performance of your network from the end-user’s point of view.
One use-case could be that you want to view some data in a way that the dashboard doesn’t readily provide at the moment. Then this information can be used to generate reports that could help you make decisions on future expansions, or report the status of your network to upper management.
In this article we will go through the steps of setting up the necessary variables, to make REST API requests to a BeezKeeper instance.
More documentation about the NetBeez API can be found here: https://api.netbeez.net (and for the legacy API which will eventually be fully replaced by the one just mentioned: https://demo.netbeezcloud.net/swagger/)
How to get started with the NetBeez API
Before you can start using the NetBeez API to get network performance data, you’ll need to perform two actions:
- Setup your NetBeez BeezKeeper details
- Setup the headers
Let’s review each one of these steps in more detail.
Setup your NetBeez BeezKeeper details
First, you need to identify your server’s FQDN and generate an API key to use in these examples.
If the URL you use to access NetBeez looks like this https://demo.netbeezcloud.net then demo.netbeezcloud.net is the fqdn.
In [1]:
beezkeeper_fqdn="demo.netbeezcloud.net"
base_url='https://' + beezkeeper_fqdn
In [2]:
from IPython.display import Markdown as md
md(f"To generate an API key head over to https://{beezkeeper_fqdn}/#settings-tab/api-keys-settings")
md(f"For documentation go to https://api.netbeez.net or https://{beezkeeper_fqdn}/swagger/")
Out[2]:
For documentation go to https://api.netbeez.net or https://demo.netbeezcloud.net/swagger/
In [3]:
api_key="7g5_hKXw_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
Set Up the Headers
We define the content-type format as JSON and provide the API key that you generated in the previous step as part of the headers.
In [4]:
import requests
import urllib3
urllib3.disable_warnings()
headers = {
'Cache-Control': 'no-store',
'Content-type': 'application/json',
'Authorization': 'Bearer ' + api_key
}
legacy_api_headers = {
'Authorization': api_key,
'API-VERSION': 'v1',
'Content-type': 'application/json'
}
Note that when the legacy API headers are used it means that we are using the legacy API which is subject to be replaced soon by the new JSON API (documentation here: https://api.netbeez.net)
How to Retrieve Ping Results via the API
In this example, we will be searching for an agent by name, then selecting the ping tests that are currently running, and then retrieving the raw measurement results for those tests from the past 60 minutes.
Retrieve the agent
First, we set ‘retrieve the agent’ by search by name using this API endpoint: https://api.netbeez.net/#f3d48709-5344-45b7-af9b-7b92feddd4ec.
In [5]:
search_name = "Virtual Agent"
url = f"{base_url}/agents?filter[name]={search_name}&type=beta"
In [6]:
response = requests.request("GET", url, headers=headers, verify=False)
import json
data = json.loads(response.text)
agent = data['data'][0]
print(agent)
{'id': '341', 'type': 'agent', 'attributes': {'name': 'Pittsburgh - Virtual Agent', 'reached_target_test_limit': False, 'reached_scheduled_test_limit': None, 'nb_target_ids': [463, 464, 465, 437, 32], 'nb_test_ids': [1667068, 1667069, 1667070, 1667065, 1667066, 1667067, 1667071, 1688201, 1688302, 1688303, 1688304, 1688305, 1688306, 1688391, 1688392, 1688393, 1688394, 1688395, 1688474, 1688475, 1688476, 1688477, 1688478], 'agent_type_name': 'virtual-model-ova', 'os_version': 'Debian GNU/Linux 8.11 (jessie)', 'kernel_version': '#1 SMP Debian 3.16.51-3 (2017-12-13)', 'uuid': '00:0c:29:09:fa:ee', 'agent_class': 'virtual', 'active': True, 'active_ts': 1624631782781, 'description': None, 'category': 'network_agent', 'software_version': '8.0.6', 'open_incident_id': None, 'test_aggregate_counts_by_type': {'1': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '2': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '3': {'success': 5, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}, '4': {'success': 4, 'fail': 0, 'warning': 0, 'paused': 0, 'unknown': 0}}}, 'relationships': {'network_interfaces': {'data': [{'id': '387', 'type': 'network_interface'}]}, 'agent_groups': {'data': [{'id': '5', 'type': 'agent_group'}]}}}
Retrieve ping tests
Then we iterate through the tests and select the ping tests.
In [7]:
nb_tests = []
for nb_test_id in agent['attributes']['nb_test_ids']:
url = f"{base_url}/nb_tests/{nb_test_id}.json"
response = requests.request("GET", url, headers=legacy_api_headers, verify=False)
data = json.loads(response.text)
if data['heir_type'] == 'PingTest':
print(data)
nb_tests.append(data)
{'id': 1667068, 'target': 'www.google.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-04-01T15:39:11.736-04:00', 'created_at': '2021-04-01T19:39:11.000Z', 'updated_at': '2021-06-03T15:34:27.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11374, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': None, 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 99, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 32, 'nb_test_template_id': 99, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': True}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1667065, 'target': 'demo-app.netbeezcloud.net', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-04-01T15:39:11.604-04:00', 'created_at': '2021-04-01T19:39:11.000Z', 'updated_at': '2021-06-03T15:34:27.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11373, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 1830, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 437, 'nb_test_template_id': 1830, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': False}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688303, 'target': 'baidu.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-17T23:56:56.125-04:00', 'created_at': '2021-05-18T03:56:56.000Z', 'updated_at': '2021-07-05T15:02:15.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11441, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2074, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 463, 'nb_test_template_id': 2074, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688391, 'target': 'in.gr', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-17T23:57:16.957-04:00', 'created_at': '2021-05-18T03:57:16.000Z', 'updated_at': '2021-07-05T18:12:00.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11461, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2078, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 464, 'nb_test_template_id': 2078, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}
{'id': 1688474, 'target': 'amazon.com', 'count': 0, 'interval': 5, 'timeout': 5, 'start_ts': '2021-05-18T00:00:42.578-04:00', 'created_at': '2021-05-18T04:00:42.000Z', 'updated_at': '2021-06-11T06:27:10.000Z', 'agent_id': 341, 'test_type_id': 1, 'heir_id': 11477, 'heir_type': 'PingTest', 'ad_hoc': False, 'test_name': '', 'nb_test_templatable_type': 'NbTestTemplate', 'nb_test_templatable_id': 2083, 'current_alert_mode': 'success', 'wifi_profile_id': None, 'network_interface_id': 387, 'nb_target_id': 465, 'nb_test_template_id': 2083, 'schedule_type': 'periodic', 'options': {'data_size': 54, 'dont_fragment': False, 'tos': None, 'port': None, 'ping_type': 2, 'flags': None, 'jitter_mos': None}, 'alert_mode': 'success', 'interface_type': 'wired'}
Retrieve the results
Then we retrieve the results for the past 60 minutes.
In [8]:
import time
import datetime
to_ts = int(time.time() * 1000)
from_ts = to_ts - (60 * 60 * 1000)
print(datetime.datetime.fromtimestamp(from_ts/1000.0))
print(datetime.datetime.fromtimestamp(to_ts/1000.0))
2021-07-06 02:18:17.003000
2021-07-06 03:18:17.003000
In [9]:
results = []
import pandas as pd
for nb_test in nb_tests:
url = f"{base_url}/results.json?nb_test_id={nb_test['id']}&from={from_ts}&to={to_ts}"
response = requests.request("GET", url, headers=legacy_api_headers, verify=False)
df = pd.json_normalize(json.loads(response.text), 'index')
if df.empty == False :
df['timestamp'] = df['timestamp'].transform(lambda x: datetime.datetime.fromtimestamp(x/1000.0))
df.set_index('id', inplace=True)
results.append(df)
results = pd.concat(results)
…and here is a printout of the results:
In [10]:
print(results)
value timestamp sequence_number nb_test_id
id
9905711541 12.10 2021-07-06 02:18:18.637 1643884 1667068
9905711772 20.40 2021-07-06 02:18:23.852 1643885 1667068
9905711878 20.40 2021-07-06 02:18:28.856 1643886 1667068
9905712156 20.10 2021-07-06 02:18:33.858 1643887 1667068
9905712268 38.40 2021-07-06 02:18:38.865 1643888 1667068
... ... ... ... ...
9905840070 9.75 2021-07-06 03:17:53.055 844807 1688474
9905840241 9.57 2021-07-06 03:17:58.073 844808 1688474
9905840449 9.75 2021-07-06 03:18:03.077 844809 1688474
9905840669 9.74 2021-07-06 03:18:08.082 844810 1688474
9905840809 9.67 2021-07-06 03:18:13.083 844811 1688474
[3595 rows x 4 columns]
In [11]: results_pivot = results.pivot(index='timestamp', columns='nb_test_id', values='value') pf = results_pivot.interpolate() print(pf) nb_test_id 1667065 1667068 1688303 1688391 1688474 timestamp 2021-07-06 02:18:18.194 NaN NaN NaN NaN 9.340 2021-07-06 02:18:18.637 NaN 12.10 NaN NaN 9.376 2021-07-06 02:18:18.871 10.60 13.76 NaN NaN 9.412 2021-07-06 02:18:18.881 10.58 15.42 NaN 105.0 9.448 2021-07-06 02:18:19.188 10.56 17.08 214.0 104.5 9.484 ... ... ... ... ... ... 2021-07-06 03:18:13.083 10.42 20.90 223.8 104.0 9.670 2021-07-06 03:18:13.394 10.40 21.10 220.6 104.0 9.670 2021-07-06 03:18:13.465 10.40 21.30 217.4 104.0 9.670 2021-07-06 03:18:13.587 10.40 21.30 214.2 104.0 9.670 2021-07-06 03:18:13.904 10.40 21.30 211.0 104.0 9.670 [3415 rows x 5 columns]
In [12]: pf.plot() Out[12]:
Conclusion
You have the knowledge of a simple example on how to get started with getting data from NetBeez, doing some transformations, and finally visualizing them. It was pretty easy, right? Before getting into writing this example I have never used Python, but with the help of Jupyter Notebooks, I was able to easily write code and immediately see the results at each step of the way.
All this code is available as a Jupyter Notebook on the NetBeez GitHub account and for a live instance of the notebook, you can check out the nb-api binder link.
