Retrieving NetBeez Data

This tutorial will explore using the NetBeez API to retrieve test data. In this example, we will be pulling ping data from a NetBeez agent and storing the results inside of CSV files. Please note that the NetBeez API has additional data that can be retrieved for agents, alerts, and other tests, such as HTTP and Iperf, and WiFi metrics. The following information will be requested from the API for this tutorial.

• Agent Information
• Test Information
• Test Results

In the next sections we will cover:

• Introduction to the NetBeez API
• How to get started
• Making API requests
  • Importing libraries
  • Getting Agent Data
  • Getting Test Data
  • Getting Test Results
• Calling the functions
  • Defining Variables
  • Making The Agent Retrieval Call
  • Making The Test Retrieval Call
  • Making The Test Results Retrieval Call
• Conclusion

Introduction to the NetBeez API

The NetBeez API allows users the ability to integrate NetBeez data with other solutions such as tools, reporting, and analytics solutions. NetBeez users can make calls to the API to retrieve data by using an API key generated from the NetBeez dashboard. Click here for more information about the NetBeez API. Click here to access a file containing all of the code for this tutorial.

How to get started

You will need the following to follow this tutorial:

• A basic understanding of Python and a way to execute Python code. 
• An API key generated from your NetBeez dashboard. To generate a new API key:

1. Navigate to your NetBeez dashboard
2. Click the Settings icon (cog) in the upper right corner
3. Click the API Keys tab on the left side
4. Click ‘+ Generate API Key’
5. A popup window will appear, enter a description and choose a role
6. Click ‘Generate New API Key’

• FQDN of your NetBeez dashboard.

Making API requests

Importing Libraries

First, we will need to import the libraries we will be using for this tutorial:

import time

import json

import requests

import urllib3

import pandas as pd

Getting Agent Data

The first function will be an API call to retrieve the agent data. Locate an agent by name on the dashboard:

This name will be one of the input parameters for our get agents function along with the FQDN and API key. We will input this name later on in the tutorial.

def get_agent (agent_name, fqdn, api_key):
    beezkeeper_fqdn = fqdn 
    base_url='https://' + beezkeeper_fqdn

    urllib3.disable_warnings()

    headers = {
                'Cache-Control': 'no-store',
                'Content-type': 'application/json',
                'Authorization': 'Bearer ' + api_key
              }

    search_name = agent_name
    url = f"{base_url}/agents?filter[name]={search_name}&type=beta"

    response = requests.request("GET", url, headers=headers, verify=False)
    data = json.loads(response.text)
    agent = data['data'][0]
    
    return agent

This agent API call will return JSON with the following information:

{'attributes': {'active': True,
                'active_ts': 1661790148787,
                'agent_class': 'windows',
                'agent_type_name': 'windows',
                'category': 'remote_worker_agent',
                'created_at': '2022-04-20T15:55:37.000Z',
                'description': None,
                'isp_asn': 'AS7018',
                'isp_name': 'AT&T Services, Inc.',
                'kernel_version': '2020-06-01 08:59 UTC',
                'name': 'Panos TV Stick',
                'nb_target_ids': [24, 11, 19, 10, 5],
                'nb_test_ids': [444392],
                'open_incident_id': None,
                'os_version': 'CYGWIN',
                'reached_scheduled_test_limit': False,
                'reached_target_test_limit': False,
                'software_version': '10.2.2',
                'test_aggregate_counts_by_type': {'1': {'fail': 0,
                                                        'paused': 0,
                                                        'success': 6,
                                                        'unknown': 0,
                                                        'warning': 0},
                                                  '2': {'fail': 0,
                                                        'paused': 0,
                                                        'success': 3,
                                                        'unknown': 0,
                                                        'warning': 0},
                                                  '3': {'fail': 0,
                                                        'paused': 0,
                                                        'success': 3,
                                                        'unknown': 0,
                                                        'warning': 0},
                                                  '4': {'fail': 0,
                                                        'paused': 0,
                                                        'success': 1,
                                                        'unknown': 0,
                                                        'warning': 0}},
                'uuid': '06A2E3D5'},
 'id': '66',
 'relationships': {'agent_groups': {'data': [{'id': '6', 'type': 'agent_group'},
                                             {'id': '5',
                                              'type': 'agent_group'}]},
                   'network_interfaces': {'data': [{'id': '108',
                                                    'type': 'network_interface'},
                                                   {'id': '109',
                                                    'type': 'network_interface'}]},
                   'scheduled_nb_test_templates': {'data': [{'id': '16',
                                                             'type': 'scheduled_nb_test_template'}]}},
 'type': 'agent'}

Getting Test Data

The next function will be an API call to retrieve the test data. The results of the agent API call will be passed as an input parameter along with the type of test (ping, dns, or http), the FQDN, and API key. From the agent data, the agent ID is needed for the API query to retrieve the test data. Let us define the get_tests function now:

def get_tests (agent, test_type, fqdn, api_key):
    beezkeeper_fqdn = fqdn 
    base_url='https://' + beezkeeper_fqdn
    
    api_headers = {
        'Cache-Control': 'no-store',
        'Content-type': 'application/json',
        'Authorization': 'Bearer ' + api_key
    }
    
    url = f"{base_url}/nb_tests?filter[agents]={agent['id']}&filter[test_types]={test_type}&type=beta"
    response = requests.request("GET", url, headers=api_headers, verify=False)
    data = json.loads(response.text)
    data['data']
            
    return data['data']

This test API call will return JSON with the following information:

[{'attributes': {'ad_hoc': False,
                 'count': 0,
                 'created_at': '2021-06-14T13:10:31.000Z',
                 'current_alert_mode': 'success',
                 'current_state': 'running',
                 'current_state_transition_ts': 1623676306717,
                 'interface_type': 'wireless',
                 'interval': 5,
                 'last_sequence_number': 0,
                 'options': {'data_size': 54,
                             'dont_fragment': False,
                             'flags': None,
                             'jitter_mos': True,
                             'ping_type': 2,
                             'port': None,
                             'tos': None},
                 'schedule_type': 'periodic',
                 'start_ts': '2021-06-14T09:10:31.457-04:00',
                 'target': 'demo-app.netbeezcloud.net',
                 'timeout': 5},
  'id': '300715',
  'relationships': {'agent': {'data': {'id': '7', 'type': 'agent'}},
                    'alert_detector_instances': {'data': [{'id': '3358',
                                                           'type': 'alert_detector_instance'},
                                                          {'id': '3359',
                                                           'type': 'alert_detector_instance'}]},
                    'nb_target': {'data': {'id': '2', 'type': 'nb_target'}},
                    'nb_test_template': {'data': {'id': '7',
                                                  'type': 'nb_test_template'}},
                    'network_interface': {'data': {'id': '10',
                                                   'type': 'network_interface'}},
                    'ping_test': {'data': {'id': '347', 'type': 'ping_test'}},
                    'test_type': {'data': {'id': '1', 'type': 'test_type'}},
                    'wifi_profile': {'data': {'id': '7',
                                              'type': 'wifi_profile'}}},
  'type': 'nb_test'}]

Getting Test Results

The last function we will need is a call to retrieve the test results. The input parameters will be the test information returned from the previous call, test type (ping, dns, http), FQDN, API key, limit for the API call (number of results), ‘to’ timestamp, and the ‘from’ timestamp. This function will retrieve the test results from the NetBeez API and store the values in a Pandas Dataframe.

def get_test_results (nb_test, test_type, fqdn, api_key, limit, to_ts, from_ts):
    beezkeeper_fqdn = fqdn 
    base_url='https://' + beezkeeper_fqdn
    
    api_headers = {
        'Cache-Control': 'no-store',
        'Content-type': 'application/json',
        'Authorization': 'Bearer ' + api_key
    }
    
    results = []

    offset = 1
    result_data = True
    
    while result_data:
        url = f"{base_url}/nb_tests/{test_type}/results?filter[agents]={str(nb_test['relationships']['agent']['data']['id'])}&page[limit]={str(limit)}&filter[ts][value1]={str(from_ts)}&filter[ts][value2]={str(to_ts)}&filter[ts][operator]=<=>&page[offset]={str(offset)}&type=beta"

        response = requests.request("GET", url, headers=api_headers, verify=False)
        result_dict = json.loads(response.text)

        result_data = result_dict['data']
        try:
            for result in result_data:
                temp = {
                    'agent_id': nb_test['relationships']['agent']['data']['id'],
                    'target_id': nb_test['relationships']['nb_target']['data']['id'],
                    'test_template_id': nb_test['relationships']['nb_test_template']['data']['id'],
                    'target_name': nb_test['attributes']['target'],
                    'interface_type': nb_test['attributes']['interface_type'],
                    'network_interface_id': nb_test['relationships']['network_interface']['data']['id'],
                    'error_code': result['attributes']['error_code'],
                    'ts': result['attributes']['ts'],
                    'value': result['attributes']['value']
                }
                results.append(temp)
        except:
            pass
        offset += 1
    df = pd.DataFrame(results)
    return df

Returned Dataframe:

Calling The Functions

Defining Variables

With all of our functions defined, it is time to call our functions. First, we will need to define our variables used as input parameters for our functions:

• agent_name: set the agent name to a string with the name of the agent you wish to query.
• fqdn: set the fqdn to a string of your dashboard’s fqdn.
• api_key: set the API key with an API key generated from your dashboard.
• hours: set the hours to an integer of how many hours into the past you would like to pull data from.
• test_type: set test type to a string of the test results you wish to retrieve: ‘ping’, ‘dns’, ‘http’.
• limit: set the limit to the number of API results you wish to retrieve at one time as an integer.
• to_ts & from_ts: leave these variables as is, you can modify with your own ‘to’ and ‘from’ timestamps if you choose.

Example:

agent_name = 'Office 1 - WiFi'
fqdn = 'demo1.netbeezcloud.net'
api_key = 'gERo9mT0qPF0Pz6BGFwi4Y13IUJ6SoGeq'
hours = 1
test_type = 'ping'
limit = 100
to_ts = int(time.time() * 1000)
from_ts = to_ts - (hours * 60 * 60 * 1000)

Making The Agent Retrieval Call

With our variables defined, we will now call the get_agent function and store it in a variable called agent_data:

agent_data = get_agent(agent_name, fqdn, api_key)

Making The Test Retrieval Call

Using the agent_data variable, we will now call the get_tests function and store it in a variable called nb_tests:

nb_tests = get_tests(agent_data, test_type, fqdn, api_key)

Making The Test Results Retrieval Call

Lastly, we will loop through the list of nb_tests and call the get_test_results function on each element of the list and save the test results into a variable called temp_df. We will then export the data to a CSV file. After this function is called, there will be a CSV generated for each test in the nb_tests variable.

for test in nb_tests:
    temp_df = get_test_results(test, test_type, fqdn, api_key, limit, to_ts, from_ts)
    temp_df.to_csv(test['attributes']['target'] + '_data.csv', index=False)

Conclusion 

The NetBeez API is a great tool to further explore network performance and the documentation can help with the implementation. Also, check out my previous blog post on working with NetBeez data using Pandas.