Internet Speed Monitor - part 2 - basic SW

 Let see in more details how to put together the basic  internet speed monitor contraption.
This article discuss setting up the Raspberry Pi.

Hardware requirements

As said the base is a Raspberry Pi.
I had one around, a 3B+ I think, so with embedded Bluetooth and WiFi, 512K.
Be sure to use at least a 2A power supply since more HW will be added.

I did grab a 32 Gb SD card and installed the latest Raspbian.
Why 32 Gb ?
At the moment I'm planning to host the database that collects the speed tests on the Raspberry Pi so I wanted to have some space available.

What is needed ?
Initially :

• a Raspberry Pi
• a keyboard 
• an HDMI monitor

Keyboard and monitor will be removed after the initialization and set up of the system

Main Software

I opted to install on the Raspberry these packages in order to build the Speedtest contraption, starting from  this article : Raspberry Pi Ookla speed monitor

• speedtest with Ookla
  This service has a bunch of custom apps for different platform, among them a speedtest-cli that can be executed on different Linux based platform.
  I did use in the past this service on different PCs and environment so I find it quite reliable.
  
  
• InfluxDb
  InfluxDB is an open source database time based, i.e. the main key is the time.
  Is perfect for datalogger applications
  
  
• Grafana
  Grafana allows to quickly set up a wen interface to show data and among many sources, can directly use InfluxDb
  
  
• Python scripts
  The article has a couple of Python scripts used to handle the system, reading the speed and storing it in the InfluxDb

Installation

After preparing the SD with the OS :

• instert the SD on the Raspberry Pi
• connect a monitor and keyboard to the Raspberry
• connecte a Cat5e via a switch, then :
• update everything
• sudo apt-get update
• sudo apt-get upgrade
• execute raspi-config (sudo raspi-config)
• enabled SSH
• expanded the filesystem so to use all the SD card
• enabled WiFi on my network
• assigned location to US
• changed password for user pi
• reboot, then :

• install speedtest (from this site)

• sudo apt update
• sudo apt upgrade
• sudo apt install apt-transport-https gnupg1 dirmngr
• wget -q -O - https://packagecloud.io/ookla/speedtest-cli/gpgkey | sudo apt-key add -
• echo "deb https://packagecloud.io/ookla/speedtest-cli/debian/ $(lsb_release -cs) main" | sudo tee  /etc/apt/sources.list.d/speedtest.list
• sudo apt-get update
• sudo apt install speedtest

• install Influx DB (from this site)

• sudo apt update
• sudo apt upgrade
• wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
• echo "deb https://repos.influxdata.com/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
• sudo apt update
• sudo apt install influxdb

• install Grafana (from this site)

• sudo apt update
• sudo apt upgrade
• wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
• echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
• sudo apt update
• sudo apt install grafana

At this point is necessary to configure the packages.

Configuration

• configure speedtest

• first run it. Approve the question with YES
• To run it :
• speedtest

• configure InfluxDb

• sudo systemctl unmask influxdb
• sudo systemctl enable influxdb
• sudo systemctl start influxdb

• configure Grafana

• sudo systemctl enable grafana-server
• sudo systemctl start grafana-server

Python scripts

The article I did follow to set up the basics for the system, propose to use a Python script to execute the Speedtest and save the results.

One script create a CSV file, another one is writing directly in the InfluxDB.
Here the script I use :

import re

import subprocess

from influxdb import InfluxDBClient

response = subprocess.Popen('/usr/bin/speedtest --accept-license --accept-gdpr', shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8')

ping = re.search('Latency:\s+(.*?)\s', response, re.MULTILINE)

download = re.search('Download:\s+(.*?)\s', response, re.MULTILINE)

upload = re.search('Upload:\s+(.*?)\s', response, re.MULTILINE)

jitter = re.search('\((.*?)\s.+jitter\)\s', response, re.MULTILINE)

ping = ping.group(1)

download = download.group(1)

upload = upload.group(1)

jitter = jitter.group(1)

speed_data = [

    {

        "measurement" : "internet_speed",

        "tags" : {

            "host": "speedtest"

        },

        "fields" : {

            "download": float(download),

            "upload": float(upload),

            "ping": float(ping),

            "jitter": float(jitter)

        }

    }

]

client = InfluxDBClient('localhost', 8086, 'speedmonitor', 'pimylifeup', 'internetspeed')

#print('Write points : ',speed_data)

client.write_points(speed_data)

Results

And this is finally the first result, after a couple of days running.
Since I know to have network problems, I set up a running time of 15 minutes, to have a better granularity.

Clearly there are problems on the line, with drops in the bandwidth from 15 Mbp/s to 0.9 Mbp/s