Monitoring Script for HTTP-APIs
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Kekskurse 7a4e8b29a3
include shit
4 years ago
data I cannot believe that it took this long to write a test for this. 4 years ago
status Fixed a bug cause Shaun said to 4 years ago
.gitignore Graphite logger 4 years ago
Readme.md include shit 4 years ago
config.json.example ajax-loader hotness, oh yeah 4 years ago
controller.py ajax-loader hotness, oh yeah 4 years ago
logger_graphite.py Fixed a bug cause Shaun said to 4 years ago
logger_stdout.py ajax-loader hotness, oh yeah 4 years ago
requirements.txt Graphite logger 4 years ago
worker.py ajax-loader hotness, oh yeah 4 years ago

Readme.md

Work in progress

HTTP-Status

HTTP-Status is a python script to check/monitor HTTP Endpoints. Its an external monitoring of the Endpoints. It can run in kubernetes or on a regular VM (for example with subervisor) and had one controll script and multible worker.

HTTP-Status is splited in 3 Parts

  • One Controll-Script which push new checks in a redis queue
    • Controller based on a json config file and work without database just ram only
    • Controller based on mysql (todo)
  • One or more Worker-Scripts which make a HTTP-Call and save the result in redis
  • One Logging Script which get the Result and saved it to $something
    • Logger stdout to just debug the System
    • Logger graphite to make graphana stuff
    • Logger to log into mysql (todo)

The Result of the Check contains:

  • HTTP-Status Code
  • Response time from the Python-Request
  • Time of the check defined by the worker

Also HTTP-Status can work with channels, if a check is needet to execute it will publish to all defined channels (or the channels defined in the check itself). Each channel can be another geo-location.

Setup

Kubanetes

ToDo

Ansible

You can use the Ansible Role to install this script with supervisor.

Manuel

Do the following Steops:

  • Install python3, pip3 and redis on the Server
  • Install the python requirements pip3 install requirements.txt
  • Start the Redis Server
  • Create a config.json
  • Start a logger (e.g. python3 logger_stdout.py)
  • Start a worker (e.g. ```python3 worker.py``)
  • Start the controler (e.g. python3 controller.py)
  • Check if results printed out in the logger_stdout window

Supervisor

For the Manuel and the Ansible installation it is may be useful (on Ansible required) to use supervisor to run the scripts. Here is an example config for supervisor

[program:controller]
process_name=%(program_name)s_%(process_num)02d
command=python /path/to/controller.py
autostart=true
autorestart=true
numprocs=1
redirect_stderr=true
stdout_logfile=/var/log/%(program_name)s.log
[program:worker]
process_name=%(program_name)s_%(process_num)02d
command=python /path/to/worker.py
autostart=true
autorestart=true
numprocs=4
redirect_stderr=true
stdout_logfile=/var/log/%(program_name)s.log
[program:logger]
process_name=%(program_name)s_%(process_num)02d
command=python /path/to/logger_graphite.py
autostart=true
autorestart=true
numprocs=1
redirect_stderr=true
stdout_logfile=/var/log/%(program_name)s.log

Config

The config at the default controler based on the config.json. It contains 3 parts:

  • database -> Where the informtion will be stored e.g. graphite
  • redits -> connection to redis, used for internal communaction
  • services -> contains all checks

Database

Where to save the Results, make sure that you run the currect logger for your configuration.

Graphite

"graphite": {
      "hostname": "127.0.0.1", #Hostname
      "prefix": "http" #Name will be PREFIX.SERVICENAME.CHANNELNAME.XY
    }

Redis

The redis config is used for the comunication between the Jobs (Controller, Worker, Logger). It must be the same redis on each Job/Server.

  "redis": {
    "hostname": "127.0.0.1", # Hostname of the Redis to connect to, if you use a VPN use the internal vpn ip
    "port": 6379, # Port of the Redis
    "database": 0, # Which Database should be used, no other script should be use this database
    "password": null # Passwort to conenct
  },

Services

This confiugration is a list of Service-Objects, each Service-Object looks like this:

{
  "name": "Twitter",
  "test": {
    "url": "https://www.twitter.com/kekskurse",
    "method": "GET"
  },
  "checkIntervall": 10,
  "channels": ["default", "helsinki", "newyork"]
},
  • name -> The Name of the Service to identify in grafana
  • test -> the Config of the test itselfs
    • url -> which url should be called
    • method -> which HTTP method should be used (currently just GET)
  • checkIntervall -> After a Check the controller will wait this number of secounds bevor retry it. If its 10 secounds and a check need 3 secounds the URL will be called each 13 Secounds
  • channels -> on which channels (e.g. location) should the chack be called. The checkIntervall will be used per channel, if there are 2 channels with 10 secounds interval the workers will be called the webpage 2 times in 10 secounds.

Naming

In the config you defined "Services". Each Service contains one "Test" which is a URI with a HTTP-Method and may some more informations. Also each Service has one or more "channels".

Based on the List of channels from each Service one or more "Check"s will be created. Each Check is one Service and one Channel. If your "Service" have 3 channels, there will be 3 Checks.

Each "Check" will be executed based on the "Service" interval Time. For the result of the "Check" a "Result" object will be created which contains the HTTP-Status, Response time and so on.

Each "Worker" listen to one "channel". There can be multible "Worker" per "channel".