Home Features History Support Docs Code People

OpenWISP

Home Features History Docs Support Code People

OpenWISP

Google Summer of Code 2020 Idea List

Do you want to apply with us? We have a page that describes how to increase your chances of success. Please read it carefully.

Read our GSoC Quick Start!

General suggestions and warnings

  • some details may be missing from the project idea: we expect students to do their own research, propose solutions and be ready to deal with uncertainty and solve challenges that may come up during the project
  • code and prototypes are preferred over detailed documents and unreliable estimates: rather than wasting your time on writing a very long application document, we suggest you to invest in writing a prototype (which means the code may be thrown out entirely) which will help you understand the challenges of the project you want to work on; your application should refer to the prototype or other github contributions you made to OpenWISP that show you have the capability to succeed in the project idea you are applying for
  • accepted students will have to create github issues and a kanban board on github for their project: the requirements listed in the idea page will have to be converted into github issues which shall be tracked from a kanban board on github to allow the openwisp community to easily keep track of the project progress
  • students who have shown to have the required skills by contributing to OpenWISP have a lot more chances of being accepted.
    In order to get started contributing refer to the OpenWISP Contributing Guidelines.
    Once applicants have completed some basic training, they may start to work on some aspects of the project they are interested in applying: all projects listed this year are improvements of existing modules so it is already possible to start working and complete some of the tasks listed in the project idea right now.

Merge OpenWISP django modules

Languages & technologies used: python, django.

OpenWISP 2 is composed of several python packages, some of which are decoupled from the rest of OpenWISP, this has been done for good reasons which are described in the following section of our documentation: OpenWISP values: software reusability means long term sustainability .

After some years in which OpenWISP 2 has been deployed and maintained we came to the conclusion that in some cases we went a bit too far with this practice, which has resulted counterproductive in some cases because it increases maintenance overhead and complexity.

The goal of this project idea is to merge some modules which perform identical operations (although with some distinction) in order to bring down the complexity of OpenWISP, hence lowering maintenance costs while keeping the extensibility and reusability of these key modules.

These modules are:

Each of this pair performs the same function, the package prefixed with "openwisp-", for example "openwisp-ipam" extends a base package (in the case of this example it's django-ipam) and adds features that are specific to OpenWISP, like multitenancy (different organization using the same application without interfering with one another).

This separation over time is brought more complexity than advantages to the project, so it's now time to put a stop to it. We just have to make sure each OpenWISP module is still extensible and reusable so other developers can use it as a base for their custom applications.

Measurable outcomes:

  • Merge django-ipam into openwisp-ipam
  • Merge django-netjsonconfig into openwisp-controller
  • Merge django-netjsongraph into openwisp-network-topology
  • Merge django-freeradius into openwisp-radius
  • Migrate the django settings of each module so that the naming is consistent. For example, all settings of django netjsonconfig are prefixed as NETJSONCONFIG_[SETTING_NAME] and have to be renamed as OPENWISP_CONTROLLER_[SETTING_NAME] but backward compatibility with the old name must be kept in order to facilitate migration: the old name should be checked first, if a setting with the old naming is present, the application should use it but also log a warning to encourage the users to upgrade the name of the setting.
  • Ensure django-swappable-models are implemented in each merged module and the naming of the swappable settings is consistent.
  • Provide ways to import, extend and override each key area of the application. Examples: extending django-netjsongraph, Extending django-netjsonconfig.
  • Ensure tests of each merged module are reusable as it's currently done in django-ipam and django-freeradius
  • Each module shall have has a sample_app in its test suite that extends the main module, adding some changes on top of it with the sole purpose of testing its extensibility.
    In order to better understand this concept take a look at how it's currently implemented in django-freeradius. and django-ipam (we must port the same concept into the merged modules).
  • Update the documentation of each merged module (eg: documentation of django-netjsonconfig has to be merged in openwisp-controller and adapted).
  • Ensure all tests pass and test coverage remains stable.
  • Create a branch on ansible-openwisp2 which makes it work with the new merged modules.
  • Create a branch on docker-openwisp which makes it work with the new merged modules.

Improving OpenWISP Monitoring towards its first release

Languages & technologies used: python, django, timeseries DB, javascript.

The monitoring module of OpenWISP has not yet been released officially, but unreleased version is available in a github.com repository, the goal of this project idea is to bring important improvements to that unreleased module in order to arrive to a first release and include openwisp-monitoring in the OpenWISP ecosystem.

Measurable outcomes:

  • Implement swappable models. See django-freeradius as a reference (look for swappable in the code).
  • Abstract the code that communicates with the timeseries DB so that new timeseries backends can be written to support other timeseries DBs.
  • Add support for another timeseries DB, it could be prometheus, opentsdb or another good option if there's any.
    The decision regarding which one to choose must be backed up by convincing research.
  • Right now the query parameter of a chart (now called Graph) is editable via the django admin. This poses several issues: from UX (it's hard to deal with it) to security (the query could be manipulated to get data to which the user does not have access to). We need to refactor this part so that the user can choose among a predefined set of queries, but the list should be customizable using a django setting, so that users can implement their own queries if needed.
  • The Metric model allows to store different aspects of a metric, for example, the ping metric allows to store the "reachable" result (1 for reachable and 0 for unreachable), the packet loss and the round trip information (rtt_min, rtt_max and rtt_average).
    Each aspect is stored with a different key in the same metric object of the timeseries DB, but the django model does not show this information and this makes some operations harder (eg: automatically creating charts which uses some or all the aspects of the metric).
    We need to find a solution to this problem, a possible solution would be to introduce the concept of a metric type and have this kind of general information stored on the metric type, provided metric types are extensible and users can define their own types (similarly to the chart query problem listed in the previous point).
  • Rename the model which is now called Threshold to AlertSettings, in a backward compatible way (eg: provide a data migration which creates the alert table first, copies the data from the threshold table into alert and then remove the old threshold table, or an equivalently working alternative).
  • Rename the model which is now called Graph to Chart, in a backward compatible way (eg: provide a data migration which creates the chart table first, copies the data from the graph table into chart and then remove the old graph table, or an equivalently working alternative).
  • Add a way to disable alerts.
  • Add an "Alert Settings" section in the device admin which allows to see what alerts are available for a device and which also allow to modify the threshold or disable/enable the alarm.
  • Add a "Monitoring Checks" section in the device admin which allows to see what checks are available for a device and add new ones if needed.
  • Add possibility to collect memory (RAM), CPU and flash (disk space) usage of devices via the API.
    The JSON data format supported by the API should follow the NetJSON DeviceMonitoring spec. Automatically create different metrics and charts (one for memory usage, one for CPU usage, one for flash space usage) whenever this data is received.
    Automatically create a related alert setting for each metric which creates an alert (and sets the device monitoring status to "problem") if the total value of each metric goes above X, where X is configurable for each metric with a different setting and defaults to: 95% for RAM, 90% for CPU and 80% for flash usage.
  • Add a check, metric and a relative alert setting (created automatically for every device, must take into account existing devices by using a data migration) that is run periodically (by default every 10 minutes). The check shall find out if the configuration related to the device has been in the "modified" state for more than X minutes, where X is configurable with a setting and defaults to 5.
    It should also change the monitoring status of the device from "ok" to either "problem" or "critical", depending on configuration (defaults to "problem").
    The check should not fail but skip if the device has no defined configuration.
    Data stored in the timeseries DB which is older than X days can be deleted, where X is configurable with a django setting and defaults to 2 days.
  • Add a check, metric and a relative alert settings that will be available only if openwisp-network-topology is in INSTALLED_APPS (created automatically for every topology, must take into account existing topologies by using a data migration) that is run periodically (by default every 5 minutes).
    The check shall write to the timeseries DB the number of links that are down, the number of links that are up, the total number of nodes and the number of isolated nodes (nodes without links).
    It should then check if the topology has X % or more of its links in "down" state, in which case it should send an alert. X should be configurable with a django setting and should default to 50%.
    Two charts should be created: one which shows the amount of links (one line for the up state and one line for the down state) and another chart for the amount of nodes (one line for total nodes and one line for isolated nodes).
  • Prepare a travis-ci build which runs tests with both timeseries DB supported, performs QA checks (like other equivalent openwisp modules) and collects test coverage data.
  • Locate slow tests (tests taking longer than 0.3 secs) and find out if mocking can be used to optimize them.
  • Add possibility of exporting data with histogram charts.
  • Ensure test coverage is above 95%.
  • Ensure the README is up to date, lists all the features available, provides some screenshots and a detailed explanation of the usage of the main features added in this project.

OpenWISP Notifications Module

Languages & technologies used: python, django, javascript.

This project idea consists of extracting the notification module currently implemented in the alpha openwisp-monitoring so that it lives on its own and can be more easily integrated with other openwisp modules.

Measurable outcomes:

  • Extract the notification module into its own module and update the code of openwisp-monitoring to make use of the new module.
    At some point early in the project we have to merge this change into openwisp-monitoring so that the decoupling is completed as soon as possible.
  • Add the concept of notification type, it may be a list with some default values which can then be extended by each module.
    The first notification types implemented shoud be the ones used by openwisp-monitoring. This logic shall reside in a sub app of openwisp-monitoring. Each notification type shall have a message which is generated from a configurable text template, the text shall be markdown formatted, the markdown text shall be expanded into HTML when the notification is sent via email or viewed via the JS widget.
  • Allow users to disable notifications by notification types, right now it's only possible to enable or disable the notifications globally.
    For each notification type and organization the user should be able to disable either all notifications or only the email notification.
    Example: Receive notifications for organization "default"?
    Two checkboxes: Web and Email; if email is checked also web is checked automatically (because the notification depends on the basic notification being created).
    Such a solution would give us the possibility to implement mobile push notifications in the future by adding a new checkbox for mobile notifications.
  • Prepare a general email notification template which is configurable (can be changed with a setting) and defaults to using the default logo of OpenWISP (available in openwisp_utils.admin_theme). Make the logo configurable and add a way to supply extra CSS if needed.
  • Create a javscript widget for the admin which allows to view all notification and expand their details by clicking on it.
    This widget shall support infinite scrolling (we have a nice implementation in django-ipam which can be reused, either partially or totally) and expand the markdown into HTML.
    The operation of opening a notification detail shall flag it as read.
    This feature will require adding API endpoints to get notification list and details.
  • Allow users to disable notifications for specific objects, either permanently or until a specific date, with predefined set of durations: permanently, 1 day, 3 days, 1 week, 1 month, custom date. In the case the disabling is temporary, the setting should be deleted automatically after expiration by a simple celery task which is run periodically with celery beat.
  • Add celery beat task to automatically delete notifications older than X days, X defaulting to 90, configurable via setting.
  • Make the notifications module an optional dependency of openwisp-controller (by creating a sub app in openwisp-controller) and create the following notification types in it:
    • new devices registered into the system;
    • configuration errors and their recovery (has to be triggered when the configuration status of a device moves into the "errored state" or when it recovers).
  • Make the notifications module an optional dependency of openwisp-network-topology (by creating a sub app in openwisp-network-topology) and create the following notification types in it:
    • link status changed
    • new node in the topology
  • Prepare a travis-ci build which run tests, performs QA checks (like other equivalent openwisp modules) and collects test coverage data.
  • Ensure test coverage is above 95%.
  • Ensure the README is up to date, lists all the features available, provides some screenshots and a detailed explanation of the usage of the main features of this module.

Netengine: pull monitoring info from network devices

Languages & technologies used: python, NetJSON, SNMP, HTTP, OpenWrt, Linux.

Netengine is a python library that aims to provide a single API to extract common information from network devices using different protocols (eg: SNMP, SSH, HTTP) and different firwmares (eg: OpenWrt, AirOS).

The library kinda worked but was not brought to production level work and since OpenWISP 2 is going to have a monitoring system soon, wa want to rewrite this library to bring its implementation to production level.

NOTE: backward compatibility does not have to be maintained we can freely change and improve the API and internal structure of the library if we think the change is going to improve the quality of the implementation.

WARNING: to work on this project you need an AirOS device; OpenWrt can be installed in a virtual box VM, but AirOS cannot (at least not as easily). If you don't have an AirOS device compatible with the latest version and you do not indend to buy one you cannot work on this project.
If it ever happens that you get accepted but you fail to buy an AirOS device, you will be failed.

Measurable outcomes:

  • change the output format of the to_json method to NetJSON DeviceMonitoring
  • mock all external network calls during tests (SSH, HTTP, SNMP)
  • upgrade the OpenWrt backends to make it work well on the latest two versions of OpenWrt; if there are incompatibilities, create different backends for each version, ensuring the shared logic is stored in a common class
  • upgrade the AirOS backends to make it work with the latest 2 version of AirOS; if there are incompatibilities, create different backends for each version, ensuring the shared logic is stored in a common class
  • achieve 95% test coverage
  • improve documentation: at the moment the documentation is really scarce, we need the documentation to mention all the backends and all the important features, providing also a few examples of how to use the library
  • set up a travis build that performs tests and checks test coverage
  • implement a check class in openwisp-monitoring that makes use of netengine to collect metrics via SNMP

NOTE: this is one of those projects that sounds easy but is not.

Apply with us

Do you want to apply with us? We have a page that describes how to increase your chances of success. Please read it carefully.

Get started!