InfluxDB 2.9 Support

The first major feature implemented during this project was support for InfluxDB 2.9 as a time-series database backend for OpenWISP Monitoring.

At the beginning, the task looked straightforward: add a new database client, wire it into the existing settings, and make the monitoring code write data to InfluxDB 2.9. However, once the implementation started, it became clear that InfluxDB 2.9 is not just a drop-in replacement for the older InfluxDB 1.8 backend. It felt like I had opened a Pandora's box.

InfluxDB 1.8 uses InfluxQL and retention policies, while InfluxDB 2.9 uses Flux queries and buckets. This meant that the existing chart queries could not simply be reused. The new backend needed its own query layer, while still returning data in the shape expected by the existing OpenWISP Monitoring charts, checks, and alert logic.

One of the most challenging parts was working with Flux queries. Writing the first queries was manageable, but translating all the chart behavior correctly took much more effort. The queries had to support different chart types, summary views, grouped data, top fields, device data, and custom chart configuration without changing the higher-level monitoring code.

Another important challenge was handling timezones correctly. Charts that use daily or weekly windows must align their buckets with the timezone selected by the user, not always with UTC. This required careful handling of Flux window and aggregateWindow queries so the chart range and aggregation interval remained separate. Several review comments helped catch these edge cases and improve the correctness of the final implementation.

The backend also had to map OpenWISP's existing retention policy model to InfluxDB 2.9 buckets. Since InfluxDB 2.9 does not support multiple retention policies inside the same bucket in the same way as InfluxDB 1.8, the implementation uses bucket naming conventions to preserve the behavior expected by OpenWISP Monitoring.

During staging and review, a few additional issues surfaced around query escaping, read/delete behavior, UDP writes through Telegraf, test stability, and compatibility with existing imports. These issues were resolved iteratively with feedback from the maintainers, making the backend more robust and closer to production-ready behavior.

The final implementation added:

• A new influxdb2 backend implementing the shared time-series client interface.
• Flux-based chart, summary, device data, and retention support.
• HTTP writes, optional UDP writes through Telegraf, tests, and documentation for the new backend.

For the full technical discussion, implementation details, and maintainer review comments, see openwisp-monitoring pull request #801.

Elasticsearch 9 Support

The second major feature was adding Elasticsearch 9 as another time-series backend for OpenWISP Monitoring.

This part of the project needed more design work before writing the final implementation. Elasticsearch is traditionally known as a search and analytics engine, especially for keyword search and log data. In recent versions, it also provides features for storing time-series data, but there are different ways to model that data. We spent time deciding whether OpenWISP should use Elasticsearch Time Series Data Streams (TSDS) or regular data streams. TSDS looked attractive at first, but with maintainer feedback we found it was too restrictive for OpenWISP's needs: WiFi metrics include non-numeric values such as MAC addresses, device data can contain JSON snapshots, and OpenWISP must support delayed writes from devices that were offline for hours, days, or even longer. So we decided to use regular Elasticsearch data streams by default.

After deciding the storage model, the implementation focused on making Elasticsearch fit into the same time-series interface used by the InfluxDB backends. The new backend stores metric points in data streams, creates index templates and ILM lifecycle policies for retention handling, and uses Elasticsearch Query DSL and aggregations to power OpenWISP's existing charts without changing the higher-level monitoring code.

This involved implementing write, batch write, read, delete, device data, retention policy, chart query, and summary query support. We also had to handle Elasticsearch-specific details such as stable operation IDs for Celery retries, type-safe mappings for values like MAC addresses, timestamp precision for deleting individual points, exact distinct counts where OpenWISP needs them for health checks, and safe cleanup of resources created by the backend.

The final implementation added:

• A new elasticsearch backend implementing the shared time-series client interface.
• Elasticsearch 9-compatible connection, authentication, and TLS configuration.
• Data streams, index templates, ILM retention, chart queries, reads, writes, deletes, tests, and documentation.

For the full technical discussion, implementation details, and maintainer review comments, see openwisp-monitoring pull request #829.

At runtime, device data moves through the monitoring API and Celery tasks before reaching the common client interface. Each backend adapter handles its own storage and query language, while the monitoring layer receives the same output for charts, alerts, and API responses.

flowchart LR
    ROUTER["OpenWrt router"] --> API["Monitoring API"]
    API --> CELERY["Celery tasks"]
    CELERY --> MODELS["Metric / Chart / DeviceData"]
    MODELS --> ABC["BaseTimeseriesClient"]

    SETUP["Database and retention setup"] --> ABC

    ABC --> INFLUX1_ADAPTER["InfluxDB 1 adapter"]
    ABC --> INFLUX2_ADAPTER["InfluxDB 2 adapter"]
    ABC --> ES_ADAPTER["Elasticsearch adapter"]

    INFLUX1_ADAPTER --> INFLUX1["InfluxDB 1.8"]
    INFLUX2_ADAPTER --> INFLUX2["InfluxDB 2.9"]
    ES_ADAPTER --> ES["Elasticsearch 9"]

    INFLUX1 --> OUTPUT["Charts, alerts and API responses"]
    INFLUX2 --> OUTPUT
    ES --> OUTPUT

Both backend changes were later combined in the gsoc26-add-more-tsdb branch. The final pull request #868 proposes bringing the combined work to master.

OpenWISP RADIUS Monitoring Integration

While working on the new time-series backends, we found that one important part of the OpenWISP ecosystem had not been calibrated in the original GSoC timeline: the monitoring integration inside OpenWISP RADIUS.

OpenWISP RADIUS uses OpenWISP Monitoring to expose charts for RADIUS accounting and user registration data. These charts were still defined only with InfluxQL queries, which meant they worked with InfluxDB 1.8 but not with the new InfluxDB 2.9 and Elasticsearch backends. If left unchanged, users switching OpenWISP Monitoring to one of the new backends would have working device monitoring charts, but broken RADIUS monitoring charts.

To solve this, I updated the RADIUS monitoring integration with backend-specific query definitions for the new time-series databases. The work involved adding the required RADIUS chart and summary queries for InfluxDB 2.9 and Elasticsearch, covering user registrations, total registered users, RADIUS traffic, and unique RADIUS sessions in both object-specific and general views.

Another important change was updating the rebuild_radius_accounting_metrics management command. It previously deleted old metric data with a raw InfluxQL query, which tied it to InfluxDB 1.8. The command now uses the shared timeseries_db.delete_metric_data() method, making the cleanup logic work across the supported backends.

The final implementation added:

• Backend-specific RADIUS monitoring queries for InfluxDB 2.9 and Elasticsearch.
• Backend-neutral deletion in the RADIUS accounting rebuild command.
• Test, documentation, and CI updates for running RADIUS monitoring with multiple time-series backends.

For the full technical discussion and implementation details, see openwisp-radius pull request #772.

Deployment Support

The final part of the feature work was making the new time-series backends easier to deploy outside the development environment. In docker-openwisp, I added configurable TIMESERIES_BACKEND support, optional Docker Compose services for InfluxDB 2.9 and Elasticsearch, Telegraf support for InfluxDB 2.9 UDP writes, default environment variables, and tests for backend selection.

In ansible-openwisp2, I updated the monitoring time-series configuration to be backend-agnostic, documented examples for switching between InfluxDB 1.8, InfluxDB 2.9, and Elasticsearch, and added Molecule verification for rendered backend settings. The related work is tracked in docker-openwisp pull request #672 and ansible-openwisp2 pull request #645.