Influxdb Historian¶

InfluxDB is an open source time series database with a fast, scalable engine and high availability. It’s often used to build DevOps Monitoring (Infrastructure Monitoring, Application Monitoring, Cloud Monitoring), IoT Monitoring, and Real-Time Analytics solutions.

More information about InfluxDB is available from https://www.influxdata.com/.

Prerequisites¶

InfluxDB Installation¶

To install InfluxDB on an Ubuntu or Debian operating system, run the script:

services/core/InfluxdbHistorian/scripts/install-influx.sh

For installation on other operating systems, see https://docs.influxdata.com/influxdb/v1.4/introduction/installation/.

Authentication in InfluxDB¶

By default, the InfluxDB Authentication option is disabled, and no user authentication is required to access any InfluxDB database. You can enable authentication by updating the InfluxDB configuration file. For detailed information on enabling authentication, see: https://docs.influxdata.com/influxdb/v1.4/query_language/authentication_and_authorization/.

If Authentication is enabled, authorization privileges are enforced. There must be at least one defined admin user with access to administrative queries as outlined in the linked document above. Additionally, you must pre-create the user and database that are specified in the configuration file (the default configuration file for InfluxDB is services/core/InfluxdbHistorian/config). If your user is a non-admin user, they must be granted a full set of privileges on the desired database.

InfluxDB Driver¶

In order to connect to an InfluxDb client, the Python library for InfluxDB must be installed in VOLTTRON’s virtual environment. From the command line, after enabling the virtual environment, install the InfluxDB library as follows:

pip install influxdb

Configuration¶

The default configuration file for VOLTTRON’s InfluxDBHistorian agent should be in the format:

{
  "connection": {
    "params": {
      "host": "localhost",
      "port": 8086,         # Don't change this unless default bind port
                            # in influxdb config is changed
      "database": "historian",
      "user": "historian",  # user is optional if authentication is turned off
      "passwd": "historian" # passwd is optional if authentication is turned off
    }
  },
  "aggregations": {
    "use_calendar_time_periods": true
  }
}

The InfluxDBHistorian agent can be packaged, installed and started according to the standard VOLTTRON agent creation procedure. A sample VOLTTRON configuration file has been provided: services/core/InfluxdbHistorian/config.

Connection¶

The host, database, user and passwd values in the VOLTTRON configuration file can be modified. user and passwd are optional if InfluxDB Authentication is disabled.

**Note:** Be sure to initialize or pre-create the database and user that you defined in the configuration file, and if user is a non-admin user, be make sure to grant privileges for the user on the specified database. For more information, see Authentication in InfluxDB.

Aggregations¶

In order to use aggregations, the VOLTTRON configuration file must also specify a value, either true or false, for use_calendar_time_periods, indicating whether the aggregation period should align to calendar time periods. If this value is omitted from the configuration file, aggregations cannot be used.

For more information on historian aggregations, see: Aggregate Historian Agent Specification <AggregateHistorianSpec>{.interpreted-text role=”ref”}.

Supported Influxdb aggregation functions:

Aggregations: COUNT(), DISTINCT(), INTEGRAL(), MEAN(), MEDIAN(), MODE(), SPREAD(), STDDEV(), SUM()

Selectors: FIRST(), LAST(), MAX(), MIN()

Transformations: CEILING(),CUMULATIVE_SUM(), DERIVATIVE(), DIFFERENCE(), ELAPSED(), NON_NEGATIVE_DERIVATIVE(), NON_NEGATIVE_DIFFERENCE()

More information how to use those functions: https://docs.influxdata.com/influxdb/v1.4/query_language/functions/

**Note:** Historian aggregations in InfluxDB are different from aggregations employed by other historian agents in VOLTTRON. InfluxDB doesn’t have a separate agent for aggregations. Instead, aggregation is supported through the query_historian function. Other agents can execute an aggregation query directly in InfluxDB by calling the RPC.export method query. For an example, see Aggregate Historian Agent Specification

Database Schema¶

Each InfluxDB database has a meta table as well as other tables for different measurements, e.g. one table for “power_kw”, one table for “energy”, one table for “voltage”, etc. (An InfluxDB measurement is similar to a relational table, so for easier understanding, InfluxDB measurements will be referred to below as tables.)

Measurement Table¶

Example: If a topic name is “CampusA/Building1/Device1/Power_KW”, the power_kw table might look as follows:

time	building	campus	device	source	value
2017-12-28T20:41:00.004260096Z	building1	campusa	device1	scrape	123.4
2017-12-30T01:05:00.004435616Z	building1	campusa	device1	scrape	567.8
2018-01-15T18:08:00.126345Z	building1	campusa	device1	scrape	10

building, campus, device, and source are InfluxDB tags. value is an InfluxDB field.

**Note:** The topic is converted to all lowercase before being stored in the table. In other words, a set of tag names, as well as a table name, are created by splitting topic_id into substrings (Meta Table section below).

So in this example, where the typical format of a topic name is <campus>/<building>/<device>/<measurement>, campus, building and device are each stored as tags in the database.

A topic name might not confirm to that convention:

The topic name might contain additional substrings, e.g. CampusA/Building1/LAB/Device/OutsideAirTemperature. In this case, campus will be campusa/building, building will be lab, and device will be device.
The topic name might contain fewer substrings, e.g. LAB/Device/OutsideAirTemperature. In this case, the campus tag will be empty, building will be lab, and device will be device.

Meta Table¶

The meta table will be structured as in the following example:

In the InfluxDB, last_updated, meta_dict and topic are fields and topic_id is a tag.

Since InfluxDB is a time series database, the time column is required, and a dummy value (time=0, which is 1970-01-01T00:00:00Z based on epoch unix time) is assigned to all topics for easier metadata updating. Hence, if the contents of meta_dict change for a specific topic, both last_updated and meta_dict values for that topic will be replaced in the table.