Commit 57977694 authored by Micha Mueller's avatar Micha Mueller
Browse files

Update Readme

parent 11971f93
......@@ -5,7 +5,7 @@ Harvesting of the data is task of the dcdbpusher.
# dcdbpusher
This is a general MQTT pusher which sends values of various sensors to the DCDB-database.
It unites, unifies and extends the functionality of former separate sensor-pushers for IPMI, sysFS, perf-event and SNMP.
It ships with plugins for BACnet, IPMI, PDU(proprietary Power Delivery Unit, but could be used as XML plugin) ,percounter, SNMP and sysFS.
Build it by simply running
```bash
......@@ -29,48 +29,12 @@ or run
```
to print the help-section of dcdbpusher.
Dcdbpusher will check the given file-path for the global configuration file `global.conf`.
Dcdbpusher will check the given file-path for the global configuration file which has to be named `global.conf`.
### <a name="GC">Global Configuration</a>
`global.conf` should have the following scheme:
```
global {
mqttBroker localhost:1883
mqttprefix /00112233445566778899AABB0000
threads 24
verbosity 5
daemonize false
tempdir ./../testDir/
cacheInterval 900
}
restAPI {
address localhost:8000
certificate path/to/file
privateKey path/to/file
dhFile path/to/file
authkey token {
PUTReq
GETReq
}
}
plugins {
plugin sysfs {
path ./
config ../data/sysfsTest.conf
}
plugin perfevent {
path
config
}
}
```
Explanation of the values:
The global configuration specifies various settings for dcdbpusher in general, e.g. which plugins should be loaded etc.
Please have a look at the provided `config/global.conf` example to get familiar with the file scheme. The example also forms a good starting point for writing a custom `global.conf`. The different sections and values are explained in the following table:
| Value | Explanation |
|:----- |:----------- |
......@@ -91,12 +55,11 @@ Explanation of the values:
| authkey | This struct is used to define authentication key tokens for the REST API. Within the struct, define which operations over the REST API are allowed for the token (e.g. PUTReq or GETReq). Each token must be unique.
| | |
| plugins | In this section one can specify the plugins which should be used.
| plugin name | The plugin name is used to build the corresponding lib-name (e.g. sysfs --> libdcdbplugin_sysfs.1.0)
| plugin _name_ | The plugin name is used to build the corresponding lib-name (e.g. sysfs --> libdcdbplugin_sysfs.1.0)
| path | Specify the path where the plugin (the shared library) is located. If left empty, dcdbpusher will look in the default lib-directories (usr/lib and friends) for the plugin-file.
| config | One can specify a separate config-file (including path to it) for the plugin to use. If not specified, dcdbpusher will look up pluginName.conf (e.g. sysfs.conf) in the same directory where global.conf is located.
| | |
Formats of the other sensor-specific config-files are explained in the corresponding [subsections](#IPMI). Example configuration-files can be found in the `config/` directory.
......@@ -135,8 +98,13 @@ Tables with allowed ressources sorted by REST methods can be found below. A quer
### Examples
Two examples for HTTPS requests:
* `GET https://localhost:8000/sysfs/freq1/avg?authkey=myToken;interval=15`
* `PUT https://localhost:8000/bacnet/stop?authkey=myToken`
```bash
GET https://localhost:8000/sysfs/freq1/avg?authkey=myToken;interval=15
```
```bash
PUT https://localhost:8000/bacnet/stop?authkey=myToken`
```
## MQTT topic
......@@ -146,7 +114,7 @@ For communication between the different DCDB-components (database, dcdbpusher) t
# Plugins
The core of dcdbpusher is responsible of collecting all the sensor values and sending them to the database. However, the main functionality of the sensors comes from the various plugins. Every plugin corresponds to a special sensor functionality.
Although every plugin requires different configuration parameters for its sensor functionality, some parameters are required equally for every sensor. The config-files for the plugins can follow two schemes: one where every sensor is defined on his own, and one (e.g. for IPMI and PDU) where sensors are aggregated to a higher level unit. The two possible schemes and uniformly needed sensor parameters shall be explained in the following. Configuration parameters which are only required specifically for sensors within a plugin are explained in the corresponding [subsections](#IPMI).
Although every plugin requires different configuration parameters for its sensor functionality, some parameters are required equally for every sensor. The config-files for the plugins can follow two schemes: one where every sensor is defined on his own, and one (e.g. for IPMI and PDU) where sensors are aggregated to a higher level unit. The two possible schemes and uniformly needed sensor parameters shall be explained below. Configuration parameters which are only required specifically for sensors within a plugin are explained in the corresponding [subsections](#IPMI). For every plugin an example configuration file is supplied in the `/config` directory. One should use them as a starting point to write own configuration files.
```
Standalone sensors: | Sensors divided into units:
------------------------------------------------+------------------------------------------------
......@@ -214,61 +182,11 @@ Explanation of the values:
| minValues | Minimum number of sensor reads the sensor should gather before they are sent together to the database. Useful to reduce MQTT-overhead. Default is 1 (every sensor value is sent on his own).
| params | Every plugin requires additional configuration parameters. Those may be unique to every plugin and are explained in the corresponding subsections below.
For an easy start one can begin with modifying the supplied example configuration files in the `/config` directory.
## <a name="IPMI">IPMI</a>
The [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) plugin enables dcdbpusher to collect sensor values offered by a baseboard management controller (BMC).
### config file format
The config file for the IPMI plugin should fulfill the following scheme:
```
global {
sessiontimeout 500
retransmissiontimeout 200
mqttprefix /AABBAABBAABBAACCDDCCDDCC
}
templateSensors {
sensor tempSens {
...
}
...
}
hosts {
host bmc1 {
username "USERID"
password "PASSW0RD"
mqttprefix "/00/11/2233445566/778899AABB"
sensors {
sensor rawSens {
default tempSens
interval 1000
cmd "0x00 0x2e 0x81 0x4d 0x4f 0x00 0x00 0x01 0x82 0x00 0x08"
start 5
stop 12
mqttsuffix 99887766
}
sensor recordSens {
recordId 4321
factor 1.5
mqttsuffix 11223345
}
...
}
}
host bmc2 {
...
}
}
```
Explanation of the values specific for the IPMI plugin:
| Value | Explanation |
......@@ -286,43 +204,18 @@ Explanation of the values specific for the IPMI plugin:
## Perf-event
The Perf-event functionality is tasked with collecting data from the CPUs various performance counters (PMUs).
The Perfevent functionality is tasked with collecting data from the CPUs various performance counters (PMUs).
> NOTE &ensp;&ensp;&ensp; The perf-event plugin measures PMUs for all processes running on a specific CPU. Therefore a value of less than 1 is required in `/proc/sys/kernel/perf_event_paranoid`. Other values (>=1) restrict the access to PMUs. See this [footnote](#fn1) for additional information.
### config file format
For the perf-event plugin the sensors are renamed to counters. Nevertheless the rules and explanations from the generic config file still apply.
```
CounterTemplate {
counter A {...}
counter B {...}
...
}
counters {
counter c_name {
default A
interval 1000
mqttsuffix 0000 // Perfpusher uses a separate MQTT-topic for every CPU. For every used CPU the mqttsuffix is increased by one.
// Dcdbpusher will terminate if an suffix is used twice! Example:
// For counter A the mqttsuffix 00A8 is given and the system on which perfpusher is run has 8 cores. Then perfpusher uses
// the mqtt-suffixe 00A8, 00A9, 00AA, 00AB, 00AC, 00AD, 00AE, 00AF for the 8 cores.
// As they are already used by counter A, any other counter must not use those. Counter B can e.g. start with mqttsuffix 00B0.
minValues 1
type XX // Type of which the counter should be. Each type determines different possible values for the config-field.
// Possible type-values are described below.
config YY // Together with the type-field config determines which performance counter should be read.
// Possible values and what they measure are listed below.
cpus 0,1,2-4 // One can define a comma-separated list of cpu numbers (also value ranges can be specified, e.g. 2-4 equals 2,3,4).
// The hardware counter will then be only opened on the specified cpus.
}
counter c_name2 {
...
}
...
}
```
Explanation of the values specific for the perfevent plugin:
| Value | Explanation |
|:----- |:----------- |
| mqttsuffix | In the context of the perfevent plugin the mqttsuffixes are interpreted as start mqttsuffix. A perfsensor uses a separate MQTT-topic for every CPU. For every used CPU the mqttsuffix is incremented by one. However, dcdbpusher does not allow the same suffix twice! Example: For sensor A the mqttsuffix 00A8 is given and the system on which perfpusher is run has 8 cores. Then perfpusher uses the mqtt-suffixe 00A8, 00A9, 00AA, 00AB, 00AC, 00AD, 00AE, 00AF for the 8 cores. As they are already used by counter A, any other sensor must not use those. Counter B can e.g. start with mqttsuffix 00B0.
| type | Type of which the counter should be. Each type determines different possible values for the config-field. Possible type-values are described below.
| config | Together with the type-field config determines which performance counter should be read. Possible values and what they measure are listed below.
| cpus | One can define a comma-separated list of cpu numbers (also value ranges can be specified, e.g. 2-4 equals 2,3,4). The hardware counter will then be only opened on the specified cpus.
### type and config
......@@ -391,6 +284,8 @@ The existence of the perf_event_paranoid file is the official method for determi
The SNMP plugin enables dcdbpusher to talk with devices which have an SNMP agent running and query requests from them. A SNMP sensor corresponds to a single value as identified by the unique OID. Sensors are aggregated by connections. See the exemplary snmp.conf file in the `config/` directory.
> NOTE &ensp;&ensp;&ensp; In the SNMP context the word privacy is used synonymously for encryption.
Explanation of the values specific for the SNMP plugin:
| Value | Explanation |
|:----- |:----------- |
| connection | An aggregating connection
......@@ -416,6 +311,8 @@ The SNMP plugin enables dcdbpusher to talk with devices which have an SNMP agent
SysFS sensors read data from sysFS files. The configuration file of the plugin corresponds to the generic plugin configuration with standalone sensors. Additionally for a sysFS sensor the following parameters are mandatory/possible:
Explanation of the values specific for the sysFS plugin:
| Value | Explanation |
|:----- |:----------- |
| path | Path to the sysFS file the sensor should read from. This parameter is mandatory.
......@@ -425,48 +322,6 @@ SysFS sensors read data from sysFS files. The configuration file of the plugin c
The Power Delivery Unit (PDU) plugin is in charge of sending a network-request to the PDUs and gathering specified sensor data from the XML-file response.
### config file format
The config file for the PDU plugin should fulfill the following scheme:
```
global {
mqttprefix /00112233445566778899AABBCCDD
}
SensorTemplate {
sensor tempSens {
...
}
...
}
pdus {
pdu rack1 {
host example.com:443
TTL 500
request "I hereby demand\n a response called \"response\""
sensors {
sensors pcs1 {
default tempSens
interval 2000
path "root.node1.node2(id=1).node3(id=1,type=2).valueNode"
mqttsuffix 0001
minValues 3
}
sensor pcs2 {
...
}
}
}
pdu rack2 {
...
}
}
```
Explanation of the values specific for the PDU plugin:
| Value | Explanation |
......@@ -481,62 +336,6 @@ Explanation of the values specific for the PDU plugin:
The BACnet plugin enables dcdbpusher to communicate and request data from devices which communicate via the BACnet protocol. A so called "read property" request is sent by the plugin to the BACnet devices as configured in the config file. The response value is then stored in the database. Usually one is only interested in collecting the current reading of a BACnet device (property PROP_PRESENT_VALUE, ID 85). However, also reading of other properties is supported.
> NOTE &ensp;&ensp;&ensp; On startup BACnet plugin does no device discovery. Instead it relies on the user providing a file with addresses of all required BACnet devices. One can generate such an address-file for example by using the `bacwi` demo tool provided by the BACnet-Stack.
### config file format
The config file for the BACnet plugin should fulfill the following scheme:
```
global {
mqttPrefix /FF112233445566778899FFFF
address_cache ../test.cache
interface eth0
port 22222
timeout 1000
apdu_timeout 200
apdu_retries 1
}
templates {
property def0 {
factor 100
id 85
interval 1000
minValues 3
}
property def1 {
...
}
}
devices {
device dev1 {
instance 1234
mqttPart FF
object o1 {
type 8
instance 1234
mqttPart FF
property test1 {
default def0
id 85
mqttsuffix 0000
}
property smth {
...
}
}
object o2 {
...
}
}
device jklo {
...
}
}
```
Explanation of the values specific for the BACnet plugin:
| Value | Explanation |
......@@ -556,7 +355,10 @@ Explanation of the values specific for the BACnet plugin:
| instance (object) | Instance of the object within the device.
| id | ID of the property to be read from the BACnet device-object. Assignment of numbers to properties is done according to the enum as defined in `bacenum.h`.
## Writing own plugins
TODO!
#### TODOS
* perhaps xml-file example for pdu
* explain sensor group concept
* rework pdu plugin: should be more generic XML plugin
* more about mqtt-topic
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment