README.md 27.1 KB
Newer Older
Micha Mueller's avatar
Micha Mueller committed
1
2
3
4
5
6
## Intro
DCDB (DataCenter DataBase) is a database to collect various (sensor-)values of a datacenter for further analysis.
Harvesting of the data is task of the dcdbpusher.

# dcdbpusher

Micha Mueller's avatar
Micha Mueller committed
7
This is a general MQTT pusher which sends values of various sensors to the DCDB-database.
Micha Mueller's avatar
Micha Mueller committed
8
It ships with plugins for BACnet, IPMI, PDU(proprietary Power Delivery Unit, but could be used as XML plugin) ,percounter, SNMP and sysFS.
Micha Mueller's avatar
Micha Mueller committed
9
10
11
12
13
14
15
16
17

Build it by simply running
```bash
make
```
or alternatively use
```bash
make debug
```
Micha Mueller's avatar
Micha Mueller committed
18
within the `dcdbpusher` directory to build a version which will print additional debug-information during runtime.
Micha Mueller's avatar
Micha Mueller committed
19

20
The logic for the various sensors is encapsulated into plugins (shared dynamic libraries; the makefile will take care of compiling them for you). The dcdbpusher will dynamically open the libraries if they are specified in the [global configuration](#GC) file. Vice versa, if selected sensor-functionality, e.g. sysFS is not specified, the corresponding shared library libdcdbplugin_sysfs.so does not have to be present. 
Micha Mueller's avatar
Micha Mueller committed
21

Micha Mueller's avatar
Micha Mueller committed
22
23
24
25
You can run dcdbpusher by executing
```bash
./dcdbpusher path/to/configfile/
```
26
or run
Micha Mueller's avatar
Micha Mueller committed
27
28
29
```bash
./dcdbpusher -h
```
Micha Mueller's avatar
Micha Mueller committed
30
to print the help-section of dcdbpusher.
Micha Mueller's avatar
Micha Mueller committed
31

Micha Mueller's avatar
Micha Mueller committed
32
Dcdbpusher will check the given file-path for the global configuration file which has to be named `global.conf`.
Micha Mueller's avatar
Micha Mueller committed
33

34
### <a name="GC">Global Configuration</a>
Micha Mueller's avatar
Micha Mueller committed
35

Micha Mueller's avatar
Micha Mueller committed
36
37
The global configuration specifies various settings for dcdbpusher in general, e.g. which plugins should be loaded etc.
Please have a look at the provided `config/global.conf` example to get familiar with the file scheme. The example also forms a good starting point for writing a custom `global.conf`. The different sections and values are explained in the following table:
Micha Mueller's avatar
Micha Mueller committed
38

Micha Mueller's avatar
Micha Mueller committed
39
| Value | Explanation |
Micha Mueller's avatar
Micha Mueller committed
40
41
|:----- |:----------- |
| global | Wrapper structure for the global values.
Micha Mueller's avatar
Micha Mueller committed
42
43
44
45
46
47
48
| mqttBroker | Define address and port of the MQTT-broker which collects the messages (sensor values) send by dcdbpusher.
| mqttprefix | To not rewrite a full MQTT-topic for every sensor one can specify here a consistent prefix.
| threads | Specify how many threads should be created to handle the sensors async. Default value of threads is 1. Note that the MQTTPusher always starts an extra thread. So the actual number of started threads is always one more than defined here. Specifying not enough threads can result in a delay for some sensors until they are read.
| verbosity | Level of detail in the logfile (dcdb.log). Set to a value between 5 (all log-messages, default) and 0 (only fatal messages). NOTE: level of verbosity for the command-line log can be set via the -v flag independently when invoking dcdbpusher.
| daemonize | Set to 'true' if dcdbpusher should run detached as daemon. Default is false.
| tempdir | One can specify a writeable directory where dcdbpusher can write its temporary and logging files to. Default is the current (' ./ ' ) directory.
| cacheInterval | Define a time interval in seconds. The last sensor readings within this time interval will be kept. This value can be overwritten by plugins.
Micha Mueller's avatar
Micha Mueller committed
49
| | |
50
51
52
53
54
| restAPI | Bundles all values related to the RestAPI. See the corresponding [section](#restApi) for more information on supported functionality.
| address | Define address and port where the REST API server should run on.
| certificate | Provide the (path and) file which the HTTPS server should use as certificate.
| privateKey | Provide the (path and) file which should be used as corresponding private key for the certificate. If private key and certificate are stored in the same file one should nevertheless provide the path to the cert-file here again.
| dhFile | Provide the (path and) file where Diffie-Hellman parameters for the key exchange are stored.
Micha Mueller's avatar
Micha Mueller committed
55
| authkey | This struct is used to define authentication key tokens for the REST API. Within the struct, define which operations over the REST API are allowed for the token (e.g. PUTReq or GETReq). Each token must be unique.
Micha Mueller's avatar
Micha Mueller committed
56
57
| | |
| plugins | In this section one can specify the plugins which should be used.
Micha Mueller's avatar
Micha Mueller committed
58
| plugin _name_ | The plugin name is used to build the corresponding lib-name (e.g. sysfs --> libdcdbplugin_sysfs.1.0)
Micha Mueller's avatar
Micha Mueller committed
59
60
| path | Specify the path where the plugin (the shared library) is located. If left empty, dcdbpusher will look in the default lib-directories (usr/lib and friends) for the plugin-file.
| config | One can specify a separate config-file (including path to it) for the plugin to use. If not specified, dcdbpusher will look up pluginName.conf (e.g. sysfs.conf) in the same directory where global.conf is located.
Micha Mueller's avatar
Micha Mueller committed
61
62
| | |

63
Formats of the other sensor-specific config-files are explained in the corresponding [subsections](#IPMI). Example configuration-files can be found in the `config/` directory.
Micha Mueller's avatar
Micha Mueller committed
64
65


66
67
## <a name="restApi">REST API</a>

68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
Dcdbpusher runs a HTTPS server which provides some functionality to be controlled over a RESTful API. The API is by default hosted at port 8000 on the localhost but the address can be changed in the [`global.conf`](#GC).

A HTTPS request to dcdbpusher should have the following format: `[GET|PUT] host:port[ressource]?[queries]`.
Tables with allowed ressources sorted by REST methods can be found below. A query consists of a key-value pair of the format `key=value`. For every request at least the authentication token has to be appended as query. Multiple queries are separated by semicolons(';').

### Table of queries

| Key | Value | Explanation |
|:--- |:----- |:----------- |
| authkey | Your authentication token | The authentication token is required to verify that you are allowed to make this particular request. The authkey query must be included by every request
| interval | Time-value in [s] | Only for GET request of sensor readings average. One can (optionally) specify a custom time interval for which the average of sensor readings is calculated. By default, every sensor reading in the cache is used to calculate the average.


### Ressources for GET request

| Ressource | Explanation |
|:--------- |:----------- |
| /help | Responds with a small cheatsheet which presents all currently supported ressources.
| /plugins | (Discovery) Returns a list of all currently loaded plugins.
| /[plugin]/sensors | (Discovery) Returns a list of all sensors which belong to [plugin]. To find out which plugins are available one can request the `/plugins` ressource.
| /[plugin]/[sensor]/avg | Calculates and returns the average of the last sensor readings. Can be combined with the interval query.


### Ressources for PUT request

| Ressource | Explanation |
|:--------- |:----------- |
95
| /[plugin]/[action] | One can request to do a action on the specified plugin. Currently supported actions are `start`, `stop` and `reload` which start or stop the polling of the sensors of the plugin respectively triggers a reload of the plugin configuration. If a configuration reload is requested, all sensors of the plugin are stopped first. Then new sensors are created and started according to the configuration.
96
97
98
99
100


### Examples

Two examples for HTTPS requests:
Micha Mueller's avatar
Micha Mueller committed
101
102
103
104
105

```bash
GET https://localhost:8000/sysfs/freq1/avg?authkey=myToken;interval=15
```
```bash
Micha Mueller's avatar
Micha Mueller committed
106
PUT https://localhost:8000/bacnet/stop?authkey=myToken
Micha Mueller's avatar
Micha Mueller committed
107
```
108
109


Micha Mueller's avatar
Micha Mueller committed
110
## MQTT topic
111

Micha Mueller's avatar
Micha Mueller committed
112
For communication between the different DCDB-components (database, dcdbpusher) the [MQTT protocol](https://mqtt.org/) is used. In order to identify each sensor, everyone has to have a unique MQTT topic assigned. A MQTT topic for DCDB consists of exactly 128 bits (= 32 hex characters), not including '/' separators.
113

Micha Mueller's avatar
Micha Mueller committed
114
115
116
# Plugins

The core of dcdbpusher is responsible of collecting all the sensor values and sending them to the database. However, the main functionality of the sensors comes from the various plugins. Every plugin corresponds to a special sensor functionality.
Micha Mueller's avatar
Micha Mueller committed
117
Although every plugin requires different configuration parameters for its sensor functionality, some parameters are required equally for every sensor. The config-files for the plugins can follow two schemes: one where every sensor is defined on his own, and one (e.g. for IPMI and PDU) where sensors are aggregated to a higher level unit. The two possible schemes and uniformly needed sensor parameters shall be explained below. Configuration parameters which are only required specifically for sensors within a plugin are explained in the corresponding [subsections](#IPMI). For every plugin an example configuration file is supplied in the `/config` directory. One should use them as a starting point to write own configuration files. 
Micha Mueller's avatar
Micha Mueller committed
118
119
120
121
122
123
```
 Standalone sensors:							| Sensors divided into units:
------------------------------------------------+------------------------------------------------
												|
global {										|global {
	mqttprefix /00112233445566778899AABB0000	|	mqttprefix /00112233445566778899AABB0000
124
	cacheInterval 120							|	cacheInterval 120
Micha Mueller's avatar
Micha Mueller committed
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
}												|}
												|
SensorTemplate {								|SensorTemplate {
	sensor name {								|	sensor name {
		...										|		...
	}											|	}
												|
	...											|	...
}												|}
												|
sensors {										|Units {
 	sensor name1 {								|	sensorUnit u1 {
		default		A							|		unitParams		?
		interval	1000						|	
		mqttsuffix	0000						|		sensors {
		minValues	1							|			sensor name1 {
		params		?							|				default		A
	}											|				interval	1000
												|				mqttsuffix	000
	sensor name2 {								|				minValues	1
		...										|				params		?
	}											|			}
												|
	...											|			sensor name2 {
}												|				...
												|			}
												|			...
												|		}
												|	}
												|
												|	sensorUnit u2 {
												|		...
												|	}
												|
												|	...
												|}
```
Explanation of the values:

Micha Mueller's avatar
Micha Mueller committed
164
| Value | Explanation |
Micha Mueller's avatar
Micha Mueller committed
165
|:----- |:----------- |
166
| global | Here one can overwrite the global values defined in `global.conf`. The overwritten values are only used in the scope of the specific plugin. Overwriting of global values is completely optional. However, even if no global values are overwritten at least the `global{}` struct should be present.
Micha Mueller's avatar
Micha Mueller committed
167
168
| mqttprefix | Define separate MQTT prefix for this plugin.
| cacheInterval | Overwrite global caching interval with plugin specific value.
Micha Mueller's avatar
Micha Mueller committed
169
170
| | |
| SensorTemplate | Define template sensors to be used later in the configuration. This feature is mainly for convenience reasons. One is not obligated to define any template sensors. However it is required to at least define an empty SensorTemplate {} structure.
Micha Mueller's avatar
Micha Mueller committed
171
| sensor name | Every template sensor needs a name for future reference. A template sensor can define every value (including values specific to a plugin) a normal sensor can (see below).
Micha Mueller's avatar
Micha Mueller committed
172
173
| | |
| Units | Section to define different units where sensors may be aggregated to. Not every plugin makes use of this subdivision. Where it is used it the Units may be named different.
Micha Mueller's avatar
Micha Mueller committed
174
175
| sensorUnit u1 | If the unit subdivision is used, subunits need to be used and also need to be named.
| unitParams | One may be able to define special parameters specific to the subunits. They should be explained with the corresponding plugin if it makes use of subunits.
Micha Mueller's avatar
Micha Mueller committed
176
| | |
Micha Mueller's avatar
Micha Mueller committed
177
178
179
180
181
182
183
| sensors | Define in this section the sensors for the plugin/subunit
| sensor name | Every sensor needs a unique name
| default | Mention here the name of the template sensor to be used. The sensor will be initialized with the exact same values as the template sensor. This field can be left out if all required values for the sensor are defined manually.
| interval | Time in [ms] between two consecutive sensor reads. Default is 1000[ms] = 1[s]
| mqttsuffix | Suffix which is appended to the MQTT-prefix. Together with the prefix this should be a globally unique MQTT-topic for every sensor.
| minValues | Minimum number of sensor reads the sensor should gather before they are sent together to the database. Useful to reduce MQTT-overhead. Default is 1 (every sensor value is sent on his own).
| params | Every plugin requires additional configuration parameters. Those may be unique to every plugin and are explained in the corresponding subsections below.
Micha Mueller's avatar
Micha Mueller committed
184

185

Micha Mueller's avatar
Micha Mueller committed
186
## <a name="IPMI">IPMI</a>
Micha Mueller's avatar
Micha Mueller committed
187

Micha Mueller's avatar
Micha Mueller committed
188
The [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) plugin enables dcdbpusher to collect sensor values offered by a baseboard management controller (BMC).
189

190
Explanation of the values specific for the IPMI plugin:
191

Micha Mueller's avatar
Micha Mueller committed
192
| Value | Explanation |
193
194
195
196
197
198
199
200
201
|:----- |:----------- |
| sessiontimeout | Session timeout value for the IPMI-connection
| retransmissiontimeout | Retransmission timeout value for the IPMI-connection
| username | For the remote IPMI-connection login credentials are required
| password | For the remote IPMI-connection login credentials are required
| mqttprefix | One can define a different MQTT-prefix per host
| cmd | One can define a raw IPMI-command (in hex-notation) to be sent. In this case also the start and stop fields for the response have to be defined. Alternatively, one can define the record-ID of the sensor (see below).
| start | Required if raw command is sent
| stop | Required if raw command is sent
Micha Mueller's avatar
Micha Mueller committed
202
| recordId | Define the record-ID of the sensor to be read. One can look up the corresponding record-IDs for every sensor with the "ipmi-sensors" command line tool (ships with the freeipmi-library). Alternatively, one can define a raw IPMI-command (see above).
203
| factor | One can specify a factor to scale the read value before it is stored in the database (to adjust precision).
204

Micha Mueller's avatar
Micha Mueller committed
205
206
## Perf-event

Micha Mueller's avatar
Micha Mueller committed
207
The Perfevent functionality is tasked with collecting data from the CPUs various performance counters (PMUs).
208
> NOTE &ensp;&ensp;&ensp; The perf-event plugin measures PMUs for all processes running on a specific CPU. Therefore a value of less than 1 is required in `/proc/sys/kernel/perf_event_paranoid`. Other values (>=1) restrict the access to PMUs. See this [footnote](#fn1) for additional information.
Micha Mueller's avatar
Micha Mueller committed
209

Micha Mueller's avatar
Micha Mueller committed
210
211
212
213
214
215
216
217
218
Explanation of the values specific for the perfevent plugin:

| Value | Explanation |
|:----- |:----------- |
| mqttsuffix | In the context of the perfevent plugin the mqttsuffixes are interpreted as start mqttsuffix. A perfsensor uses a separate MQTT-topic for every CPU. For every used CPU the mqttsuffix is incremented by one. However, dcdbpusher does not allow the same suffix twice! Example: For sensor A the mqttsuffix 00A8 is given and the system on which perfpusher is run has 8 cores. Then perfpusher uses the mqtt-suffixe 00A8, 00A9, 00AA, 00AB, 00AC, 00AD, 00AE, 00AF for the 8 cores. As they are already used by counter A, any other sensor must not use those. Counter B can e.g. start with mqttsuffix 00B0.
| type | Type of which the counter should be. Each type determines different possible values for the config-field. Possible type-values are described below.
| config | Together with the type-field config determines which performance counter should be read. Possible values and what they measure are listed below.
| cpus | One can define a comma-separated list of cpu numbers (also value ranges can be specified, e.g. 2-4 equals 2,3,4). The hardware counter will then be only opened on the specified cpus.

Micha Mueller's avatar
Micha Mueller committed
219
220
221

### type and config

Micha Mueller's avatar
Micha Mueller committed
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
(see the [perf_event_open man-page](http://man7.org/linux/man-pages/man2/perf_event_open.2.html) for more detailed explanations)

| Type | Config | Explanation |
|:----:|:------ |:----------- |
| PERF_TYPE_HARDWARE | | generalized hardware CPU events
| " | PERF_COUNT_HW_CPU_CYCLES | total cycles (affected by frequency scaling)
| " | PERF_COUNT_HW_INSTRUCTIONS | retired instructions
| " | PERF_COUNT_HW_CACHE_REFERENCES | cache accesses (usually last level)
| " | PERF_COUNT_HW_CACHE_MISSES | cache misses (usually last level)
| " | PERF_COUNT_HW_BRANCH_INSTRUCTIONS | retired branch instructions
| " | PERF_COUNT_HW_BRANCH_MISSES | mispredicted branch instructions
| " | PERF_COUNT_HW_BUS_CYCLES | bus cycles
| " | PERF_COUNT_HW_STALLED_CYCLES_FRONTEND | stalled cycles during issue
| " | PERF_COUNT_HW_STALLED_CYCLES_BACKEND  | stalled cycles during retirement
| " | PERF_COUNT_HW_REF_CPU_CYCLES | total cycles (unaffected by frequency scaling)
237
| | | |
Micha Mueller's avatar
Micha Mueller committed
238
239
240
241
242
243
244
245
246
247
248
| PERF_TYPE_SOFTWARE | | software events provided by the kernel
| " | PERF_COUNT_SW_CPU_CLOCK | reports CPU clock
| " | PERF_COUNT_SW_TASK_CLOCK | clock count specific to the running task
| " | PERF_COUNT_SW_PAGE_FAULTS | number of page faults
| " | PERF_COUNT_SW_CONTEXT_SWITCHES | count of context switches
| " | PERF_COUNT_SW_CPU_MIGRATIONS | times the process has migrated to a new CPU
| " | PERF_COUNT_SW_PAGE_FAULTS_MIN | number of minor page faults (no disk-I/O)
| " | PERF_COUNT_SW_PAGE_FAULTS_MAJ | number of major page faults (disk-I/O was required)
| " | PERF_COUNT_SW_ALIGNMENT_FAULTS | alignment faults when accessing unaligned memory
| " | PERF_COUNT_SW_EMULATION_FAULTS | number of unimplemented instructions which had to be emulated
| " | PERF_COUNT_SW_DUMMY | placeholder which counts nothing
249
| | | |
Micha Mueller's avatar
Micha Mueller committed
250
251
| PERF_TYPE_TRACEPOINT | | not yet implemented
| PERF_TYPE_HW_CACHE | | not yet implemented
252
| | | |
Micha Mueller's avatar
Micha Mueller committed
253
254
| PERF_TYPE_RAW | | user can define architecture-specific raw events here.
| " | *XXXX* | Config must be an hex value (without 0x prefix). See <sup>[2](#fn2)</sup>
255
| | | |
Micha Mueller's avatar
Micha Mueller committed
256
| PERF_TYPE_BREAKPOINT | --- | config not required, any values will be ignored. However config must still be specified (even if empty)
Micha Mueller's avatar
Micha Mueller committed
257

258
#### Footnotes
Micha Mueller's avatar
Micha Mueller committed
259
260
261

Taken from the [perf_event_open man-page](http://man7.org/linux/man-pages/man2/perf_event_open.2.html):

262
263
264
<a name="fn1">**1**</a>: &ensp; The pid and cpu arguments allow specifying which process and CPU to monitor:  
[...]  
pid == -1 and cpu >= 0  
265
This measures all processes/threads on the specified CPU. This requires CAP_SYS_ADMIN capability or a /proc/sys/kernel/perf_event_paranoid value of less than 1.
266
267
268
269

[...]

The perf_event_paranoid file can be set to restrict access to the performance counters.
Micha Mueller's avatar
Micha Mueller committed
270
271
272
273
274
275
276

| Value | Restriction |
|:-----:|:----------- |
| 2 | allow only user-space measurements (default since Linux 4.6) |
| 1 | allow both kernel and user measurements (default before Linux 4.6) |
| 0 | allow access to CPU-specific data but not raw trace-point samples |
| -1 | no restrictions |
Micha Mueller's avatar
Micha Mueller committed
277
278
279
	
The existence of the perf_event_paranoid file is the official method for determining if a kernel supports perf_event_open()

Micha Mueller's avatar
Micha Mueller committed
280
<a name="fn2">**2**</a>: &ensp; If type is *PERF_TYPE_RAW*, then a custom "raw" config value is needed. Most CPUs support events that are not covered by the "generalized" events. These are implementation defined; see your CPU manual (for example the Intel Volume 3B documentation or the AMD BIOS and Kernel Developer Guide). The libpfm4 library can be used to translate from the name in the architectural manual to the raw hex value perf_event_open() expects in this field.
Micha Mueller's avatar
Micha Mueller committed
281
282
283

## snmp

Micha Mueller's avatar
Micha Mueller committed
284
The SNMP plugin enables dcdbpusher to talk with devices which have an SNMP agent running and query requests from them. A SNMP sensor corresponds to a single value as identified by the unique OID. Sensors are aggregated by connections. See the exemplary snmp.conf file in the `config/` directory.
285
> NOTE &ensp;&ensp;&ensp; In the SNMP context the word privacy is used synonymously for encryption.
Micha Mueller's avatar
Micha Mueller committed
286

Micha Mueller's avatar
Micha Mueller committed
287
288
Explanation of the values specific for the SNMP plugin:

Micha Mueller's avatar
Micha Mueller committed
289
290
291
292
293
294
295
| Value | Explanation |
|:----- |:----------- |
| connection | An aggregating connection
| Type | Type of the SNMP application which runs on the device queried by the connection. Currently only the type Agent is supported.
| Host | Host name of the device which is to be queried.
| Port | The SNMP port should be usually 161. No changes should be required here.
| OIDPrefix | This OIDPrefix is used for all following sensors.
Micha Mueller's avatar
Micha Mueller committed
296
297
298
299
300
301
302
303
304
| |
| Version | Which SNMP version to use (either 2 (maps to 2c) or 3).
| Community | Which SNMP community to use (required only if version 2 is used).
| Username | Username to authenticate with (only required for version 3).
| SecLevel | The security level to be used (only required for version 3). Can be either `noAuthNoPriv` for no authentication and privacy ("privacy" is SNMPs synonym for encryption), `authNoPriv` for only authentication and `authPriv` for authentication and privacy.
| AuthProto | Which protocol to use for authentication (only required for version 3 and if SecLevel != noAuthNoPriv). Can be MD5 or SHA1.
| AuthKey | The passphrase for authentication (only required for version 3 and if SecLevel != noAuthNoPriv). Must be at least 8 characters long.
| PrivProto | Which protocol to use for privacy (only required for version 3 and if SecLevel = AuthPriv). Can be DES or AES.
| PrivKey | The passphrase for privacy encryption (only required for version 3 and if SecLevel = AuthPriv). Must be at least 8 characters long.
Micha Mueller's avatar
Micha Mueller committed
305
| mqttPart | Connection specific MQTT-part which is appended to the MQTT-prefix and succeded by the sensor specific suffix.
Micha Mueller's avatar
Micha Mueller committed
306
| |
Micha Mueller's avatar
Micha Mueller committed
307
| OID | OID suffix which together with the OIDPrefix forms the unique OID identifying a value to query.
Micha Mueller's avatar
Micha Mueller committed
308
| passphrase | has to be at least 8 characters long
Micha Mueller's avatar
Micha Mueller committed
309

Micha Mueller's avatar
Micha Mueller committed
310
311
312
## sysFS

SysFS sensors read data from sysFS files. The configuration file of the plugin corresponds to the generic plugin configuration with standalone sensors. Additionally for a sysFS sensor the following parameters are mandatory/possible:
313

Micha Mueller's avatar
Micha Mueller committed
314
315
Explanation of the values specific for the sysFS plugin:

Micha Mueller's avatar
Micha Mueller committed
316
317
318
| Value | Explanation |
|:----- |:----------- |
| path | Path to the sysFS file the sensor should read from. This parameter is mandatory.
319
| filter | One can define an optional filter if the sysFS file consists of more than only the sensor value. Please note the following points for filters: <br> 1.  The filter supports substitutions. For substitution sed syntax ("s/.../.../") is used. Therefore extended regular expressions (ERE) are used as regex-syntax. ERE is closest to Basic RE (BRE), which is actually used by sed, but requires less escaping. <br> 2.  If a \ ("backslash") is needed in the regex (for escaping), always use \\ ("double backslash") as the regex is read in as string and strings also escape with backslash <br> 3.  Whitespaces are actually used as value separators in the config files. If your filter requires whitespaces either use [[:space:]] in the regex or put it in quotation marks ("") <br> 4.  To be able to reference parts of the match (for substitution) use groups. Groups are created with parentheses. <br>  5.  If using character classes like [[:digit:]] always make sure to use double brackets ("[[" and "]]") or they will not be recognized. <br>  See [ERE-syntax](https://www.gnu.org/software/sed/manual/html_node/ERE-syntax.html#ERE-syntax) <br>  See [substitution syntax](http://www.boost.org/doc/libs/1_65_1/libs/regex/doc/html/boost_regex/format/sed_format.html)
320
321
322
323
324

## PDU

The Power Delivery Unit (PDU) plugin is in charge of sending a network-request to the PDUs and gathering specified sensor data from the XML-file response.

Micha Mueller's avatar
Micha Mueller committed
325
Explanation of the values specific for the PDU plugin:
Micha Mueller's avatar
Micha Mueller committed
326

327
| Value | Explanation |
328
|:----- |:----------- |
329
| host | Hostname and (optional) port where to fetch the XML-file with sensor data from. If no port is specified, 443 is used. The plugin requests the file via HTTPS.
330
| TTL | To avoid requesting a current XML-file every time a sensor wants to read his value, one can define a time to live (TTL) for the file here. A new XML-file is requested at the earliest if the TTL has expired. Default value is 1000[ms].
Micha Mueller's avatar
Micha Mueller committed
331
| request | Define the request to be sent to the host via HTTPS as a string. One should put the request in quotation marks (' " ') to enable the use of whitespaces within the request. Special characters (like usage of ' " ' within the request) should be escaped (' " ' --> ' \" '; ' \ ' --> ' \\\\ '; newline --> ' \n '; ...).
332
| path | Define a dot-separated path to the value to be read in the XML file. One can specify attribute values a node has to fulfil in brackets after the node. Even multiple (comma-separated) attributes can be given, however no whitespaces should be used (!) as they will not be filtered and could therefore be treat as part of the attributes name.
333

334
335
336
## BACnet

The BACnet plugin enables dcdbpusher to communicate and request data from devices which communicate via the BACnet protocol. A so called "read property" request is sent by the plugin to the BACnet devices as configured in the config file. The response value is then stored in the database. Usually one is only interested in collecting the current reading of a BACnet device (property PROP_PRESENT_VALUE, ID 85). However, also reading of other properties is supported.
337
> NOTE &ensp;&ensp;&ensp; On startup BACnet plugin does no device discovery. Instead it relies on the user providing a file with addresses of all required BACnet devices. One can generate such an address-file for example by using the `bacwi` demo tool provided by the BACnet-Stack.
338
339
340
341
342

Explanation of the values specific for the BACnet plugin:

| Value | Explanation |
|:----- |:----------- |
343
| address_cache | (Path to and) filename of the address cache file where the addresses of BACnet devices are stored (as noted above).
Micha Mueller's avatar
Micha Mueller committed
344
| interface | Network interface (IPv4) which is to be used by the plugin to send its "Read Property" requests.
345
| port | Port to use on the interface
346
| timeout |	Value of µ-seconds to wait for a response packet.
347
348
349
| apdu_timeout | Value of µ-seconds before sending a request times out.
| apdu_retries | How often should sending a request be retried.
| templates | One can define template properties in this section for convenience.
350
| factor | Described in the section for the [IPMI-plugin](#IPMI).
351
| devices | Starts the part in the config file where the actual BACnet devices are configured. A BACnet device consists of multiple nested parts: device > objects > properties.
Micha Mueller's avatar
Micha Mueller committed
352
| mqttPart | One can define a partial MQTT topic on each nesting level, which will be appended to the MQTT prefix. Together with the mqttsuffix of a property this will make up the whole MQTT topic of a property (aka sensor). MQTT topic = mqttPrefix + mqttPart (device) + mqttPart (object) + mqttsuffix.
353
354
355
356
357
| instance (device) | Instance of the BACnet-device.
| type | Type of the object within the device.
| instance (object) | Instance of the object within the device.
| id | ID of the property to be read from the BACnet device-object. Assignment of numbers to properties is done according to the enum as defined in `bacenum.h`.

358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
## Opa (Intel Omni-Path Archtiecture)

The Opa plugin enables dcdbpusher to query various counters from omni-path interconnects.

Explanation of the values specific for the Opa plugin:

| Value | Explanation |
|:----- |:----------- |
| hfiNum | Number of which omni-path Host Fabric Interface to query (starting with 1)
| portNum | Number of which omni-path port to query (starting with 1)
| cntData | Name which data counter to query. A list of possible values can be found below.

> NOTE &ensp;&ensp;&ensp; Specifying `hfiNum` or `portNum` within a template sensor is currently not supported.

### counterData

Possible values for cntData:
* portXmitData
* portRcvData
* portXmitPkts
* portRcvPkts
* portMulticastXmitPkts
* portMulticastRcvPkts
* localLinkIntegrityErrors
* fmConfigErrors
* portRcvErrors
* excessiveBufferOverruns
* portRcvConstraintErrors
* portRcvSwitchRelayErrors
* portXmitDiscards
* portXmitConstraintErrors
* portRcvRemotePhysicalErrors
* swPortCongestion
* portXmitWait
* portRcvFECN
* portRcvBECN
* portXmitTimeCong
* portXmitWastedBW
* portXmitWaitData
* portRcvBubble
* portMarkFECN
* linkErrorRecovery
* linkDowned
* uncorrectableErrors

Micha Mueller's avatar
Micha Mueller committed
403
## Writing own plugins
404
Try out the `pluginGenerator/generatePlugin.sh` script!
Micha Mueller's avatar
Micha Mueller committed
405
TODO!
406

Micha Mueller's avatar
Micha Mueller committed
407
#### TODOS
Micha Mueller's avatar
Micha Mueller committed
408
409
* explain sensor group concept
* rework pdu plugin: should be more generic XML plugin
Micha Mueller's avatar
Micha Mueller committed
410
* more about mqtt-topic