README.md 29.7 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Table of contents
1. [Introduction](#introduction)
2. [dcdbpusher](#dcdbpusher)
    1. [Global Configuration](#globalConfiguration)
    2. [Rest API](#restApi)
        1. [Table of queries](#tableOfQueries)
        2. [Ressources for GET request](#ressourcesGET)
        3. [Ressources for PUT request](#ressourcesPUT)
        4. [Examples](#restExamples)
    3. [MQTT topic](#mqttTopic)
3. [Plugins](#plugins)
    1. [IPMI](#ipmi)
    2. [Perf-event](#perf)
        1. [type and config](#perfTypeConfig)
        2. [Footnotes](#perfFootnotes)
    3. [SNMP](#snmp)
    4. [SysFS](#sysfs)
    5. [PDU](#pdu)
    6. [BACnet](#bacnet)
    7. [OPA](#opa)
        1. [counterData](#opaCounterData)
    8. [Writing own plugins](#writingOwnPlugins)
3. [TODOS](#todos)

## Introduction <a name="introduction"></a>
Micha Mueller's avatar
Micha Mueller committed
26
27
28
DCDB (DataCenter DataBase) is a database to collect various (sensor-)values of a datacenter for further analysis.
Harvesting of the data is task of the dcdbpusher.

29
# dcdbpusher <a name="dcdbpusher"></a>
Micha Mueller's avatar
Micha Mueller committed
30

Micha Mueller's avatar
Micha Mueller committed
31
This is a general MQTT pusher which sends values of various sensors to the DCDB-database.
Micha Mueller's avatar
Micha Mueller committed
32
It ships with plugins for BACnet, IPMI, PDU(proprietary Power Delivery Unit, but could be used as XML plugin) ,percounter, SNMP and sysFS.
Micha Mueller's avatar
Micha Mueller committed
33
34
35
36
37
38
39
40
41

Build it by simply running
```bash
make
```
or alternatively use
```bash
make debug
```
Micha Mueller's avatar
Micha Mueller committed
42
within the `dcdbpusher` directory to build a version which will print additional debug-information during runtime.
Micha Mueller's avatar
Micha Mueller committed
43

44
The logic for the various sensors is encapsulated into plugins (shared dynamic libraries; the makefile will take care of compiling them for you). The dcdbpusher will dynamically open the libraries if they are specified in the [global configuration](#GC) file. Vice versa, if selected sensor-functionality, e.g. sysFS is not specified, the corresponding shared library libdcdbplugin_sysfs.so does not have to be present. 
Micha Mueller's avatar
Micha Mueller committed
45

Micha Mueller's avatar
Micha Mueller committed
46
47
48
49
You can run dcdbpusher by executing
```bash
./dcdbpusher path/to/configfile/
```
50
or run
Micha Mueller's avatar
Micha Mueller committed
51
52
53
```bash
./dcdbpusher -h
```
Micha Mueller's avatar
Micha Mueller committed
54
to print the help-section of dcdbpusher.
Micha Mueller's avatar
Micha Mueller committed
55

Micha Mueller's avatar
Micha Mueller committed
56
Dcdbpusher will check the given file-path for the global configuration file which has to be named `global.conf`.
Micha Mueller's avatar
Micha Mueller committed
57

58
### Global Configuration  <a name="globalConfiguration"></a>
Micha Mueller's avatar
Micha Mueller committed
59

Micha Mueller's avatar
Micha Mueller committed
60
61
The global configuration specifies various settings for dcdbpusher in general, e.g. which plugins should be loaded etc.
Please have a look at the provided `config/global.conf` example to get familiar with the file scheme. The example also forms a good starting point for writing a custom `global.conf`. The different sections and values are explained in the following table:
Micha Mueller's avatar
Micha Mueller committed
62

Micha Mueller's avatar
Micha Mueller committed
63
| Value | Explanation |
Micha Mueller's avatar
Micha Mueller committed
64
65
|:----- |:----------- |
| global | Wrapper structure for the global values.
Micha Mueller's avatar
Micha Mueller committed
66
67
68
69
70
71
72
| mqttBroker | Define address and port of the MQTT-broker which collects the messages (sensor values) send by dcdbpusher.
| mqttprefix | To not rewrite a full MQTT-topic for every sensor one can specify here a consistent prefix.
| threads | Specify how many threads should be created to handle the sensors async. Default value of threads is 1. Note that the MQTTPusher always starts an extra thread. So the actual number of started threads is always one more than defined here. Specifying not enough threads can result in a delay for some sensors until they are read.
| verbosity | Level of detail in the logfile (dcdb.log). Set to a value between 5 (all log-messages, default) and 0 (only fatal messages). NOTE: level of verbosity for the command-line log can be set via the -v flag independently when invoking dcdbpusher.
| daemonize | Set to 'true' if dcdbpusher should run detached as daemon. Default is false.
| tempdir | One can specify a writeable directory where dcdbpusher can write its temporary and logging files to. Default is the current (' ./ ' ) directory.
| cacheInterval | Define a time interval in seconds. The last sensor readings within this time interval will be kept. This value can be overwritten by plugins.
Micha Mueller's avatar
Micha Mueller committed
73
| | |
74
75
76
77
78
| restAPI | Bundles all values related to the RestAPI. See the corresponding [section](#restApi) for more information on supported functionality.
| address | Define address and port where the REST API server should run on.
| certificate | Provide the (path and) file which the HTTPS server should use as certificate.
| privateKey | Provide the (path and) file which should be used as corresponding private key for the certificate. If private key and certificate are stored in the same file one should nevertheless provide the path to the cert-file here again.
| dhFile | Provide the (path and) file where Diffie-Hellman parameters for the key exchange are stored.
Micha Mueller's avatar
Micha Mueller committed
79
| authkey | This struct is used to define authentication key tokens for the REST API. Within the struct, define which operations over the REST API are allowed for the token (e.g. PUTReq or GETReq). Each token must be unique.
Micha Mueller's avatar
Micha Mueller committed
80
81
| | |
| plugins | In this section one can specify the plugins which should be used.
Micha Mueller's avatar
Micha Mueller committed
82
| plugin _name_ | The plugin name is used to build the corresponding lib-name (e.g. sysfs --> libdcdbplugin_sysfs.1.0)
Micha Mueller's avatar
Micha Mueller committed
83
84
| path | Specify the path where the plugin (the shared library) is located. If left empty, dcdbpusher will look in the default lib-directories (usr/lib and friends) for the plugin-file.
| config | One can specify a separate config-file (including path to it) for the plugin to use. If not specified, dcdbpusher will look up pluginName.conf (e.g. sysfs.conf) in the same directory where global.conf is located.
Micha Mueller's avatar
Micha Mueller committed
85
86
| | |

87
Formats of the other sensor-specific config-files are explained in the corresponding [subsections](#ipmi). Example configuration-files can be found in the `config/` directory.
Micha Mueller's avatar
Micha Mueller committed
88
89


90
## REST API <a name="restApi"></a>
91

92
Dcdbpusher runs a HTTPS server which provides some functionality to be controlled over a RESTful API. The API is by default hosted at port 8000 on the localhost but the address can be changed in the [`global.conf`](#globalConfiguration).
93
94
95
96

A HTTPS request to dcdbpusher should have the following format: `[GET|PUT] host:port[ressource]?[queries]`.
Tables with allowed ressources sorted by REST methods can be found below. A query consists of a key-value pair of the format `key=value`. For every request at least the authentication token has to be appended as query. Multiple queries are separated by semicolons(';').

97
### Table of queries <a name="tableOfQueries"></a>
98
99
100
101
102
103
104

| Key | Value | Explanation |
|:--- |:----- |:----------- |
| authkey | Your authentication token | The authentication token is required to verify that you are allowed to make this particular request. The authkey query must be included by every request
| interval | Time-value in [s] | Only for GET request of sensor readings average. One can (optionally) specify a custom time interval for which the average of sensor readings is calculated. By default, every sensor reading in the cache is used to calculate the average.


105
### Ressources for GET request <a name="ressourcesGET"></a>
106
107
108
109
110
111
112
113
114

| Ressource | Explanation |
|:--------- |:----------- |
| /help | Responds with a small cheatsheet which presents all currently supported ressources.
| /plugins | (Discovery) Returns a list of all currently loaded plugins.
| /[plugin]/sensors | (Discovery) Returns a list of all sensors which belong to [plugin]. To find out which plugins are available one can request the `/plugins` ressource.
| /[plugin]/[sensor]/avg | Calculates and returns the average of the last sensor readings. Can be combined with the interval query.


115
### Ressources for PUT request <a name="ressourcesPUT"></a>
116
117
118

| Ressource | Explanation |
|:--------- |:----------- |
119
| /[plugin]/[action] | One can request to do a action on the specified plugin. Currently supported actions are `start`, `stop` and `reload` which start or stop the polling of the sensors of the plugin respectively triggers a reload of the plugin configuration. If a configuration reload is requested, all sensors of the plugin are stopped first. Then new sensors are created and started according to the configuration.
120
121


122
### Examples <a name="restExamples"></a>
123
124

Two examples for HTTPS requests:
Micha Mueller's avatar
Micha Mueller committed
125
126
127
128
129

```bash
GET https://localhost:8000/sysfs/freq1/avg?authkey=myToken;interval=15
```
```bash
Micha Mueller's avatar
Micha Mueller committed
130
PUT https://localhost:8000/bacnet/stop?authkey=myToken
Micha Mueller's avatar
Micha Mueller committed
131
```
132

Micha Mueller's avatar
Micha Mueller committed
133
## MQTT topic <a name="mqttTopic"></a>
134

Micha Mueller's avatar
Micha Mueller committed
135
136
137
138
139
For communication between the different DCDB-components (database, dcdbpusher) the [MQTT protocol](https://mqtt.org/) is used. In order to identify each sensor, everyone has to have a unique MQTT topic assigned. A MQTT topic for DCDB consists of exactly 128 bits (= 32 hex characters), not including '/' separators. The topic for a sensor is build by appending up to 4 parts:
1. mqttprefix    (e.g. /00112233445566778899AA)
2. mqttpart of entity (if supported by plugin, e.g. /BB)
3. mqttpart of group    (e.g. /1122)
4. mqttsuffix    (e.g. /3344)
Micha Mueller's avatar
Micha Mueller committed
140

Micha Mueller's avatar
Micha Mueller committed
141
Then the topic for the sensor is /00112233445566778899AA/BB/1122/3344.
142

143
# Plugins <a name ="plugins"></a>
Micha Mueller's avatar
Micha Mueller committed
144

Micha Mueller's avatar
Micha Mueller committed
145
The core of dcdbpusher is responsible of collecting all the values read by the sensors and sending them to the database. However, the main functionality of the sensors comes from the various plugins. Every plugin corresponds to a special sensor functionality.
146
All the different plugins share some same general principles in common regarding the sensor structure and configuration. Those principles should also be obeyed when [writing own plugins](#writingOwnPlugins).
Micha Mueller's avatar
Micha Mueller committed
147
1. There are three hierarchical levels (from bottom up):
Micha Mueller's avatar
Micha Mueller committed
148
149
150
    1. Sensors
    2. Groups
    3. Entities (optional)
Micha Mueller's avatar
Micha Mueller committed
151
152
2. There are no sensors on its own. Every sensor belongs to a group.
3. Multiple groups may or may not be aggregated by an entity. Entities can be optionally used by the plugin developer to aggregate groups which belong together, e.g. because they all query the same host.
Micha Mueller's avatar
Micha Mueller committed
153
154
155
156
157
158
159
4. Every hierarchical level is associated with some attributes. In the following are some hints on how one (when developing own plugins) should decide which attributes are associated with which level. Also for every level the common base attributes are listed (with explanation), which are specified independently of a plugin:
    1. Entities (if present) hold all attributes which are required to query the represented entity or all its associated groups have in common. Common entity attributes:
        * __default__     (One can define the name of a template group (see below) whose values and groups should be used as default)
        * Other entity attributes could be: mqttPart, protocol-version, host address and port.
    2. Groups hold all attributes which multiple sensors belonging to it share in common. Common group attributes:
        * __interval__    (Time in [ms] between two consecutive sensor reads. Default is 1000[ms] = 1[s])
        * __minValues__   (Minimum number of sensor reads the sensors in a group should gather before they are sent together to the database. Useful to reduce MQTT-overhead. Default is 1 (every sensor value is sent on its own))
160
        * __mqttPart__    (Part for the [mqtt-topic](#mqttTopic) all sensors in this group should share in common)
Micha Mueller's avatar
Micha Mueller committed
161
        * __default__     (One can define the name of a template group (see below) whose values and sensors should be used as default)
Micha Mueller's avatar
Micha Mueller committed
162
    3. Sensors hold only those attributes which are necessary to uniquely identify the target sensor. Common base attributes:
163
        * __mqttsuffix__  (to make its [mqtt-topic](#mqttTopic) unique)
Micha Mueller's avatar
Micha Mueller committed
164
165
166
167
5. Be aware that naming of sensor/group/entity is not fixed. A plugin developer can name them as he likes, e.g. counter/multicounter/host.
6. It is possible to define template groups or entities in the config file, but not template sensors (as a sensor should only consists of attributs which make him unique this would not be too useful). To specify a template group/entity simply prefix its definition with `template_` (see the example below). You can reference them later by using the `default` attribute. A template entity can consist of groups and these in turn can consist of sensors. When using a template, all of its attribute values are copied to the actual sensor. Copied attributes can be overwritten in the actual entity/sensor (some of them even should be overwritten, e.g. the mqttPart). However, groups/sensors associated with a template are copied to the actual entity/group and can NOT be overwritten. One can specify further groups/sensors which are then added to those copied from the template. Template entitys/groups itself or sensors within them are never used in live operation of the plugin. They are purely cosmetic for convenient configuration.
 
In the following two abstract config files are shown to visualize the structure, one with the optional entity level and one without. A real example configuration file for every plugin should be provided in the `/config` directory. One should use them as a starting point to write own configuration files. 
Micha Mueller's avatar
Micha Mueller committed
168
```
Micha Mueller's avatar
Micha Mueller committed
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 Without entity:
------------------------------------------------

global {
	mqttprefix /00112233445566778899AABB0000
	cacheInterval 120
	...
}

template_group temp1 {			;template group named temp1 (is not used in live operation)
	interval	1000			;While it is possible define entities/groups/sensors without
	minValues	3				;name it is strictly disregarded. Naming entities/groups/sensors
	mqttPart	AA				;simplifies debugging and especially enables one to reference
								;templates later on. Also names should be always unique.
	sensor s1 {
		mqttsuffix		01
		...						;usually the sensor would require additional attributes
	}

	sensor s2 {
		mqttsuffix		02
		...
	}
}

group g1 {
	default		temp1			;use temp1 as template group
	mqttPart	BB				;overwrite the mqttPart from temp1, to avoid identical
								;mqtt-topics if another group uses the same template
	sensor s3 {					;g1 has now 3 sensors: s1, s2 (both taken over from temp1)
		mqttsuffix		03		;and s3
		...
	}
}

group g2 {						;g2 consists of only one sensor (s21) and uses
 	sensor s21 {				;for every attribute the default value
		mqttsuffix	0000		;by using a longer mqttsuffix we do not need a
		...						;group mqtt-part
	}
}

...
Micha Mueller's avatar
Micha Mueller committed
212
213
```

Micha Mueller's avatar
Micha Mueller committed
214
215
216
217
218
219
220
221
```
 With entity:
------------------------------------------------

global {
	...
}

Micha Mueller's avatar
Micha Mueller committed
222
template_entity temp1 {				;template entity which is not used in live operation
Micha Mueller's avatar
Micha Mueller committed
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
	...								;here go entity attributes

	group g1 {						
		interval	1000
		minValues	3
		mqttPart	AA
		
		sensor s1 {
			mqttsuffix		01
			...						;usually the sensor would require additional attributes
		}
	
		sensor s2 {
			mqttsuffix		02
			...
		}
	}
}

entity ent1 {
	default		temp1				;use temp1 as template entity
	
	group g2 {						;ent1 has now two groups (g1 and g2) with a total of
	 	sensor s21 {				;3 sensors (s1, s2, s21)
			mqttsuffix	0000
			...
		}
	}
}

...
```
Micha Mueller's avatar
Micha Mueller committed
255
One should have noticed the global section in the examples which was not mentioned before. In this section the user can (but is not obligated to) overwrite values from the `global.conf` for this plugin or specify other settings which are global for this plugin.
256

257
## IPMI <a name="ipmi"></a>
Micha Mueller's avatar
Micha Mueller committed
258

Micha Mueller's avatar
Micha Mueller committed
259
The [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) plugin enables dcdbpusher to collect sensor values offered by a baseboard management controller (BMC).
260

261
Explanation of the values specific for the IPMI plugin:
262

Micha Mueller's avatar
Micha Mueller committed
263
| Value | Explanation |
264
265
266
267
268
269
270
271
|:----- |:----------- |
| sessiontimeout | Session timeout value for the IPMI-connection
| retransmissiontimeout | Retransmission timeout value for the IPMI-connection
| username | For the remote IPMI-connection login credentials are required
| password | For the remote IPMI-connection login credentials are required
| cmd | One can define a raw IPMI-command (in hex-notation) to be sent. In this case also the start and stop fields for the response have to be defined. Alternatively, one can define the record-ID of the sensor (see below).
| start | Required if raw command is sent
| stop | Required if raw command is sent
Micha Mueller's avatar
Micha Mueller committed
272
| recordId | Define the record-ID of the sensor to be read. One can look up the corresponding record-IDs for every sensor with the "ipmi-sensors" command line tool (ships with the freeipmi-library). Alternatively, one can define a raw IPMI-command (see above).
273
| factor | One can specify a factor to scale the read value before it is stored in the database (to adjust precision).
274

275
## Perf-event <a name="perf"></a>
Micha Mueller's avatar
Micha Mueller committed
276

Micha Mueller's avatar
Micha Mueller committed
277
The Perfevent functionality is tasked with collecting data from the CPUs various performance counters (PMUs).
278
> NOTE &ensp;&ensp;&ensp; The perf-event plugin measures PMUs for all processes running on a specific CPU. Therefore a value of less than 1 is required in `/proc/sys/kernel/perf_event_paranoid`. Other values (>=1) restrict the access to PMUs. See this [footnote](#fn1) for additional information.
Micha Mueller's avatar
Micha Mueller committed
279

Micha Mueller's avatar
Micha Mueller committed
280
281
282
283
284
285
286
287
288
Explanation of the values specific for the perfevent plugin:

| Value | Explanation |
|:----- |:----------- |
| mqttsuffix | In the context of the perfevent plugin the mqttsuffixes are interpreted as start mqttsuffix. A perfsensor uses a separate MQTT-topic for every CPU. For every used CPU the mqttsuffix is incremented by one. However, dcdbpusher does not allow the same suffix twice! Example: For sensor A the mqttsuffix 00A8 is given and the system on which perfpusher is run has 8 cores. Then perfpusher uses the mqtt-suffixe 00A8, 00A9, 00AA, 00AB, 00AC, 00AD, 00AE, 00AF for the 8 cores. As they are already used by counter A, any other sensor must not use those. Counter B can e.g. start with mqttsuffix 00B0.
| type | Type of which the counter should be. Each type determines different possible values for the config-field. Possible type-values are described below.
| config | Together with the type-field config determines which performance counter should be read. Possible values and what they measure are listed below.
| cpus | One can define a comma-separated list of cpu numbers (also value ranges can be specified, e.g. 2-4 equals 2,3,4). The hardware counter will then be only opened on the specified cpus.

Micha Mueller's avatar
Micha Mueller committed
289

290
### type and config <a name="perfTypeConfig"></a>
Micha Mueller's avatar
Micha Mueller committed
291

Micha Mueller's avatar
Micha Mueller committed
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
(see the [perf_event_open man-page](http://man7.org/linux/man-pages/man2/perf_event_open.2.html) for more detailed explanations)

| Type | Config | Explanation |
|:----:|:------ |:----------- |
| PERF_TYPE_HARDWARE | | generalized hardware CPU events
| " | PERF_COUNT_HW_CPU_CYCLES | total cycles (affected by frequency scaling)
| " | PERF_COUNT_HW_INSTRUCTIONS | retired instructions
| " | PERF_COUNT_HW_CACHE_REFERENCES | cache accesses (usually last level)
| " | PERF_COUNT_HW_CACHE_MISSES | cache misses (usually last level)
| " | PERF_COUNT_HW_BRANCH_INSTRUCTIONS | retired branch instructions
| " | PERF_COUNT_HW_BRANCH_MISSES | mispredicted branch instructions
| " | PERF_COUNT_HW_BUS_CYCLES | bus cycles
| " | PERF_COUNT_HW_STALLED_CYCLES_FRONTEND | stalled cycles during issue
| " | PERF_COUNT_HW_STALLED_CYCLES_BACKEND  | stalled cycles during retirement
| " | PERF_COUNT_HW_REF_CPU_CYCLES | total cycles (unaffected by frequency scaling)
307
| | | |
Micha Mueller's avatar
Micha Mueller committed
308
309
310
311
312
313
314
315
316
317
318
| PERF_TYPE_SOFTWARE | | software events provided by the kernel
| " | PERF_COUNT_SW_CPU_CLOCK | reports CPU clock
| " | PERF_COUNT_SW_TASK_CLOCK | clock count specific to the running task
| " | PERF_COUNT_SW_PAGE_FAULTS | number of page faults
| " | PERF_COUNT_SW_CONTEXT_SWITCHES | count of context switches
| " | PERF_COUNT_SW_CPU_MIGRATIONS | times the process has migrated to a new CPU
| " | PERF_COUNT_SW_PAGE_FAULTS_MIN | number of minor page faults (no disk-I/O)
| " | PERF_COUNT_SW_PAGE_FAULTS_MAJ | number of major page faults (disk-I/O was required)
| " | PERF_COUNT_SW_ALIGNMENT_FAULTS | alignment faults when accessing unaligned memory
| " | PERF_COUNT_SW_EMULATION_FAULTS | number of unimplemented instructions which had to be emulated
| " | PERF_COUNT_SW_DUMMY | placeholder which counts nothing
319
| | | |
Micha Mueller's avatar
Micha Mueller committed
320
321
| PERF_TYPE_TRACEPOINT | | not yet implemented
| PERF_TYPE_HW_CACHE | | not yet implemented
322
| | | |
Micha Mueller's avatar
Micha Mueller committed
323
324
| PERF_TYPE_RAW | | user can define architecture-specific raw events here.
| " | *XXXX* | Config must be an hex value (without 0x prefix). See <sup>[2](#fn2)</sup>
325
| | | |
Micha Mueller's avatar
Micha Mueller committed
326
| PERF_TYPE_BREAKPOINT | --- | config not required, any values will be ignored. However config must still be specified (even if empty)
Micha Mueller's avatar
Micha Mueller committed
327

328
#### Footnotes <a name="perfFootnotes"></a>
Micha Mueller's avatar
Micha Mueller committed
329
330
331

Taken from the [perf_event_open man-page](http://man7.org/linux/man-pages/man2/perf_event_open.2.html):

332
333
334
<a name="fn1">**1**</a>: &ensp; The pid and cpu arguments allow specifying which process and CPU to monitor:  
[...]  
pid == -1 and cpu >= 0  
335
This measures all processes/threads on the specified CPU. This requires CAP_SYS_ADMIN capability or a /proc/sys/kernel/perf_event_paranoid value of less than 1.
336
337
338
339

[...]

The perf_event_paranoid file can be set to restrict access to the performance counters.
Micha Mueller's avatar
Micha Mueller committed
340
341
342
343
344
345
346

| Value | Restriction |
|:-----:|:----------- |
| 2 | allow only user-space measurements (default since Linux 4.6) |
| 1 | allow both kernel and user measurements (default before Linux 4.6) |
| 0 | allow access to CPU-specific data but not raw trace-point samples |
| -1 | no restrictions |
Micha Mueller's avatar
Micha Mueller committed
347
348
349
	
The existence of the perf_event_paranoid file is the official method for determining if a kernel supports perf_event_open()

Micha Mueller's avatar
Micha Mueller committed
350
<a name="fn2">**2**</a>: &ensp; If type is *PERF_TYPE_RAW*, then a custom "raw" config value is needed. Most CPUs support events that are not covered by the "generalized" events. These are implementation defined; see your CPU manual (for example the Intel Volume 3B documentation or the AMD BIOS and Kernel Developer Guide). The libpfm4 library can be used to translate from the name in the architectural manual to the raw hex value perf_event_open() expects in this field.
Micha Mueller's avatar
Micha Mueller committed
351

352
## snmp <a name="snmp"></a>
Micha Mueller's avatar
Micha Mueller committed
353

Micha Mueller's avatar
Micha Mueller committed
354
The SNMP plugin enables dcdbpusher to talk with devices which have an SNMP agent running and query requests from them. A SNMP sensor corresponds to a single value as identified by the unique OID. Sensors are aggregated by connections. See the exemplary snmp.conf file in the `config/` directory.
355
> NOTE &ensp;&ensp;&ensp; In the SNMP context the word privacy is used synonymously for encryption.
Micha Mueller's avatar
Micha Mueller committed
356

Micha Mueller's avatar
Micha Mueller committed
357
358
Explanation of the values specific for the SNMP plugin:

Micha Mueller's avatar
Micha Mueller committed
359
360
361
362
363
364
365
| Value | Explanation |
|:----- |:----------- |
| connection | An aggregating connection
| Type | Type of the SNMP application which runs on the device queried by the connection. Currently only the type Agent is supported.
| Host | Host name of the device which is to be queried.
| Port | The SNMP port should be usually 161. No changes should be required here.
| OIDPrefix | This OIDPrefix is used for all following sensors.
Micha Mueller's avatar
Micha Mueller committed
366
367
368
369
370
371
372
373
374
| |
| Version | Which SNMP version to use (either 2 (maps to 2c) or 3).
| Community | Which SNMP community to use (required only if version 2 is used).
| Username | Username to authenticate with (only required for version 3).
| SecLevel | The security level to be used (only required for version 3). Can be either `noAuthNoPriv` for no authentication and privacy ("privacy" is SNMPs synonym for encryption), `authNoPriv` for only authentication and `authPriv` for authentication and privacy.
| AuthProto | Which protocol to use for authentication (only required for version 3 and if SecLevel != noAuthNoPriv). Can be MD5 or SHA1.
| AuthKey | The passphrase for authentication (only required for version 3 and if SecLevel != noAuthNoPriv). Must be at least 8 characters long.
| PrivProto | Which protocol to use for privacy (only required for version 3 and if SecLevel = AuthPriv). Can be DES or AES.
| PrivKey | The passphrase for privacy encryption (only required for version 3 and if SecLevel = AuthPriv). Must be at least 8 characters long.
Micha Mueller's avatar
Micha Mueller committed
375
| mqttPart | Connection specific MQTT-part which is appended to the MQTT-prefix and succeded by the sensor specific suffix.
Micha Mueller's avatar
Micha Mueller committed
376
| |
Micha Mueller's avatar
Micha Mueller committed
377
| OID | OID suffix which together with the OIDPrefix forms the unique OID identifying a value to query.
Micha Mueller's avatar
Micha Mueller committed
378
| passphrase | has to be at least 8 characters long
Micha Mueller's avatar
Micha Mueller committed
379

380
## sysFS <a name="sysfs"></a>
Micha Mueller's avatar
Micha Mueller committed
381
382

SysFS sensors read data from sysFS files. The configuration file of the plugin corresponds to the generic plugin configuration with standalone sensors. Additionally for a sysFS sensor the following parameters are mandatory/possible:
383

Micha Mueller's avatar
Micha Mueller committed
384
385
Explanation of the values specific for the sysFS plugin:

Micha Mueller's avatar
Micha Mueller committed
386
387
388
| Value | Explanation |
|:----- |:----------- |
| path | Path to the sysFS file the sensor should read from. This parameter is mandatory.
389
| filter | One can define an optional filter if the sysFS file consists of more than only the sensor value. Please note the following points for filters: <br> 1.  The filter supports substitutions. For substitution sed syntax ("s/.../.../") is used. Therefore extended regular expressions (ERE) are used as regex-syntax. ERE is closest to Basic RE (BRE), which is actually used by sed, but requires less escaping. <br> 2.  If a \ ("backslash") is needed in the regex (for escaping), always use \\ ("double backslash") as the regex is read in as string and strings also escape with backslash <br> 3.  Whitespaces are actually used as value separators in the config files. If your filter requires whitespaces either use [[:space:]] in the regex or put it in quotation marks ("") <br> 4.  To be able to reference parts of the match (for substitution) use groups. Groups are created with parentheses. <br>  5.  If using character classes like [[:digit:]] always make sure to use double brackets ("[[" and "]]") or they will not be recognized. <br>  See [ERE-syntax](https://www.gnu.org/software/sed/manual/html_node/ERE-syntax.html#ERE-syntax) <br>  See [substitution syntax](http://www.boost.org/doc/libs/1_65_1/libs/regex/doc/html/boost_regex/format/sed_format.html)
390

391
## PDU <a name="pdu"></a>
392
393
394

The Power Delivery Unit (PDU) plugin is in charge of sending a network-request to the PDUs and gathering specified sensor data from the XML-file response.

Micha Mueller's avatar
Micha Mueller committed
395
Explanation of the values specific for the PDU plugin:
Micha Mueller's avatar
Micha Mueller committed
396

397
| Value | Explanation |
398
|:----- |:----------- |
399
| host | Hostname and (optional) port where to fetch the XML-file with sensor data from. If no port is specified, 443 is used. The plugin requests the file via HTTPS.
400
| TTL | To avoid requesting a current XML-file every time a sensor wants to read his value, one can define a time to live (TTL) for the file here. A new XML-file is requested at the earliest if the TTL has expired. Default value is 1000[ms].
Micha Mueller's avatar
Micha Mueller committed
401
| request | Define the request to be sent to the host via HTTPS as a string. One should put the request in quotation marks (' " ') to enable the use of whitespaces within the request. Special characters (like usage of ' " ' within the request) should be escaped (' " ' --> ' \" '; ' \ ' --> ' \\\\ '; newline --> ' \n '; ...).
402
| path | Define a dot-separated path to the value to be read in the XML file. One can specify attribute values a node has to fulfil in brackets after the node. Even multiple (comma-separated) attributes can be given, however no whitespaces should be used (!) as they will not be filtered and could therefore be treat as part of the attributes name.
403

404
## BACnet <a name="bacnet"></a>
405
406

The BACnet plugin enables dcdbpusher to communicate and request data from devices which communicate via the BACnet protocol. A so called "read property" request is sent by the plugin to the BACnet devices as configured in the config file. The response value is then stored in the database. Usually one is only interested in collecting the current reading of a BACnet device (property PROP_PRESENT_VALUE, ID 85). However, also reading of other properties is supported.
407
> NOTE &ensp;&ensp;&ensp; On startup BACnet plugin does no device discovery. Instead it relies on the user providing a file with addresses of all required BACnet devices. One can generate such an address-file for example by using the `bacwi` demo tool provided by the BACnet-Stack.
408
409
410
411
412

Explanation of the values specific for the BACnet plugin:

| Value | Explanation |
|:----- |:----------- |
413
| address_cache | (Path to and) filename of the address cache file where the addresses of BACnet devices are stored (as noted above).
Micha Mueller's avatar
Micha Mueller committed
414
| interface | Network interface (IPv4) which is to be used by the plugin to send its "Read Property" requests.
415
| port | Port to use on the interface
416
| timeout |	Value of µ-seconds to wait for a response packet.
417
418
419
| apdu_timeout | Value of µ-seconds before sending a request times out.
| apdu_retries | How often should sending a request be retried.
| templates | One can define template properties in this section for convenience.
420
| factor | Described in the section for the [IPMI-plugin](#ipmi).
421
422
423
424
425
426
| devices | Starts the part in the config file where the actual BACnet devices are configured. A BACnet device consists of multiple nested parts: device > objects > properties.
| instance (device) | Instance of the BACnet-device.
| type | Type of the object within the device.
| instance (object) | Instance of the object within the device.
| id | ID of the property to be read from the BACnet device-object. Assignment of numbers to properties is done according to the enum as defined in `bacenum.h`.

427
## Opa (Intel Omni-Path Architecture) <a name="opa"></a>
428
429
430
431
432
433
434
435
436
437
438

The Opa plugin enables dcdbpusher to query various counters from omni-path interconnects.

Explanation of the values specific for the Opa plugin:

| Value | Explanation |
|:----- |:----------- |
| hfiNum | Number of which omni-path Host Fabric Interface to query (starting with 1)
| portNum | Number of which omni-path port to query (starting with 1)
| cntData | Name which data counter to query. A list of possible values can be found below.

439
### counterData <a name="opaCounterData"></a>
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469

Possible values for cntData:
* portXmitData
* portRcvData
* portXmitPkts
* portRcvPkts
* portMulticastXmitPkts
* portMulticastRcvPkts
* localLinkIntegrityErrors
* fmConfigErrors
* portRcvErrors
* excessiveBufferOverruns
* portRcvConstraintErrors
* portRcvSwitchRelayErrors
* portXmitDiscards
* portXmitConstraintErrors
* portRcvRemotePhysicalErrors
* swPortCongestion
* portXmitWait
* portRcvFECN
* portRcvBECN
* portXmitTimeCong
* portXmitWastedBW
* portXmitWaitData
* portRcvBubble
* portMarkFECN
* linkErrorRecovery
* linkDowned
* uncorrectableErrors

470
471
## Writing own plugins <a name="writingOwnPlugins"></a>
First make sure you read the [plugins](#plugins) section.
472
Try out the `pluginGenerator/generatePlugin.sh` script!
Micha Mueller's avatar
Micha Mueller committed
473
TODO!
474

475
#### TODOS <a name="todos"></a>
Micha Mueller's avatar
Micha Mueller committed
476
* explain special cases in the code for every plugin?
Micha Mueller's avatar
Micha Mueller committed
477
* more about mqtt-topic