mirror of
https://github.com/ClusterCockpit/cc-metric-collector.git
synced 2024-12-26 15:29:04 +01:00
7840de7b82
* Add cpu_used (all-cpu_idle) to CpustatCollector * Update cc-metric-collector.init * Allow selection of timestamp precision in HttpSink * Add comment about precision requirement for cc-metric-store * Fix for API changes in gofish@v0.15.0 * Update requirements to latest version * Read sensors through redfish * Update golang toolchain to 1.21 * Remove stray error check * Update main config in configuration.md * Update Release action to use golang 1.22 stable release, no golang RPMs anymore * Update runonce action to use golang 1.22 stable release, no golang RPMs anymore * Update README.md Use right JSON type in configuration * Update sink's README * Test whether ipmitool or ipmi-sensors can be executed without errors * Little fixes to the prometheus sink (#115) * Add uint64 to float64 cast option * Add prometheus sink to the list of available sinks * Add aggregated counters by gpu for nvlink errors --------- Co-authored-by: Michael Schwarz <schwarz@uni-paderborn.de> * Ccmessage migration (#119) * Add cpu_used (all-cpu_idle) to CpustatCollector * Update cc-metric-collector.init * Allow selection of timestamp precision in HttpSink * Add comment about precision requirement for cc-metric-store * Fix for API changes in gofish@v0.15.0 * Update requirements to latest version * Read sensors through redfish * Update golang toolchain to 1.21 * Remove stray error check * Update main config in configuration.md * Update Release action to use golang 1.22 stable release, no golang RPMs anymore * Update runonce action to use golang 1.22 stable release, no golang RPMs anymore * Switch to CCMessage for all files. --------- Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> * Switch to ccmessage also for latest additions in nvidiaMetric * New Message processor (#118) * Add cpu_used (all-cpu_idle) to CpustatCollector * Update cc-metric-collector.init * Allow selection of timestamp precision in HttpSink * Add comment about precision requirement for cc-metric-store * Fix for API changes in gofish@v0.15.0 * Update requirements to latest version * Read sensors through redfish * Update golang toolchain to 1.21 * Remove stray error check * Update main config in configuration.md * Update Release action to use golang 1.22 stable release, no golang RPMs anymore * Update runonce action to use golang 1.22 stable release, no golang RPMs anymore * New message processor to check whether a message should be dropped or manipulate it in flight * Create a copy of message before manipulation --------- Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> * Update collector's Makefile and go.mod/sum files * Use message processor in router, all sinks and all receivers * Add support for credential file (NKEY) to NATS sink and receiver * Fix JSON keys in message processor configuration * Update docs for message processor, router and the default router config file * Add link to expr syntax and fix regex matching docs * Update sample collectors * Minor style change in collector manager * Some helpers for ccTopology * LIKWID collector: write log owner change only once * Fix for metrics without units and reduce debugging messages for messageProcessor * Use shorted hostname for hostname added by router * Define default port for NATS * CPUstat collector: only add unit for applicable metrics * Add precision option to all sinks using Influx's encoder * Add message processor to all sink documentation * Add units to documentation of cpustat collector --------- Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> Co-authored-by: oscarminus <me@oscarminus.de> Co-authored-by: Michael Schwarz <schwarz@uni-paderborn.de>
54 lines
2.3 KiB
Markdown
54 lines
2.3 KiB
Markdown
## `influxasync` sink
|
|
|
|
The `influxasync` sink uses the official [InfluxDB golang client](https://pkg.go.dev/github.com/influxdata/influxdb-client-go/v2) to write the metrics to an InfluxDB database in a **non-blocking** fashion. It provides only support for V2 write endpoints (InfluxDB 1.8.0 or later).
|
|
|
|
|
|
### Configuration structure
|
|
|
|
```json
|
|
{
|
|
"<name>": {
|
|
"type": "influxasync",
|
|
"database" : "mymetrics",
|
|
"host": "dbhost.example.com",
|
|
"port": "4222",
|
|
"user": "exampleuser",
|
|
"password" : "examplepw",
|
|
"organization": "myorg",
|
|
"ssl": true,
|
|
"batch_size": 200,
|
|
"retry_interval" : "1s",
|
|
"retry_exponential_base" : 2,
|
|
"precision": "s",
|
|
"max_retries": 20,
|
|
"max_retry_time" : "168h",
|
|
"process_messages" : {
|
|
"see" : "docs of message processor for valid fields"
|
|
},
|
|
"meta_as_tags" : []
|
|
}
|
|
}
|
|
```
|
|
|
|
- `type`: makes the sink an `influxdb` sink
|
|
- `database`: All metrics are written to this bucket
|
|
- `host`: Hostname of the InfluxDB database server
|
|
- `port`: Portnumber (as string) of the InfluxDB database server
|
|
- `user`: Username for basic authentification
|
|
- `password`: Password for basic authentification
|
|
- `organization`: Organization in the InfluxDB
|
|
- `ssl`: Use SSL connection
|
|
- `batch_size`: batch up metrics internally, default 100
|
|
- `retry_interval`: Base retry interval for failed write requests, default 1s
|
|
- `retry_exponential_base`: The retry interval is exponentially increased with this base, default 2
|
|
- `max_retries`: Maximal number of retry attempts
|
|
- `max_retry_time`: Maximal time to retry failed writes, default 168h (one week)
|
|
- `precision`: Precision of the timestamp. Valid values are 's', 'ms', 'us' and 'ns'. (default is 's')
|
|
- `process_messages`: Process messages with given rules before progressing or dropping, see [here](../pkg/messageProcessor/README.md) (optional)
|
|
- `meta_as_tags`: print all meta information as tags in the output (deprecated, optional)
|
|
|
|
For information about the calculation of the retry interval settings, see [offical influxdb-client-go documentation](https://github.com/influxdata/influxdb-client-go#handling-of-failed-async-writes)
|
|
|
|
### Using `influxasync` sink for communication with cc-metric-store
|
|
|
|
The cc-metric-store only accepts metrics with a timestamp precision in seconds, so it is required to use `"precision": "s"`. |