Commit Graph

687 Commits

Author SHA1 Message Date
Thomas Roehl
857903be2b Skip disks in DiskstatCollector that have size=0 2022-05-13 13:31:22 +02:00
Thomas Gruber
80d92d6d28
Units with cc-units (#64)
* Add option to normalize units with cc-unit

* Add unit conversion to router

* Add option to change unit prefix in the router

* Add to MetricRouter README

* Add order of operations in router to README

* Use second add_tags/del_tags only if metric gets renamed
2022-05-13 13:30:02 +02:00
Thomas Roehl
8068e59818 Update handling of LIKWID headers. Download only if not already present in the system. Fixes #73 2022-05-13 13:14:47 +02:00
Thomas Roehl
8abedac0fe Use Golang duration parser for 'interval' and 'duration'
in main config
2022-05-13 12:33:33 +02:00
Holger Obermaier
ee4bd558f1 Cleanup: Remove unused code 2022-05-06 11:44:57 +02:00
Holger Obermaier
186a62a86b Fix: influx sink ignores config batch_size.
Feature: Add Redfish receiver
2022-05-04 13:00:56 +02:00
Holger Obermaier
e098c33179 Add some golang debug options 2022-05-04 12:48:46 +02:00
Thomas Roehl
38d4e0a730 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-05-04 11:54:55 +02:00
Thomas Roehl
54d14519ca Skip mount points in DiskstatCollector if statfs() call does not work (bind mounts, ...) 2022-05-04 11:54:34 +02:00
Holger Obermaier
c35ac9dba8 Flush if batch size is reached 2022-05-04 11:28:06 +02:00
Holger Obermaier
c019f8e7ad Reuse tags and meta data tags 2022-05-03 17:55:33 +02:00
Holger Obermaier
fb6f6a4daa Fix GPFS collector last state handling 2022-05-02 16:57:19 +02:00
Holger Obermaier
9d6d0dbd93 Delete empty tags and meta data tags 2022-04-20 14:39:26 +02:00
Holger Obermaier
c2d4272fdf Clear workerInput channel after done event 2022-04-20 12:36:45 +02:00
Holger Obermaier
8c73095548 Allow to shutdown redfish receiver during metric read 2022-04-20 09:58:02 +02:00
Holger Obermaier
31c5c89a5a Fix: Close done channel 2022-04-19 14:01:23 +02:00
Holger Obermaier
bf9c7e1830 Update requirements 2022-04-19 12:15:51 +02:00
Holger Obermaier
48d34bf564 Adopt sinks.json for new meta_as_tags usage 2022-04-19 12:06:53 +02:00
Holger Obermaier
a1d85fa886 Add redfish receiver 2022-04-19 12:05:03 +02:00
Holger Obermaier
96ee16398e Removed unused done channel and wg wait group 2022-04-19 11:53:11 +02:00
Holger Obermaier
e7b8088c41 Extended go routine use case in sample receiver 2022-04-19 11:42:46 +02:00
Thomas Roehl
017cd58247 Updating page for LikwidCollector 2022-04-05 10:57:09 +02:00
Thomas Roehl
36dd440864 Merge branch 'develop' into main 2022-04-04 15:16:33 +02:00
Thomas Roehl
7b098e0b1b Fix for missing metrics in LikwidCollector is hwthread is inactive 2022-04-04 15:16:11 +02:00
Thomas Roehl
229a57b16a Merge branch 'develop' into main 2022-04-04 11:49:33 +02:00
Thomas Roehl
70a9530aba Set WriteFailedCallback to get some error message 2022-04-04 11:48:54 +02:00
Thomas Roehl
2f0b6057ca Merge branch 'develop' into main
- MetricRouter: Fix interval_timestamp option
- InfluxSink & InfluxAsyncSink: Add own flush mechanism
- InfluxSink & InfluxAsyncSink: Use default options, overwrite if
configured otherwise
- InfinibandCollector: Add units to metrics
- LikwidCollector: Fix for dying accessdaemon
2022-04-04 02:59:42 +02:00
Thomas Roehl
69f7c19659 InfluxAsyncSink: Add custom flush mechanism 2022-04-04 02:56:23 +02:00
Thomas Roehl
ecdb4c1bcf Add debug message when updating interval_timestep 2022-04-04 02:55:44 +02:00
Thomas Roehl
4d5b1adbc8 Fix for interval_timestamp option 2022-04-04 02:26:04 +02:00
Thomas Roehl
28348bd108 InfluxSink: Use batch&flush logic from HttpSink 2022-04-01 18:37:45 +02:00
Thomas Roehl
a3b9d8a90b HttpSink: Use sink name in error outputs 2022-04-01 18:36:54 +02:00
Thomas Roehl
7e43e9171e Use default options. Overwrite if anything is configured differently. Use seconds as precision 2022-04-01 17:26:56 +02:00
Thomas Roehl
5d25a7bf12 Add units to InfiniBandCollector 2022-04-01 17:14:26 +02:00
Thomas Roehl
83b4343310 Likwid receives signal at first Read, check when re-initializing 2022-04-01 17:10:31 +02:00
Thomas Roehl
f1d3cabdc6 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-04-01 12:45:25 +02:00
Thomas Röhl
4763733d8d Merge branch 'develop' into main 2022-03-31 11:57:19 +02:00
Thomas Gruber
2a014b6fba
Read unit of values from /proc/meminfo (#68) 2022-03-31 11:56:31 +02:00
Thomas Röhl
16e898ecca Merge branch 'develop' into main 2022-03-31 11:47:02 +02:00
Thomas Roehl
50479f9325 Move all LIKWID related stuff to late initialization routine 2022-03-24 18:12:23 +01:00
Thomas Roehl
e0e91844bc Use late initialization of LIKWID and catch access daemon death. Fixes #70 and fixes #71. 2022-03-24 17:56:51 +01:00
Thomas Roehl
296225f3a8 Always export all metrics in NfsCollectors 2022-03-24 13:50:35 +01:00
Thomas Roehl
43bcce6fb5 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-22 18:05:38 +01:00
Thomas Roehl
622e94ae0e Fix DieList() if system does not support dies. Explicitly set entries in CpuData list 2022-03-22 15:58:10 +01:00
Thomas Roehl
c506114480 Add processing order to MetricRouter README and add missing options 2022-03-18 12:29:00 +01:00
Thomas Roehl
657543dded Ensure max_forward is at least 1 2022-03-18 12:28:52 +01:00
Thomas Roehl
4851382ad7 Merge branch 'develop' into main 2022-03-16 19:08:13 +01:00
Thomas Roehl
b66fdd1436 Add missing socket->thread_id map for LikwidCollector 2022-03-16 19:04:39 +01:00
Thomas Gruber
3f76947f54
Merge latest developments into main (#67)
* Update configuration.md

Add an additional receiver to have better alignment of components

* Change default GpfsCollector command to `mmpmon` (#53)

* Set default cmd to 'mmpmon'

* Reuse looked up path

* Cast const to string

* Just download LIKWID to get the headers (#54)

* Just download LIKWID to get the headers

* Remove perl-Data-Dumper from BuildRequires, only required by LIKWID build

* Add HttpReceiver as counterpart to the HttpSink (#49)

* Use GBytes as unit for large memory numbers

* Make maxForward configurable, save old name in meta in rename metrics and make the hostname tag key configurable

* Single release action (#55)

Building all RPMs and releasing in a single workflow

* Makefile target to build binary-only Debian packages (#61)

* Add 'install' and 'DEB' make targets to build binary-only Debian packages

* Add control file for DEB builds

* Use a single line for bash loop in make clean

* Add config options for retry intervals of InfluxDB clients (#59)

* Refactoring of LikwidCollector and metric units (#62)

* Reduce complexity of LikwidCollector and allow metric units

* Add unit to LikwidCollector docu and fix some typos

* Make library path configurable

* Use old metric name in Ganglia if rename has happened in the router (#60)

* Use old metric name if rename has happened in the router

* Also check for Ganglia renames for the oldname

* Derived metrics (#57)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Simplified iota usage

* Move unit tag to meta data tags

* Derived metrics (#65)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

* Update LustreCollector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Meta to tags list and map for sinks (#63)

* Change ccMetric->Influx functions

* Use a meta_as_tags string list in config but create a lookup map afterwards

* Add meta as tag logic to sampleSink

* Fix staticcheck warnings (#66)

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-03-15 16:41:11 +01:00
Thomas Gruber
c182d295f4
Fix staticcheck warnings (#66) 2022-03-15 16:38:20 +01:00