Commit Graph

656 Commits

Author SHA1 Message Date
Thomas Roehl
a3b9d8a90b HttpSink: Use sink name in error outputs 2022-04-01 18:36:54 +02:00
Thomas Roehl
7e43e9171e Use default options. Overwrite if anything is configured differently. Use seconds as precision 2022-04-01 17:26:56 +02:00
Thomas Roehl
5d25a7bf12 Add units to InfiniBandCollector 2022-04-01 17:14:26 +02:00
Thomas Roehl
83b4343310 Likwid receives signal at first Read, check when re-initializing 2022-04-01 17:10:31 +02:00
Thomas Roehl
f1d3cabdc6 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-04-01 12:45:25 +02:00
Thomas Röhl
4763733d8d Merge branch 'develop' into main 2022-03-31 11:57:19 +02:00
Thomas Gruber
2a014b6fba
Read unit of values from /proc/meminfo (#68) 2022-03-31 11:56:31 +02:00
Thomas Röhl
16e898ecca Merge branch 'develop' into main 2022-03-31 11:47:02 +02:00
Thomas Roehl
50479f9325 Move all LIKWID related stuff to late initialization routine 2022-03-24 18:12:23 +01:00
Thomas Roehl
e0e91844bc Use late initialization of LIKWID and catch access daemon death. Fixes #70 and fixes #71. 2022-03-24 17:56:51 +01:00
Thomas Roehl
296225f3a8 Always export all metrics in NfsCollectors 2022-03-24 13:50:35 +01:00
Thomas Roehl
43bcce6fb5 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-22 18:05:38 +01:00
Thomas Roehl
622e94ae0e Fix DieList() if system does not support dies. Explicitly set entries in CpuData list 2022-03-22 15:58:10 +01:00
Thomas Roehl
c506114480 Add processing order to MetricRouter README and add missing options 2022-03-18 12:29:00 +01:00
Thomas Roehl
657543dded Ensure max_forward is at least 1 2022-03-18 12:28:52 +01:00
Thomas Roehl
4851382ad7 Merge branch 'develop' into main 2022-03-16 19:08:13 +01:00
Thomas Roehl
b66fdd1436 Add missing socket->thread_id map for LikwidCollector 2022-03-16 19:04:39 +01:00
Thomas Gruber
3f76947f54
Merge latest developments into main (#67)
* Update configuration.md

Add an additional receiver to have better alignment of components

* Change default GpfsCollector command to `mmpmon` (#53)

* Set default cmd to 'mmpmon'

* Reuse looked up path

* Cast const to string

* Just download LIKWID to get the headers (#54)

* Just download LIKWID to get the headers

* Remove perl-Data-Dumper from BuildRequires, only required by LIKWID build

* Add HttpReceiver as counterpart to the HttpSink (#49)

* Use GBytes as unit for large memory numbers

* Make maxForward configurable, save old name in meta in rename metrics and make the hostname tag key configurable

* Single release action (#55)

Building all RPMs and releasing in a single workflow

* Makefile target to build binary-only Debian packages (#61)

* Add 'install' and 'DEB' make targets to build binary-only Debian packages

* Add control file for DEB builds

* Use a single line for bash loop in make clean

* Add config options for retry intervals of InfluxDB clients (#59)

* Refactoring of LikwidCollector and metric units (#62)

* Reduce complexity of LikwidCollector and allow metric units

* Add unit to LikwidCollector docu and fix some typos

* Make library path configurable

* Use old metric name in Ganglia if rename has happened in the router (#60)

* Use old metric name if rename has happened in the router

* Also check for Ganglia renames for the oldname

* Derived metrics (#57)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Simplified iota usage

* Move unit tag to meta data tags

* Derived metrics (#65)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

* Update LustreCollector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Meta to tags list and map for sinks (#63)

* Change ccMetric->Influx functions

* Use a meta_as_tags string list in config but create a lookup map afterwards

* Add meta as tag logic to sampleSink

* Fix staticcheck warnings (#66)

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-03-15 16:41:11 +01:00
Thomas Gruber
c182d295f4
Fix staticcheck warnings (#66) 2022-03-15 16:38:20 +01:00
Thomas Roehl
beebcd7145 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-15 16:16:39 +01:00
Thomas Gruber
57629a2e0a
Meta to tags list and map for sinks (#63)
* Change ccMetric->Influx functions

* Use a meta_as_tags string list in config but create a lookup map afterwards

* Add meta as tag logic to sampleSink
2022-03-15 16:16:26 +01:00
Thomas Roehl
082eea525a Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-15 16:10:41 +01:00
Thomas Gruber
aa1afd745e
Derived metrics (#65)
* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

* Update LustreCollector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-03-15 16:09:47 +01:00
Thomas Roehl
2b8266d1d2 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-15 15:24:29 +01:00
Holger Obermaier
992b19d354 Move unit tag to meta data tags 2022-03-11 14:47:18 +01:00
Holger Obermaier
0b08ca9ae0 Simplified iota usage 2022-03-11 14:09:22 +01:00
Thomas Gruber
f6dae7c013
Derived metrics (#57)
* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-03-11 13:48:18 +01:00
Thomas Gruber
1de3dda7be
Use old metric name in Ganglia if rename has happened in the router (#60)
* Use old metric name if rename has happened in the router

* Also check for Ganglia renames for the oldname
2022-03-11 13:44:32 +01:00
Thomas Gruber
73f22c1041
Refactoring of LikwidCollector and metric units (#62)
* Reduce complexity of LikwidCollector and allow metric units

* Add unit to LikwidCollector docu and fix some typos

* Make library path configurable
2022-03-11 13:43:17 +01:00
Thomas Gruber
c9b8fcdaa7
Add config options for retry intervals of InfluxDB clients (#59) 2022-03-11 13:43:03 +01:00
Thomas Roehl
17f37583fc Use a single line for bash loop in make clean 2022-03-11 13:41:13 +01:00
Thomas Gruber
fb9ea992ea
Makefile target to build binary-only Debian packages (#61)
* Add 'install' and 'DEB' make targets to build binary-only Debian packages

* Add control file for DEB builds
2022-03-11 13:39:09 +01:00
Thomas Gruber
21edca5f88
Single release action (#55)
Building all RPMs and releasing in a single workflow
2022-03-11 13:37:47 +01:00
Thomas Roehl
d835724d93 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-09 17:02:58 +01:00
Thomas Roehl
3cf2f69a07 Make maxForward configurable, save old name in meta in rename metrics and make the hostname tag key configurable 2022-03-09 11:23:17 +01:00
Thomas Roehl
e7f7e68095 Use GBytes as unit for large memory numbers 2022-03-09 11:05:26 +01:00
Thomas Roehl
c5082bbffe Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-07 16:13:08 +01:00
Thomas Gruber
c0e600269a
Add HttpReceiver as counterpart to the HttpSink (#49) 2022-03-05 17:30:55 +01:00
Thomas Gruber
f2486abeab
Just download LIKWID to get the headers (#54)
* Just download LIKWID to get the headers

* Remove perl-Data-Dumper from BuildRequires, only required by LIKWID build
2022-03-05 17:30:40 +01:00
Thomas Gruber
21864e0ac4
Change default GpfsCollector command to mmpmon (#53)
* Set default cmd to 'mmpmon'

* Reuse looked up path

* Cast const to string
2022-03-05 14:42:04 +01:00
Thomas Roehl
3157386b3e Merge branch 'develop' into main 2022-03-04 23:34:28 +01:00
Thomas Gruber
8c668fcc6f
Update configuration.md
Add an additional receiver to have better alignment of components
2022-03-04 18:33:57 +01:00
Thomas Roehl
1961edc659 Add documentation to help configuring the CC metric collector 2022-03-04 15:42:25 +01:00
Mehmet Soysal
547bc0461f
Beegfs collector (#50)
* added beegfs collectors to collectors/README.md

* added beegfs collectors and docs

* added new beegfs collectors to AvailableCollectors list

* Feedback implemented

* changed error type

* changed error to only return

* changed beegfs lookup path

* fixed typo in md files

Co-authored-by: Mehmet Soysal <mehmet.soysal@kit.edu>
2022-03-04 14:35:47 +01:00
Thomas Roehl
7f62975a68 Set proper user for files 2022-03-04 11:52:48 +01:00
Thomas Roehl
5e64c01c9d Fix name for ClusterCockpit user 2022-03-04 11:51:21 +01:00
Thomas Roehl
ff08eaeb43 Set proper user for files 2022-03-04 11:49:55 +01:00
Thomas Roehl
64c41be34c Fix name for ClusterCockpit 2022-03-04 11:37:45 +01:00
Thomas Roehl
f4af520b2a Fix error print in LustreCollector 2022-03-04 11:32:39 +01:00
Thomas Roehl
f1d2828e1d Fix error print in LustreCollector 2022-03-04 11:32:10 +01:00