ClusterCockpit/cc-metric-collector
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-metric-collector.git synced 2026-07-26 10:50:37 +02:00
Code Issues Packages Projects Releases Wiki Activity
432 Commits 37 Branches 23 Tags
50479f932587781e51c679a597ad0d30d8a8920a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Thomas Roehl 50479f9325 Move all LIKWID related stuff to late initialization routine 2022-03-24 18:12:23 +01:00
Thomas Roehl e0e91844bc Use late initialization of LIKWID and catch access daemon death. Fixes #70 and fixes #71. 2022-03-24 17:56:51 +01:00
Thomas Roehl 296225f3a8 Always export all metrics in NfsCollectors 2022-03-24 13:50:35 +01:00
Thomas Roehl 622e94ae0e Fix DieList() if system does not support dies. Explicitly set entries in CpuData list 2022-03-22 15:58:10 +01:00
Thomas Roehl c506114480 Add processing order to MetricRouter README and add missing options 2022-03-18 12:29:00 +01:00
Thomas Roehl 657543dded Ensure max_forward is at least 1 2022-03-18 12:28:52 +01:00
Thomas Roehl b66fdd1436 Add missing socket->thread_id map for LikwidCollector 2022-03-16 19:04:39 +01:00
Thomas GruberandGitHub c182d295f4 Fix staticcheck warnings (#66) 2022-03-15 16:38:20 +01:00
Thomas GruberandGitHub 57629a2e0a Meta to tags list and map for sinks (#63) 
* Change ccMetric->Influx functions

* Use a meta_as_tags string list in config but create a lookup map afterwards

* Add meta as tag logic to sampleSink
 2022-03-15 16:16:26 +01:00
Thomas GruberGitHubHolger Obermaier
 		aa1afd745e Derived metrics (#65) 
* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

* Update LustreCollector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-03-15 16:09:47 +01:00
Thomas GruberGitHubHolger Obermaier
 		f6dae7c013 Derived metrics (#57) 
* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-03-11 13:48:18 +01:00
Thomas GruberandGitHub 1de3dda7be Use old metric name in Ganglia if rename has happened in the router (#60) 
* Use old metric name if rename has happened in the router

* Also check for Ganglia renames for the oldname
 2022-03-11 13:44:32 +01:00
Thomas GruberandGitHub 73f22c1041 Refactoring of LikwidCollector and metric units (#62) 
* Reduce complexity of LikwidCollector and allow metric units

* Add unit to LikwidCollector docu and fix some typos

* Make library path configurable
 2022-03-11 13:43:17 +01:00
Thomas GruberandGitHub c9b8fcdaa7 Add config options for retry intervals of InfluxDB clients (#59) 2022-03-11 13:43:03 +01:00
Thomas Roehl 17f37583fc Use a single line for bash loop in make clean 2022-03-11 13:41:13 +01:00
Thomas GruberandGitHub fb9ea992ea Makefile target to build binary-only Debian packages (#61) 
* Add 'install' and 'DEB' make targets to build binary-only Debian packages

* Add control file for DEB builds
 2022-03-11 13:39:09 +01:00
Thomas GruberandGitHub 21edca5f88 Single release action (#55) 
Building all RPMs and releasing in a single workflow
 2022-03-11 13:37:47 +01:00
Thomas Roehl 3cf2f69a07 Make maxForward configurable, save old name in meta in rename metrics and make the hostname tag key configurable 2022-03-09 11:23:17 +01:00
Thomas Roehl e7f7e68095 Use GBytes as unit for large memory numbers 2022-03-09 11:05:26 +01:00
Thomas GruberandGitHub c0e600269a Add HttpReceiver as counterpart to the HttpSink (#49) 2022-03-05 17:30:55 +01:00
Thomas GruberandGitHub f2486abeab Just download LIKWID to get the headers (#54) 
* Just download LIKWID to get the headers

* Remove perl-Data-Dumper from BuildRequires, only required by LIKWID build
 2022-03-05 17:30:40 +01:00
Thomas GruberandGitHub 21864e0ac4 Change default GpfsCollector command to mmpmon (#53) 
* Set default cmd to 'mmpmon'

* Reuse looked up path

* Cast const to string
 2022-03-05 14:42:04 +01:00
Thomas GruberandGitHub 8c668fcc6f Update configuration.md 
Add an additional receiver to have better alignment of components
 2022-03-04 18:33:57 +01:00
Thomas Roehl 1961edc659 Add documentation to help configuring the CC metric collector 2022-03-04 15:42:25 +01:00
Thomas Roehl 7f62975a68 Set proper user for files 2022-03-04 11:52:48 +01:00
Thomas Roehl 5e64c01c9d Fix name for ClusterCockpit user 2022-03-04 11:51:21 +01:00
Thomas Roehl f1d2828e1d Fix error print in LustreCollector 2022-03-04 11:32:10 +01:00
Thomas Roehl 07af660e50 Merge branch 'main' into develop 2022-03-03 17:33:13 +01:00
Thomas Roehl 31f10b3163 Fix user creation type 2022-03-03 17:31:15 +01:00
Thomas Roehl 9ece27eec6 Use right systemd macro to create the user 2022-03-03 17:26:32 +01:00
Thomas Roehl af4c468ab5 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-03-03 17:24:43 +01:00
Thomas Roehl b3030a8d44 Use right systemd macro to create the user 2022-03-03 17:24:32 +01:00
Thomas Roehl fdbdb79527 Merge branch 'develop' into main 2022-03-03 13:45:51 +01:00
Thomas Roehl 948c34d74d Add user creation in RPM 2022-03-03 13:43:43 +01:00
Thomas Roehl 60de21c41e Switch access mode of LikwidCollector in config file 2022-03-03 13:03:58 +01:00
Thomas Roehl 276c00442a Add option to LustreCollector to call lctl with sudo 2022-03-03 13:02:00 +01:00
Thomas Roehl c61b8d2877 Set name change in Makefile 2022-03-03 11:03:51 +01:00
Thomas Roehl 6023abd028 Rename main file to match with executable name 2022-03-03 11:02:37 +01:00
Thomas Roehl 0753c81156 Add/Remove clustercockpit user and group in RPM 2022-03-02 15:10:14 +01:00
Thomas Roehl 092e7f6a71 Add section how to temporarly disable LIKWID access to page 2022-03-02 13:54:43 +01:00
Thomas Roehl f7e8b52667 Run RPM build actions only on tag push 2022-03-02 13:21:54 +01:00
Thomas Roehl 02baef8c71 Merge branch 'develop' into main 2022-03-02 10:38:17 +01:00
Thomas Roehl d40163cf8f Update README and receiver-specific pages 2022-02-28 17:26:28 +01:00
Thomas GruberGitHubHolger ObermaierHolger Obermaier
 		d98076c792 Merge current development version into main (#48) 
* DiskstatCollector: cast part_max_used metric to int

* Add uint types to GangliaSink and LibgangliaSink

* Use new sink instances to allow multiple of same sink type

* Update sink README and SampleSink

* Use new receiver instances to allow multiple of same receiver type

* Fix metric scope in likwid configuration script

* Mention likwid config script in LikwidCollector README

* Refactor: Embed Init() into New() function

* Refactor: Embed Init() into New() function

* Fix: MetricReceiver uses uninitialized values, when initialization fails

* Use Ganglia configuration (#44)

* Copy all metric configurations from original Ganglia code

* Use metric configurations from Ganglia for some metrics

* Format value string also for known metrics

* Numa-aware memstat collector (#45)

* Add samples for collectors, sinks and receivers

* Ping InfluxDB server after connecting to recognize faulty connections

* Add sink for Prometheus monitoring system (#46)

* Add sink for Prometheus monitoring system

* Add prometheus sink to README

* Add scraper for Prometheus clients (#47)

Co-authored-by: Holger Obermaier <holgerob@gmx.de>
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-02-25 14:49:49 +01:00
Thomas GruberandGitHub a203370aaa Add scraper for Prometheus clients (#47) 2022-02-25 14:46:29 +01:00
Thomas GruberandGitHub f099a311a0 Add sink for Prometheus monitoring system (#46) 
* Add sink for Prometheus monitoring system

* Add prometheus sink to README
 2022-02-25 14:33:20 +01:00
Thomas Roehl fe3a8d59b0 Ping InfluxDB server after connecting to recognize faulty connections 2022-02-25 13:51:52 +01:00
Thomas Roehl bac1f18b1d Add samples for collectors, sinks and receivers 2022-02-25 13:47:19 +01:00
Thomas GruberandGitHub c8bca59de4 Numa-aware memstat collector (#45) 2022-02-24 18:27:05 +01:00
Thomas GruberandGitHub 16c03d2aa2 Use Ganglia configuration (#44) 
* Copy all metric configurations from original Ganglia code

* Use metric configurations from Ganglia for some metrics

* Format value string also for known metrics
 2022-02-24 18:22:20 +01:00
Thomas Roehl d542f32baa Mention likwid config script in LikwidCollector README 2022-02-22 17:46:44 +01:00
Thomas Roehl 6b6566b0aa Fix metric scope in likwid configuration script 2022-02-22 17:46:17 +01:00
Thomas Roehl 3598aed090 Use new receiver instances to allow multiple of same receiver type 2022-02-22 16:33:38 +01:00
Thomas Roehl 24e12ccc57 Update sink README and SampleSink 2022-02-22 16:19:46 +01:00
Thomas Roehl 18a226183c Use new sink instances to allow multiple of same sink type 2022-02-22 16:15:25 +01:00
Thomas Roehl 9cfbe10247 Add uint types to GangliaSink and LibgangliaSink 2022-02-22 15:51:08 +01:00
Thomas Roehl 66275ecf74 DiskstatCollector: cast part_max_used metric to int 2022-02-22 15:50:49 +01:00
Thomas GruberandGitHub b4cc6d54ea Update README.md 2022-02-22 15:10:27 +01:00
Thomas GruberandGitHub 45714fe337 Update README.md 2022-02-22 15:09:12 +01:00
Thomas Roehl 5742721cbd Proper tag name in release 2022-02-21 18:58:13 +01:00
Thomas Roehl 888292dbef Remove ganglia build tag documentation, not needed anymore 2022-02-21 18:53:03 +01:00
Thomas Roehl 1425463a26 Upload RPM not SRPM 2022-02-21 18:49:39 +01:00
Thomas GruberandGitHub 5cd7ac2541 Update rpmbuild.yml 2022-02-21 18:28:07 +01:00
Thomas GruberandGitHub afa831aff8 Update rpmbuild.yml 2022-02-21 18:17:12 +01:00
Thomas Roehl a1328b30e9 Remove CentOS8 from RPMBUILD action. End of Life 2022-02-21 17:16:07 +01:00
Thomas Roehl 842395682c Remove -lganglia ldflag and check dl lib 2022-02-21 17:01:13 +01:00
Thomas Roehl 3ade75490c Add RHEL UBI 8 to rpmbuild action 2022-02-21 15:09:42 +01:00
Thomas Roehl eed9cd227c Remove doubled import and remove merge artifacts 2022-02-21 14:50:11 +01:00
Thomas Roehl 24a2c9992f Merge branch 'develop' into main 2022-02-21 14:32:24 +01:00
Thomas GruberandGitHub f683f2e6da Dynamically load liblikwid (#40) 
* Check whether LIKWID library is present

* Generalize nan_to_zero option to invalid_to_zero including +Inf,+Inf and NaN

* Remove double error printing and return if measurements do not work
 2022-02-21 13:29:33 +01:00
Thomas GruberandGitHub ea5b3bdbd6 Use receiver names from config (#34) 
* Use common configuration format of 'receiver_name' -> 'receiver_config'

* Adjust receiver configuration files
 2022-02-21 12:45:08 +01:00
Thomas GruberandGitHub 435528fa97 Split diskstat Collector (#38) 
* Split diskstats (free, total space) and iostats (reads, writes, ...

* Add iostat Collector to CollectorManager
 2022-02-21 12:44:26 +01:00
Thomas Roehl d9a81501e5 Use common functions in both Ganglia sinks 2022-02-18 15:05:45 +01:00
Thomas Roehl e2f78fe1c0 Add linker flag -Wl,--unresolved-symbols=ignore-in-object-files to build without library. Remove build tags 2022-02-18 11:41:15 +01:00
Thomas Roehl 4e8ee59211 Update NetstatCollector to derive bandwidths and use an include list 2022-02-18 02:25:23 +01:00
Thomas GruberandGitHub 0152c0dc1e Update CpustatCollector (#36) 
* Update cpustat collector

* Update CpustatCollector to use percentages and add 'num_cpus' metric
 2022-02-17 15:46:06 +01:00
Thomas Roehl 873befb41f Add ganglia build tag to README 2022-02-17 15:06:55 +01:00
Thomas Roehl 7be6d2a338 Use only readable files when searching for libganglia.so 2022-02-16 23:23:07 +01:00
Thomas Roehl e5585eaaa0 Add additional lines for build tags 2022-02-16 19:07:00 +01:00
Thomas GruberandGitHub e28c1fb30b Ganglia sink using libganglia.so directly (#35) 
* Add sink directly using libganglia.so

* Remove unneeded confuse header

* add submodule init to build action

* add submodule init to runonce action

* add installation og ganglia to runonce

* add installation of ganglia to runonce

* add installation of ganglia to runonce

* libconfuse not required

* Remove ganglia submodule

* Remove ganglia.h

* Add Makefile to help creating the libganglia.so link

* Fix cgo header

* Rename new Ganglia sink to 'libgangliaSink'

* Add documentation for libgangliaSink

* Extend make buildsystem with find&symlink helper for libgangliaSink

* Add metric renaming function

* Add build tag 'ganglia' and create corresponding files
 2022-02-16 18:33:46 +01:00
Thomas Roehl 416a94aceb Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-14 18:12:59 +01:00
Thomas Roehl 247fb23de1 Try to operate on multiple metrics if channels if filled 2022-02-14 18:12:50 +01:00
Thomas GruberandGitHub 95a58c29e2 Set version in RPM spec file 2022-02-14 15:21:01 +01:00
Thomas GruberandGitHub 56363a6636 Update cc-metric-collector.spec 2022-02-14 15:20:38 +01:00
Thomas Roehl bd246bdacf Fix group for netstat collector 2022-02-11 18:18:10 +01:00
Thomas Roehl 23d13b2ceb Fix group for netstat collector 2022-02-11 18:09:39 +01:00
Thomas Roehl dc1e4f28a0 Check for Ganglia group explicitly in tags and meta info 2022-02-11 16:10:59 +01:00
Thomas Roehl b15fdf72b9 Exclude metrics and devices in Init() for NvidiaCollector 2022-02-11 14:20:06 +01:00
Thomas Roehl 184d60cc58 Locking in MetricCache 2022-02-10 15:21:26 +01:00
Thomas GruberandGitHub 1ea63332d3 Update README.md 2022-02-08 13:49:48 +01:00
Thomas GruberandGitHub e1cf682989 Add other collectors to README 2022-02-08 13:22:20 +01:00
Thomas GruberandGitHub d1e66201a6 Add non-blocking InfluxDB sink (#29) 
* Add non-blocking InfluxDB sink

* Add configurable batch size
 2022-02-07 16:51:46 +01:00
Thomas GruberandGitHub 5263a974d1 Split NfsCollector in Nfs3Collector and Nfs4Collector (#28) 
* Split NfsCollector in Nfs3Collector and Nfs4Collector

* Add documentation
 2022-02-07 15:43:01 +01:00
Thomas Roehl bb87046501 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 15:42:33 +01:00
Thomas Roehl 0bd638f211 Add script to help create configs for LikwidCollector 2022-02-07 15:42:25 +01:00
Thomas Roehl b7ee125942 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 13:47:06 +01:00
Thomas Roehl 52458ce5a1 Fix for LustreCollector. Check for root user 2022-02-07 13:27:35 +01:00
Thomas Roehl c313055570 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 13:22:34 +01:00
Thomas Roehl 25bb395f02 Fix for NumaDomain getter in ccTopology 2022-02-07 13:22:26 +01:00
Thomas GruberandGitHub fdb58b0be2 Sink specific configuration maps (#25) 
* Use sink-specific configurations to have more flexibility. Adjust sample sink configuration files

* Add documentation

* Add links to individual sink readmes

* Fix link in README

* HTTPS for HttpSink

* If no CPU die id available, use the socket id instead
 2022-02-04 18:12:24 +01:00