Commit Graph

687 Commits

Author SHA1 Message Date
Thomas Roehl
251ae8e879 Update link to cc-specifications repo with line protocol 2022-07-27 17:46:27 +02:00
Thomas Roehl
edd33d5810 Add docs to README 2022-07-27 17:45:13 +02:00
Thomas Roehl
88b3fe1e41 Add some documentation about building 2022-07-27 17:38:51 +02:00
Thomas Roehl
96b4a2aec1 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-07-26 14:59:27 +02:00
Thomas Roehl
4b5c2f4e37 Some introduction to CC Metric Collector and the other components 2022-07-26 14:59:08 +02:00
Holger Obermaier
f818bf4c11 Read durations as string from json config 2022-07-22 17:48:11 +02:00
Holger Obermaier
aedc1be277 Set HTTP timeout for redfish device connections 2022-07-22 12:06:02 +02:00
Holger Obermaier
c75d394e11 Corrected json syntax for interval and duration 2022-07-14 16:07:45 +02:00
Holger Obermaier
bae36473f7 Minimum requirement: golang version >= 1.18 2022-07-14 13:55:48 +02:00
Thomas Gruber
b3c27e0af5
Merge latest development changes (#80)
* Cleanup: Remove unused code

* Use Golang duration parser for 'interval' and 'duration'
 in main config

* Update handling of LIKWID headers. Download only if not already present in the system. Fixes #73

* Units with cc-units (#64)

* Add option to normalize units with cc-unit

* Add unit conversion to router

* Add option to change unit prefix in the router

* Add to MetricRouter README

* Add order of operations in router to README

* Use second add_tags/del_tags only if metric gets renamed

* Skip disks in DiskstatCollector that have size=0

* Check readability of sensor files in TempCollector

* Fix for --once option

* Rename `cpu` type to `hwthread` (#69)

* Rename 'cpu' type to 'hwthread' to avoid naming clashes with MetricStore and CC-Webfrontend

* Collectors in parallel (#74)

* Provide info to CollectorManager whether the collector can be executed in parallel with others

* Split serial and parallel collectors. Read in parallel first

* Update NvidiaCollector with new metrics, MIG and NvLink support (#75)

* CC topology module update (#76)

* Rename CPU to hardware thread, write some comments

* Do renaming in other parts

* Remove CpuList and SocketList function from metricCollector. Available in ccTopology

* Option to use MIG UUID as subtype-id in NvidiaCollector

* Option to use MIG slice name as subtype-id in NvidiaCollector

* MetricRouter: Fix JSON in README

* Fix for Github Action to really use the selected version

* Remove Ganglia installation in runonce Action and add Go 1.18

* Fix daemon options in init script

* Add separate go.mod files to use it with deprecated 1.16

* Minor updates for Makefiles

* fix string comparison

* AMD ROCm SMI collector (#77)

* Add collector for AMD ROCm SMI metrics

* Fix import path

* Fix imports

* Remove Board Number

* store GPU index explicitly

* Remove board number from description

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

* Improved http sink (#78)

* automatic flush in NatsSink

* tweak default options of HttpSink

* shorter cirt. section and retries for HttpSink

* fix error handling

* Remove file added by mistake.

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

Co-authored-by: Thomas Roehl <thomas.roehl@fau.de>

* Fix: When sending metrics failed the batch size could be exceeded

* Improved dropping of metrics failed to send

* Add memstats and topprocs metric

* Updated to latest modules

* Check that at least one sink is running

* Add drop rate, when send buffer is full

* Allow only one timer at a time

* Use mutex to ensure only on flush timer is running

* Fix for NvidiaCollector when devices are not in MiG mode

* Remove Golang version 1.16 an 1.17 from Action. Latest commits require Golang 1.18

* Use Golang 1.18 in Release action to build RPMs

* Change unit of CpufreqCollector to Hz. That's what the sysfs outputs

* Make wget quiet in Release action to reduce log size

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
Co-authored-by: Lou <lou.knauer@gmx.de>
2022-07-13 10:09:49 +02:00
Thomas Roehl
09b740b82e Make wget quiet in Release action to reduce log size 2022-07-12 12:37:10 +02:00
Thomas Roehl
2adf9484a3 Redo fix for NvidiaCollector and MiG. Got lost somehow 2022-07-12 12:31:24 +02:00
Thomas Roehl
27f17b88af Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-07-12 11:58:50 +02:00
Thomas Roehl
b2bc7b95d3 Change unit of CpufreqCollector to Hz. That's what the sysfs outputs 2022-07-12 11:58:37 +02:00
Thomas Gruber
f79b7b5e2b
Merge branch 'main' into develop 2022-07-12 11:36:46 +02:00
Thomas Roehl
b16343e5e2 Use Golang 1.18 in Release action to build RPMs 2022-07-12 11:30:27 +02:00
Thomas Roehl
4fa37a58f2 Remove Golang version 1.16 an 1.17 from Action. Latest commits require Golang 1.18 2022-07-11 16:01:45 +02:00
Thomas Roehl
addbfd40a1 Fix for NvidiaCollector when devices are not in MiG mode 2022-07-11 13:05:15 +02:00
Holger Obermaier
04819d9db2 Use mutex to ensure only on flush timer is running 2022-06-24 09:08:20 +02:00
Holger Obermaier
9ccc5a6ca7 Allow only one timer at a time 2022-06-23 21:53:02 +02:00
Holger Obermaier
b7dcbaebcf Add drop rate, when send buffer is full 2022-06-23 18:27:03 +02:00
Holger Obermaier
a3ac8f2ead Check that at least one sink is running 2022-06-23 15:44:02 +02:00
Holger Obermaier
8e7143a20a Updated to latest modules 2022-06-23 11:49:18 +02:00
Holger Obermaier
3a10f7cfdb Add memstats and topprocs metric 2022-06-23 11:44:06 +02:00
Holger Obermaier
0ca6d1a794 Improved dropping of metrics failed to send 2022-06-21 07:59:24 +02:00
Holger Obermaier
580d21d8bb Fix: When sending metrics failed the batch size could be exceeded 2022-06-20 18:06:27 +02:00
Thomas Roehl
31a38bc17d Update release action 2022-06-09 14:36:25 +02:00
Thomas Roehl
dbdec1eab8 Merge branch 'main' of github.com:ClusterCockpit/cc-metric-collector into main 2022-06-09 12:46:47 +02:00
Thomas Gruber
0d31ec481b
Update Release.yml 2022-06-09 12:42:11 +02:00
Thomas Roehl
e22c3287e9 Merge branch 'main' of github.com:ClusterCockpit/cc-metric-collector into main 2022-06-08 15:26:05 +02:00
Thomas Gruber
8d85bd53f1
Merge latest development changes to main branch (#79)
* Cleanup: Remove unused code

* Use Golang duration parser for 'interval' and 'duration'
 in main config

* Update handling of LIKWID headers. Download only if not already present in the system. Fixes #73

* Units with cc-units (#64)

* Add option to normalize units with cc-unit

* Add unit conversion to router

* Add option to change unit prefix in the router

* Add to MetricRouter README

* Add order of operations in router to README

* Use second add_tags/del_tags only if metric gets renamed

* Skip disks in DiskstatCollector that have size=0

* Check readability of sensor files in TempCollector

* Fix for --once option

* Rename `cpu` type to `hwthread` (#69)

* Rename 'cpu' type to 'hwthread' to avoid naming clashes with MetricStore and CC-Webfrontend

* Collectors in parallel (#74)

* Provide info to CollectorManager whether the collector can be executed in parallel with others

* Split serial and parallel collectors. Read in parallel first

* Update NvidiaCollector with new metrics, MIG and NvLink support (#75)

* CC topology module update (#76)

* Rename CPU to hardware thread, write some comments

* Do renaming in other parts

* Remove CpuList and SocketList function from metricCollector. Available in ccTopology

* Option to use MIG UUID as subtype-id in NvidiaCollector

* Option to use MIG slice name as subtype-id in NvidiaCollector

* MetricRouter: Fix JSON in README

* Fix for Github Action to really use the selected version

* Remove Ganglia installation in runonce Action and add Go 1.18

* Fix daemon options in init script

* Add separate go.mod files to use it with deprecated 1.16

* Minor updates for Makefiles

* fix string comparison

* AMD ROCm SMI collector (#77)

* Add collector for AMD ROCm SMI metrics

* Fix import path

* Fix imports

* Remove Board Number

* store GPU index explicitly

* Remove board number from description

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

* Improved http sink (#78)

* automatic flush in NatsSink

* tweak default options of HttpSink

* shorter cirt. section and retries for HttpSink

* fix error handling

* Remove file added by mistake.

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

Co-authored-by: Thomas Roehl <thomas.roehl@fau.de>

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
Co-authored-by: Lou <lou.knauer@gmx.de>
2022-06-08 15:25:40 +02:00
Lou
b732b2d739
Improved http sink (#78)
* automatic flush in NatsSink

* tweak default options of HttpSink

* shorter cirt. section and retries for HttpSink

* fix error handling

* Remove file added by mistake.

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

Co-authored-by: Thomas Roehl <thomas.roehl@fau.de>
2022-06-08 14:12:35 +02:00
Thomas Roehl
bef807dd44 Fix serial number in rocmCollector 2022-06-05 15:53:39 +02:00
Thomas Roehl
659d0115c0 Use http instead of ftp to download likwid 2022-06-05 15:50:04 +02:00
Thomas Gruber
e13695307f
AMD ROCm SMI collector (#77)
* Add collector for AMD ROCm SMI metrics

* Fix import path

* Fix imports

* Remove Board Number

* store GPU index explicitly

* Remove board number from description
2022-05-25 15:55:43 +02:00
Thomas Roehl
4ed07cad77 fix string comparison 2022-05-25 15:48:55 +02:00
Thomas Roehl
ad5dbd85ea Minor updates for Makefiles 2022-05-25 15:45:21 +02:00
Thomas Roehl
132ebabd45 Add separate go.mod files to use it with deprecated 1.16 2022-05-25 15:35:11 +02:00
Thomas Roehl
f8d91d9cf1 Fix daemon options in init script 2022-05-25 15:16:01 +02:00
Thomas Roehl
cc84a94647 Remove Ganglia installation in runonce Action and add Go 1.18 2022-05-23 17:37:14 +02:00
Thomas Roehl
838b8d824d Fix for Github Action to really use the selected version 2022-05-23 16:50:58 +02:00
Thomas Roehl
7ddc889f06 MetricRouter: Fix JSON in README 2022-05-20 16:06:54 +02:00
Thomas Roehl
500685672b Option to use MIG slice name as subtype-id in NvidiaCollector 2022-05-13 15:26:47 +02:00
Thomas Roehl
d4c89a4206 Option to use MIG UUID as subtype-id in NvidiaCollector 2022-05-13 14:34:32 +02:00
Thomas Gruber
826f364772
CC topology module update (#76)
* Rename CPU to hardware thread, write some comments

* Do renaming in other parts

* Remove CpuList and SocketList function from metricCollector. Available in ccTopology
2022-05-13 14:28:07 +02:00
Thomas Gruber
5df550b208
Update NvidiaCollector with new metrics, MIG and NvLink support (#75) 2022-05-13 14:11:55 +02:00
Thomas Gruber
5c34805918
Collectors in parallel (#74)
* Provide info to CollectorManager whether the collector can be executed in parallel with others

* Split serial and parallel collectors. Read in parallel first
2022-05-13 14:10:39 +02:00
Thomas Gruber
1db5f3b29a
Rename cpu type to hwthread (#69)
* Rename 'cpu' type to 'hwthread' to avoid naming clashes with MetricStore and CC-Webfrontend
2022-05-13 14:09:45 +02:00
Thomas Roehl
0623691bab Fix for --once option 2022-05-13 13:50:19 +02:00
Thomas Roehl
9886f14d14 Check readability of sensor files in TempCollector 2022-05-13 13:32:54 +02:00