ClusterCockpit/cc-metric-collector
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-metric-collector.git synced 2024-11-10 04:27:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
565 Commits 19 Branches 14 Tags 19 MiB
9ae0806aa9
Commit Graph

227 Commits

Author SHA1 Message Date
Holger Obermaier
5ac3af895d Moved documentation to markdown file 2022-02-07 09:22:59 +01:00
Holger Obermaier
9ab7a6424b Moved check which metric to skip to Init() 2022-02-04 19:22:42 +01:00
Holger Obermaier
f719f1915c Add error handling 2022-02-04 16:11:56 +01:00
Holger Obermaier
76b69c59b4 Switched to cclog.ComponentError() for error reporting in Read() 2022-02-04 14:42:53 +01:00
Thomas Roehl
66b9a25a88 Prefix metrics from NetstatCollector with 'net' 2022-02-04 12:39:59 +01:00
Thomas Roehl
db02c89683 Update LustreCollector to use lctl. Sysfs version is commented out 2022-02-03 22:05:16 +01:00
Thomas Gruber
92d4a9c2b9
Split MetricRouter and MetricAggregator (#24) 
* Split MetricRouter and MetricAggregator

* Missing change in MetricCache

* Add README for MetricAggregator
 2022-02-03 16:52:55 +01:00
Holger Obermaier
d5ff5b83ce Add NUMA metric collector 2022-02-03 16:19:45 +01:00
Holger Obermaier
a016483012 Add NUMA metric collector. 2022-02-03 15:02:13 +01:00
Thomas Roehl
2806b1e7cc Remove debugging artifacts 2022-02-02 17:14:29 +01:00
Thomas Roehl
e59852be03 Fix LikwidCollector, merge artifact causes problems 2022-02-02 16:55:15 +01:00
Thomas Roehl
6f399d5f08 Add scope guidelines in LikwidCollector page 2022-02-02 16:46:35 +01:00
Thomas Roehl
5bf538bf97 Update LikwidCollector page 2022-02-02 16:40:20 +01:00
Thomas Roehl
ed62e952ce Use MetricAggregator to calculate metrics in LIKWID collector. 2022-02-02 14:52:07 +01:00
Thomas Roehl
e550226416 Use gval in LikwidCollector 2022-02-01 16:01:31 +01:00
Holger Obermaier
9e99e47d73 Wait for close of done channel, to ensure manager finished. 2022-01-30 12:08:33 +01:00
Holger Obermaier
8df58c051f Lower minimum required golang version to 1.16. 2022-01-29 10:04:31 +01:00
Holger Obermaier
4e408f9490 Add documentation 2022-01-28 15:16:58 +01:00
Holger Obermaier
82f5c1c5d0 Minimum requirement go version 1.17 2022-01-28 09:42:19 +01:00
Holger Obermaier
db5b4e4f65 Add type=node to gpf metric tags 2022-01-28 09:14:25 +01:00
Holger Obermaier
aea3e2c6b1 Place wait group Add() and Done() near to each other 2022-01-27 20:45:22 +01:00
Holger Obermaier
b9236dcc31 Handle shutdown sequentially 2022-01-27 17:43:00 +01:00
Holger Obermaier
e1d0aacd1e Moved as much work as possible to Init() 2022-01-27 11:08:27 +01:00
Holger Obermaier
7077452a5d Split InfiniBand metric collector, one using 
/sys filesystem reads and one using perfquery.
 2022-01-26 20:18:47 +01:00
Thomas Roehl
76884c3380 Prefix Nvidia metrics with 'nv_' 2022-01-26 18:45:23 +01:00
Thomas Roehl
86e9b55bc9 Fix for documentation 2022-01-26 18:41:25 +01:00
Thomas Roehl
78834337b0 Fix for documentation 2022-01-26 18:37:59 +01:00
Thomas Roehl
babd7a9af8 Use non-blocking send at close 2022-01-26 16:52:56 +01:00
Holger Obermaier
09b7538479 Avoid labels in collector manager loop 2022-01-26 15:54:49 +01:00
Holger Obermaier
c193b80083 Add documentation 2022-01-26 12:31:04 +01:00
Thomas Roehl
2925ad9f40 Use ccLogger anywhere 2022-01-25 17:43:10 +01:00
Holger Obermaier
b4fde31626 Add documentation 2022-01-25 17:20:20 +01:00
Thomas Roehl
bafc6322e6 Change to own Logger 2022-01-25 16:40:02 +01:00
Thomas Gruber
200af84c54
Modularize the whole thing (#16) 
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-01-25 15:37:43 +01:00
Holger Obermaier
d903fc6daa Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:16:46 +01:00
Holger Obermaier
222862af32 Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:15:36 +01:00
Holger Obermaier
9f8d3ddbd3 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:34:02 +01:00
Holger Obermaier
df77c3fd60 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:33:20 +01:00
Holger Obermaier
ae6ffd4974 Refactoring 2022-01-25 09:48:22 +01:00
Holger Obermaier
e095e4f202 Refactoring 2022-01-25 09:47:24 +01:00
Holger Obermaier
3d377760b8 Refactoring 2022-01-24 22:04:05 +01:00
Holger Obermaier
be8c92676a Refactoring 2022-01-24 22:03:13 +01:00
Holger Obermaier
9157fdbab2 Fixed topology detection 2022-01-24 20:23:24 +01:00
Holger Obermaier
2026c3acd9 Fixed topology detection 2022-01-24 20:22:08 +01:00
Holger Obermaier
f84f7de05c Add CPUFreqCollectorCpuinfo 
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
 2022-01-24 13:12:25 +01:00
Holger Obermaier
8d314ecb19 Add CPUFreqCollectorCpuinfo 
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
 2022-01-24 13:10:33 +01:00
Holger Obermaier
bcce471b27 Simplified code 2022-01-24 11:33:04 +01:00
Holger Obermaier
daa7c6bf99 Simplified code 2022-01-24 11:31:45 +01:00
Holger Obermaier
5987901005 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:25:13 +01:00
Holger Obermaier
25b9268b24 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:20:53 +01:00
Holger Obermaier
72722eff31 Avoid staticcheck warning: redundant return statement 2022-01-21 14:52:45 +01:00
Holger Obermaier
5dd2af4e8f Avoid staticcheck warning: redundant return statement 2022-01-21 14:35:52 +01:00
Holger Obermaier
611ac0fcb0 Add CPU frequency collector 2022-01-21 13:52:27 +01:00
Holger Obermaier
83b784e6f0 Add CPU frequency collector 2022-01-21 09:59:57 +01:00
Holger Obermaier
3a9ea0042c Correct go syntax in README.md 2022-01-20 16:33:21 +01:00
Holger Obermaier
0feb880c3b Correct go syntax in README.md 2022-01-20 16:32:10 +01:00
Holger Obermaier
458f6dec08 Avoid go vet warning: 
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
 2022-01-20 12:59:43 +01:00
Holger Obermaier
5d263addde Avoid go vet warning: 
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
 2022-01-20 12:38:52 +01:00
Holger Obermaier
f17719113d Updated to latest stable version of likwid 2022-01-19 15:55:48 +01:00
Holger Obermaier
caebca5609 Updated to latest stable version of likwid 2022-01-19 15:55:15 +01:00
Holger Obermaier
82b10b365e Fix to work with golang 1.16 2022-01-19 14:52:37 +01:00
Holger Obermaier
38cba10fb6 Fix to work with golang 1.16 2022-01-19 14:47:59 +01:00
Holger Obermaier
b97c588660 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:58 +01:00
Holger Obermaier
a6cc914b99 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:24 +01:00
Holger Obermaier
bd831060c7 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:26:38 +01:00
Holger Obermaier
11e40c6ee3 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:15:41 +01:00
Thomas Roehl
830b7de9ea Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:26:51 +01:00
Thomas Roehl
43a8ea683d Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:25:51 +01:00
Thomas Roehl
28163f5d2e Format fixes 2021-11-29 15:32:58 +01:00
Thomas Roehl
6d14e66505 Fix missing colon in netstat collector 2021-11-26 19:02:09 +01:00
Thomas Roehl
3a13a41a15 Catch panic in Nvidia Go Bindings 2021-11-26 19:01:47 +01:00
Thomas Roehl
75f5ac8980 Catch panicing collectors (hello Nvidia Go Bindings) late. 2021-11-26 19:01:31 +01:00
Thomas Roehl
cb8221e73c Read config JSON only if there is any 2021-11-26 18:19:45 +01:00
Thomas Roehl
8d643bb5e8 Set default for topprocs collector to Top2 processes if no config is given 2021-11-26 18:19:26 +01:00
Thomas Roehl
bf43f6d975 Updated README.md for collectors. Fix TODO 2021-11-26 15:58:59 +01:00
Thomas Roehl
734d9e5b8f Updated README.md for collectors. Not finished 2021-11-26 15:57:11 +01:00
Thomas Gruber
3997984714
Temp collector from sysfs hwmon (#8) 
* Add collector for hwmon temperature

* Comment out local tag overrides

* Add temperature collector to README

* Update temperature collector with own config parser
 2021-11-25 18:19:09 +01:00
Thomas Gruber
1e7a75598e
Add collector to read data from ipmitool or ipmi-sensors (fallback) (#9) 
* Add collector to read data from ipmitool or ipmi-sensors

* Update IPMI collector to use own config

* Add 'ipmitool sensor' parser
 2021-11-25 18:15:56 +01:00
Thomas Roehl
59ac123662 Add Makefile, update LIKWID Makefile and fix Github Action 2021-11-25 17:51:34 +01:00
Thomas Roehl
156664d78f Make Infiniband collector more configurable 2021-11-25 16:25:20 +01:00
Thomas Roehl
512fa44a74 Remove unneeded stuff 2021-11-25 16:20:52 +01:00
Thomas Roehl
f6a707ac31 Remove unneeded function and const values 2021-11-25 16:01:16 +01:00
Thomas Roehl
9232dd9732 Formatting 2021-11-25 15:11:39 +01:00
Thomas Roehl
51b8c62d4d Collector-specific configuration. LIKWID collector derives metrics itself, Run once CLI option 2021-11-25 14:04:03 +01:00
Thomas Roehl
7044858c2c Change CPI metric to IPC and inverse measurement 2021-11-02 15:50:18 +01:00
Thomas Roehl
dc4b8d13c2 Add collectors for custom commands and /proc/diskstat. Per default add a hostname tag to all measurements 2021-10-08 13:29:57 +02:00
Thomas Roehl
4688997071 Add two helper functions to collector definition 2021-10-08 13:28:02 +02:00
Thomas Roehl
7f1694adc7 Fix format of infiniband collector 2021-10-08 13:27:36 +02:00
Thomas Roehl
cdc253b569 Return proper error for init() of memstat collector 2021-10-08 13:27:15 +02:00
Thomas Gruber
d7ef32de18
Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00
Thomas Roehl
486df91a9d Fix format 2021-10-04 15:47:03 +02:00
Thomas Roehl
558bbaba59 Change storage format 2021-10-04 15:23:43 +02:00
Thomas Roehl
34585d88df Skip InfiniBand collector if perfquery does not exist 2021-06-30 14:14:04 +02:00
Thomas Roehl
ac41f02dfc Update READMEs 2021-05-18 15:42:11 +02:00
Thomas Roehl
9a205717f7 Unify collector names 2021-05-18 15:30:11 +02:00
Thomas Roehl
36f34e13c7 Don't use absolute path for 'ps' 2021-05-18 15:14:37 +02:00
Thomas Roehl
cda6ebf884 Properly check error codes in LikwidCollector 2021-05-18 15:14:11 +02:00
Thomas Roehl
6e7babe084 Adding Nvidia NVML collector 2021-05-14 19:22:59 +02:00
Thomas Roehl
6b6e28d4c7 Return error at Init for skip collectors early 2021-05-14 19:21:16 +02:00
Thomas Roehl
9b59429a93 Add some more memory metrics to memstat collector 2021-05-12 18:09:15 +02:00
Thomas Roehl
fd6f19ea07 Add proc_total and proc_run to loadavg collector 2021-05-12 18:08:43 +02:00
Thomas Roehl
14bc29f766 Add collector for the top5 processes in %cpu 2021-05-12 17:47:12 +02:00
Thomas Roehl
e8b2bbdae1 Fix format for cpustat collector 2021-05-12 17:45:41 +02:00
Thomas Roehl
e35a9f6b86 Add Cpustat collector for /proc/stat 2021-05-12 16:57:18 +02:00
Thomas Roehl
1d3a11016e Update README with infos for InfiniBand and Lustre collector and updates to LIKWID Makefile 2021-05-12 14:41:54 +02:00
Thomas Roehl
fc01781ce5 Update Makefile to use central LIKWID installation 2021-05-12 14:28:10 +02:00
Thomas Roehl
eab45102ea Update collectors' README 2021-03-29 15:34:32 +02:00
Thomas Roehl
aa5c05dd6a Update collectors' README 2021-03-29 15:23:55 +02:00
Thomas Roehl
a501aa20be Update collectors' README 2021-03-29 15:21:56 +02:00
Thomas Roehl
162f0f14d6 Fixing flops_any metric in LikwidCollector 2021-03-27 13:50:25 +01:00
Thomas Roehl
bc5d4b288e Add READMEs and Makefile to build and integrate LIKWID 2021-03-27 13:39:43 +01:00
Thomas Roehl
5c5e8b8f2d Remove prebuild LIKWID and add Makefile that prepares LIKWID 2021-03-27 12:44:11 +01:00
Jan Eitzinger
e7d933e60f Reformat. Cosmetic changes in structure 2021-03-26 10:19:54 +01:00
Thomas Roehl
f32ff9d006 Fixing format with gofmt 2021-03-25 17:47:08 +01:00
Thomas Roehl
e9a74a4c9c Add LustreCollector 2021-03-25 16:52:28 +01:00
Thomas Roehl
642314c102 Add InfinibandCollector 2021-03-25 15:55:06 +01:00
Thomas Roehl
5659607993 Add network device to metric names to NetstatCollector 2021-03-25 15:54:17 +01:00
Thomas Roehl
0110547b68 Add load_five and load_fifteen to LoadavgCollector 2021-03-25 15:53:43 +01:00
Thomas Roehl
73fd59537e Add simpler collectors for load, mem_used and network 2021-03-25 14:47:35 +01:00
Thomas Roehl
a6ac0c5373 Add likwid collector 2021-03-25 14:47:10 +01:00
Thomas Roehl
1c7aa6477b Add collectors base type 2021-03-25 14:46:21 +01:00
Thomas Roehl
da7faad595 Delete old files 2021-03-25 14:45:15 +01:00
Jan Eitzinger
4519632ef2 Reformat, remove x bit on memavg Collector 2021-03-19 09:34:12 +01:00
Thomas Roehl
9597798ebc memavg collector in Go 2021-03-18 18:01:59 +01:00
Thomas Roehl
627d268d0b Line protocol wants field keys 2021-03-18 17:30:08 +01:00
Jan Eitzinger
e0b9c9e8b2 Add example for collector 2021-03-18 09:11:25 +01:00
Thomas Roehl
24c5a80585 Script version, no time loop 2021-03-17 18:00:09 +01:00