ClusterCockpit/cc-metric-collector
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-metric-collector.git synced 2024-12-26 07:29:04 +01:00
Code Issues Packages Projects Releases Wiki Activity
418 Commits 18 Branches 15 Tags 19 MiB
73f22c1041
Commit Graph

185 Commits

Author SHA1 Message Date
Thomas Gruber
5263a974d1
Split NfsCollector in Nfs3Collector and Nfs4Collector (#28) 
* Split NfsCollector in Nfs3Collector and Nfs4Collector

* Add documentation
 2022-02-07 15:43:01 +01:00
Thomas Roehl
b7ee125942 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 13:47:06 +01:00
Holger Obermaier
ead7117cad Add skip_filesystem configuration 2022-02-07 13:30:42 +01:00
Thomas Roehl
52458ce5a1 Fix for LustreCollector. Check for root user 2022-02-07 13:27:35 +01:00
Holger Obermaier
a534f16685 Add documentation for GPFS metric 2022-02-07 11:37:34 +01:00
Holger Obermaier
25c2ae4910 Avoid int -> int64 conversions 2022-02-07 11:12:03 +01:00
Holger Obermaier
3c10c6b340 Add error handling to Read() 2022-02-07 10:02:38 +01:00
Holger Obermaier
79b25ddbee Add markdown documentation for metric collector ibstat_perfquery 2022-02-07 09:46:19 +01:00
Holger Obermaier
5ac3af895d Moved documentation to markdown file 2022-02-07 09:22:59 +01:00
Holger Obermaier
9ab7a6424b Moved check which metric to skip to Init() 2022-02-04 19:22:42 +01:00
Holger Obermaier
f719f1915c Add error handling 2022-02-04 16:11:56 +01:00
Holger Obermaier
76b69c59b4 Switched to cclog.ComponentError() for error reporting in Read() 2022-02-04 14:42:53 +01:00
Thomas Roehl
66b9a25a88 Prefix metrics from NetstatCollector with 'net' 2022-02-04 12:39:59 +01:00
Thomas Roehl
db02c89683 Update LustreCollector to use lctl. Sysfs version is commented out 2022-02-03 22:05:16 +01:00
Thomas Gruber
92d4a9c2b9
Split MetricRouter and MetricAggregator (#24) 
* Split MetricRouter and MetricAggregator

* Missing change in MetricCache

* Add README for MetricAggregator
 2022-02-03 16:52:55 +01:00
Holger Obermaier
d5ff5b83ce Add NUMA metric collector 2022-02-03 16:19:45 +01:00
Holger Obermaier
a016483012 Add NUMA metric collector. 2022-02-03 15:02:13 +01:00
Thomas Roehl
2806b1e7cc Remove debugging artifacts 2022-02-02 17:14:29 +01:00
Thomas Roehl
e59852be03 Fix LikwidCollector, merge artifact causes problems 2022-02-02 16:55:15 +01:00
Thomas Roehl
6f399d5f08 Add scope guidelines in LikwidCollector page 2022-02-02 16:46:35 +01:00
Thomas Roehl
5bf538bf97 Update LikwidCollector page 2022-02-02 16:40:20 +01:00
Thomas Roehl
ed62e952ce Use MetricAggregator to calculate metrics in LIKWID collector. 2022-02-02 14:52:07 +01:00
Thomas Roehl
e550226416 Use gval in LikwidCollector 2022-02-01 16:01:31 +01:00
Holger Obermaier
9e99e47d73 Wait for close of done channel, to ensure manager finished. 2022-01-30 12:08:33 +01:00
Holger Obermaier
8df58c051f Lower minimum required golang version to 1.16. 2022-01-29 10:04:31 +01:00
Holger Obermaier
4e408f9490 Add documentation 2022-01-28 15:16:58 +01:00
Holger Obermaier
82f5c1c5d0 Minimum requirement go version 1.17 2022-01-28 09:42:19 +01:00
Holger Obermaier
db5b4e4f65 Add type=node to gpf metric tags 2022-01-28 09:14:25 +01:00
Holger Obermaier
aea3e2c6b1 Place wait group Add() and Done() near to each other 2022-01-27 20:45:22 +01:00
Holger Obermaier
b9236dcc31 Handle shutdown sequentially 2022-01-27 17:43:00 +01:00
Holger Obermaier
e1d0aacd1e Moved as much work as possible to Init() 2022-01-27 11:08:27 +01:00
Holger Obermaier
7077452a5d Split InfiniBand metric collector, one using 
/sys filesystem reads and one using perfquery.
 2022-01-26 20:18:47 +01:00
Thomas Roehl
76884c3380 Prefix Nvidia metrics with 'nv_' 2022-01-26 18:45:23 +01:00
Thomas Roehl
86e9b55bc9 Fix for documentation 2022-01-26 18:41:25 +01:00
Thomas Roehl
78834337b0 Fix for documentation 2022-01-26 18:37:59 +01:00
Thomas Roehl
babd7a9af8 Use non-blocking send at close 2022-01-26 16:52:56 +01:00
Holger Obermaier
09b7538479 Avoid labels in collector manager loop 2022-01-26 15:54:49 +01:00
Holger Obermaier
c193b80083 Add documentation 2022-01-26 12:31:04 +01:00
Thomas Roehl
2925ad9f40 Use ccLogger anywhere 2022-01-25 17:43:10 +01:00
Holger Obermaier
b4fde31626 Add documentation 2022-01-25 17:20:20 +01:00
Thomas Roehl
bafc6322e6 Change to own Logger 2022-01-25 16:40:02 +01:00
Thomas Gruber
200af84c54
Modularize the whole thing (#16) 
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-01-25 15:37:43 +01:00
Holger Obermaier
d903fc6daa Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:16:46 +01:00
Holger Obermaier
222862af32 Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:15:36 +01:00
Holger Obermaier
9f8d3ddbd3 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:34:02 +01:00
Holger Obermaier
df77c3fd60 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:33:20 +01:00
Holger Obermaier
ae6ffd4974 Refactoring 2022-01-25 09:48:22 +01:00
Holger Obermaier
e095e4f202 Refactoring 2022-01-25 09:47:24 +01:00
Holger Obermaier
3d377760b8 Refactoring 2022-01-24 22:04:05 +01:00
Holger Obermaier
be8c92676a Refactoring 2022-01-24 22:03:13 +01:00
Holger Obermaier
9157fdbab2 Fixed topology detection 2022-01-24 20:23:24 +01:00
Holger Obermaier
2026c3acd9 Fixed topology detection 2022-01-24 20:22:08 +01:00
Holger Obermaier
f84f7de05c Add CPUFreqCollectorCpuinfo 
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
 2022-01-24 13:12:25 +01:00
Holger Obermaier
8d314ecb19 Add CPUFreqCollectorCpuinfo 
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
 2022-01-24 13:10:33 +01:00
Holger Obermaier
bcce471b27 Simplified code 2022-01-24 11:33:04 +01:00
Holger Obermaier
daa7c6bf99 Simplified code 2022-01-24 11:31:45 +01:00
Holger Obermaier
5987901005 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:25:13 +01:00
Holger Obermaier
25b9268b24 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:20:53 +01:00
Holger Obermaier
72722eff31 Avoid staticcheck warning: redundant return statement 2022-01-21 14:52:45 +01:00
Holger Obermaier
5dd2af4e8f Avoid staticcheck warning: redundant return statement 2022-01-21 14:35:52 +01:00
Holger Obermaier
611ac0fcb0 Add CPU frequency collector 2022-01-21 13:52:27 +01:00
Holger Obermaier
83b784e6f0 Add CPU frequency collector 2022-01-21 09:59:57 +01:00
Holger Obermaier
3a9ea0042c Correct go syntax in README.md 2022-01-20 16:33:21 +01:00
Holger Obermaier
0feb880c3b Correct go syntax in README.md 2022-01-20 16:32:10 +01:00
Holger Obermaier
458f6dec08 Avoid go vet warning: 
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
 2022-01-20 12:59:43 +01:00
Holger Obermaier
5d263addde Avoid go vet warning: 
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
 2022-01-20 12:38:52 +01:00
Holger Obermaier
f17719113d Updated to latest stable version of likwid 2022-01-19 15:55:48 +01:00
Holger Obermaier
caebca5609 Updated to latest stable version of likwid 2022-01-19 15:55:15 +01:00
Holger Obermaier
82b10b365e Fix to work with golang 1.16 2022-01-19 14:52:37 +01:00
Holger Obermaier
38cba10fb6 Fix to work with golang 1.16 2022-01-19 14:47:59 +01:00
Holger Obermaier
b97c588660 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:58 +01:00
Holger Obermaier
a6cc914b99 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:24 +01:00
Holger Obermaier
bd831060c7 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:26:38 +01:00
Holger Obermaier
11e40c6ee3 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:15:41 +01:00
Thomas Roehl
830b7de9ea Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:26:51 +01:00
Thomas Roehl
43a8ea683d Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:25:51 +01:00
Thomas Roehl
28163f5d2e Format fixes 2021-11-29 15:32:58 +01:00
Thomas Roehl
6d14e66505 Fix missing colon in netstat collector 2021-11-26 19:02:09 +01:00
Thomas Roehl
3a13a41a15 Catch panic in Nvidia Go Bindings 2021-11-26 19:01:47 +01:00
Thomas Roehl
75f5ac8980 Catch panicing collectors (hello Nvidia Go Bindings) late. 2021-11-26 19:01:31 +01:00
Thomas Roehl
cb8221e73c Read config JSON only if there is any 2021-11-26 18:19:45 +01:00
Thomas Roehl
8d643bb5e8 Set default for topprocs collector to Top2 processes if no config is given 2021-11-26 18:19:26 +01:00
Thomas Roehl
bf43f6d975 Updated README.md for collectors. Fix TODO 2021-11-26 15:58:59 +01:00
Thomas Roehl
734d9e5b8f Updated README.md for collectors. Not finished 2021-11-26 15:57:11 +01:00
Thomas Gruber
3997984714
Temp collector from sysfs hwmon (#8) 
* Add collector for hwmon temperature

* Comment out local tag overrides

* Add temperature collector to README

* Update temperature collector with own config parser
 2021-11-25 18:19:09 +01:00
Thomas Gruber
1e7a75598e
Add collector to read data from ipmitool or ipmi-sensors (fallback) (#9) 
* Add collector to read data from ipmitool or ipmi-sensors

* Update IPMI collector to use own config

* Add 'ipmitool sensor' parser
 2021-11-25 18:15:56 +01:00
Thomas Roehl
59ac123662 Add Makefile, update LIKWID Makefile and fix Github Action 2021-11-25 17:51:34 +01:00
Thomas Roehl
156664d78f Make Infiniband collector more configurable 2021-11-25 16:25:20 +01:00
Thomas Roehl
512fa44a74 Remove unneeded stuff 2021-11-25 16:20:52 +01:00
Thomas Roehl
f6a707ac31 Remove unneeded function and const values 2021-11-25 16:01:16 +01:00
Thomas Roehl
9232dd9732 Formatting 2021-11-25 15:11:39 +01:00
Thomas Roehl
51b8c62d4d Collector-specific configuration. LIKWID collector derives metrics itself, Run once CLI option 2021-11-25 14:04:03 +01:00
Thomas Roehl
7044858c2c Change CPI metric to IPC and inverse measurement 2021-11-02 15:50:18 +01:00
Thomas Roehl
dc4b8d13c2 Add collectors for custom commands and /proc/diskstat. Per default add a hostname tag to all measurements 2021-10-08 13:29:57 +02:00
Thomas Roehl
4688997071 Add two helper functions to collector definition 2021-10-08 13:28:02 +02:00
Thomas Roehl
7f1694adc7 Fix format of infiniband collector 2021-10-08 13:27:36 +02:00
Thomas Roehl
cdc253b569 Return proper error for init() of memstat collector 2021-10-08 13:27:15 +02:00
Thomas Gruber
d7ef32de18
Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00
Thomas Roehl
486df91a9d Fix format 2021-10-04 15:47:03 +02:00
Thomas Roehl
558bbaba59 Change storage format 2021-10-04 15:23:43 +02:00