Commit Graph

256 Commits

Author SHA1 Message Date
Holger Obermaier
b9236dcc31 Handle shutdown sequentially 2022-01-27 17:43:00 +01:00
Holger Obermaier
e1d0aacd1e Moved as much work as possible to Init() 2022-01-27 11:08:27 +01:00
Holger Obermaier
7077452a5d Split InfiniBand metric collector, one using
/sys filesystem reads and one using perfquery.
2022-01-26 20:18:47 +01:00
Thomas Roehl
76884c3380 Prefix Nvidia metrics with 'nv_' 2022-01-26 18:45:23 +01:00
Thomas Roehl
86e9b55bc9 Fix for documentation 2022-01-26 18:41:25 +01:00
Thomas Roehl
78834337b0 Fix for documentation 2022-01-26 18:37:59 +01:00
Thomas Roehl
babd7a9af8 Use non-blocking send at close 2022-01-26 16:52:56 +01:00
Holger Obermaier
09b7538479 Avoid labels in collector manager loop 2022-01-26 15:54:49 +01:00
Holger Obermaier
c193b80083 Add documentation 2022-01-26 12:31:04 +01:00
Thomas Roehl
2925ad9f40 Use ccLogger anywhere 2022-01-25 17:43:10 +01:00
Holger Obermaier
b4fde31626 Add documentation 2022-01-25 17:20:20 +01:00
Thomas Roehl
bafc6322e6 Change to own Logger 2022-01-25 16:40:02 +01:00
Thomas Gruber
200af84c54
Modularize the whole thing (#16)
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-01-25 15:37:43 +01:00
Holger Obermaier
d903fc6daa Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:16:46 +01:00
Holger Obermaier
222862af32 Avoid vet warning struct field commands has json tag but is not exported 2022-01-25 11:15:36 +01:00
Holger Obermaier
9f8d3ddbd3 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:34:02 +01:00
Holger Obermaier
df77c3fd60 Avoid vet warning: Println arg list ends with redundant newline 2022-01-25 10:33:20 +01:00
Holger Obermaier
ae6ffd4974 Refactoring 2022-01-25 09:48:22 +01:00
Holger Obermaier
e095e4f202 Refactoring 2022-01-25 09:47:24 +01:00
Holger Obermaier
3d377760b8 Refactoring 2022-01-24 22:04:05 +01:00
Holger Obermaier
be8c92676a Refactoring 2022-01-24 22:03:13 +01:00
Holger Obermaier
9157fdbab2 Fixed topology detection 2022-01-24 20:23:24 +01:00
Holger Obermaier
2026c3acd9 Fixed topology detection 2022-01-24 20:22:08 +01:00
Holger Obermaier
f84f7de05c Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
2022-01-24 13:12:25 +01:00
Holger Obermaier
8d314ecb19 Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
2022-01-24 13:10:33 +01:00
Holger Obermaier
bcce471b27 Simplified code 2022-01-24 11:33:04 +01:00
Holger Obermaier
daa7c6bf99 Simplified code 2022-01-24 11:31:45 +01:00
Holger Obermaier
5987901005 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:25:13 +01:00
Holger Obermaier
25b9268b24 Avoid staticcheck warning: unnecessary assignment to the blank identifier 2022-01-21 15:20:53 +01:00
Holger Obermaier
72722eff31 Avoid staticcheck warning: redundant return statement 2022-01-21 14:52:45 +01:00
Holger Obermaier
5dd2af4e8f Avoid staticcheck warning: redundant return statement 2022-01-21 14:35:52 +01:00
Holger Obermaier
611ac0fcb0 Add CPU frequency collector 2022-01-21 13:52:27 +01:00
Holger Obermaier
83b784e6f0 Add CPU frequency collector 2022-01-21 09:59:57 +01:00
Holger Obermaier
3a9ea0042c Correct go syntax in README.md 2022-01-20 16:33:21 +01:00
Holger Obermaier
0feb880c3b Correct go syntax in README.md 2022-01-20 16:32:10 +01:00
Holger Obermaier
458f6dec08 Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
2022-01-20 12:59:43 +01:00
Holger Obermaier
5d263addde Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
2022-01-20 12:38:52 +01:00
Holger Obermaier
f17719113d Updated to latest stable version of likwid 2022-01-19 15:55:48 +01:00
Holger Obermaier
caebca5609 Updated to latest stable version of likwid 2022-01-19 15:55:15 +01:00
Holger Obermaier
82b10b365e Fix to work with golang 1.16 2022-01-19 14:52:37 +01:00
Holger Obermaier
38cba10fb6 Fix to work with golang 1.16 2022-01-19 14:47:59 +01:00
Holger Obermaier
b97c588660 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:58 +01:00
Holger Obermaier
a6cc914b99 Add GPFS / IBM Spectrum Scale collector 2022-01-19 14:25:24 +01:00
Holger Obermaier
bd831060c7 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:26:38 +01:00
Holger Obermaier
11e40c6ee3 Add IB metrics ib_recv_pkts and ib_xmit_pkts 2022-01-19 10:15:41 +01:00
Thomas Roehl
830b7de9ea Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:26:51 +01:00
Thomas Roehl
43a8ea683d Cast collector measurement duration to seconds. Thanks to KIT 2022-01-06 15:25:51 +01:00
Thomas Roehl
28163f5d2e Format fixes 2021-11-29 15:32:58 +01:00
Thomas Roehl
6d14e66505 Fix missing colon in netstat collector 2021-11-26 19:02:09 +01:00
Thomas Roehl
3a13a41a15 Catch panic in Nvidia Go Bindings 2021-11-26 19:01:47 +01:00
Thomas Roehl
75f5ac8980 Catch panicing collectors (hello Nvidia Go Bindings) late. 2021-11-26 19:01:31 +01:00
Thomas Roehl
cb8221e73c Read config JSON only if there is any 2021-11-26 18:19:45 +01:00
Thomas Roehl
8d643bb5e8 Set default for topprocs collector to Top2 processes if no config is given 2021-11-26 18:19:26 +01:00
Thomas Roehl
bf43f6d975 Updated README.md for collectors. Fix TODO 2021-11-26 15:58:59 +01:00
Thomas Roehl
734d9e5b8f Updated README.md for collectors. Not finished 2021-11-26 15:57:11 +01:00
Thomas Gruber
3997984714
Temp collector from sysfs hwmon (#8)
* Add collector for hwmon temperature

* Comment out local tag overrides

* Add temperature collector to README

* Update temperature collector with own config parser
2021-11-25 18:19:09 +01:00
Thomas Gruber
1e7a75598e
Add collector to read data from ipmitool or ipmi-sensors (fallback) (#9)
* Add collector to read data from ipmitool or ipmi-sensors

* Update IPMI collector to use own config

* Add 'ipmitool sensor' parser
2021-11-25 18:15:56 +01:00
Thomas Roehl
59ac123662 Add Makefile, update LIKWID Makefile and fix Github Action 2021-11-25 17:51:34 +01:00
Thomas Roehl
156664d78f Make Infiniband collector more configurable 2021-11-25 16:25:20 +01:00
Thomas Roehl
512fa44a74 Remove unneeded stuff 2021-11-25 16:20:52 +01:00
Thomas Roehl
f6a707ac31 Remove unneeded function and const values 2021-11-25 16:01:16 +01:00
Thomas Roehl
9232dd9732 Formatting 2021-11-25 15:11:39 +01:00
Thomas Roehl
51b8c62d4d Collector-specific configuration. LIKWID collector derives metrics itself, Run once CLI option 2021-11-25 14:04:03 +01:00
Thomas Roehl
7044858c2c Change CPI metric to IPC and inverse measurement 2021-11-02 15:50:18 +01:00
Thomas Roehl
dc4b8d13c2 Add collectors for custom commands and /proc/diskstat. Per default add a hostname tag to all measurements 2021-10-08 13:29:57 +02:00
Thomas Roehl
4688997071 Add two helper functions to collector definition 2021-10-08 13:28:02 +02:00
Thomas Roehl
7f1694adc7 Fix format of infiniband collector 2021-10-08 13:27:36 +02:00
Thomas Roehl
cdc253b569 Return proper error for init() of memstat collector 2021-10-08 13:27:15 +02:00
Thomas Gruber
d7ef32de18
Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00
Thomas Roehl
486df91a9d Fix format 2021-10-04 15:47:03 +02:00
Thomas Roehl
558bbaba59 Change storage format 2021-10-04 15:23:43 +02:00
Thomas Roehl
34585d88df Skip InfiniBand collector if perfquery does not exist 2021-06-30 14:14:04 +02:00
Thomas Roehl
ac41f02dfc Update READMEs 2021-05-18 15:42:11 +02:00
Thomas Roehl
9a205717f7 Unify collector names 2021-05-18 15:30:11 +02:00
Thomas Roehl
36f34e13c7 Don't use absolute path for 'ps' 2021-05-18 15:14:37 +02:00
Thomas Roehl
cda6ebf884 Properly check error codes in LikwidCollector 2021-05-18 15:14:11 +02:00
Thomas Roehl
6e7babe084 Adding Nvidia NVML collector 2021-05-14 19:22:59 +02:00
Thomas Roehl
6b6e28d4c7 Return error at Init for skip collectors early 2021-05-14 19:21:16 +02:00
Thomas Roehl
9b59429a93 Add some more memory metrics to memstat collector 2021-05-12 18:09:15 +02:00
Thomas Roehl
fd6f19ea07 Add proc_total and proc_run to loadavg collector 2021-05-12 18:08:43 +02:00
Thomas Roehl
14bc29f766 Add collector for the top5 processes in %cpu 2021-05-12 17:47:12 +02:00
Thomas Roehl
e8b2bbdae1 Fix format for cpustat collector 2021-05-12 17:45:41 +02:00
Thomas Roehl
e35a9f6b86 Add Cpustat collector for /proc/stat 2021-05-12 16:57:18 +02:00
Thomas Roehl
1d3a11016e Update README with infos for InfiniBand and Lustre collector and updates to LIKWID Makefile 2021-05-12 14:41:54 +02:00
Thomas Roehl
fc01781ce5 Update Makefile to use central LIKWID installation 2021-05-12 14:28:10 +02:00
Thomas Roehl
eab45102ea Update collectors' README 2021-03-29 15:34:32 +02:00
Thomas Roehl
aa5c05dd6a Update collectors' README 2021-03-29 15:23:55 +02:00
Thomas Roehl
a501aa20be Update collectors' README 2021-03-29 15:21:56 +02:00
Thomas Roehl
162f0f14d6 Fixing flops_any metric in LikwidCollector 2021-03-27 13:50:25 +01:00
Thomas Roehl
bc5d4b288e Add READMEs and Makefile to build and integrate LIKWID 2021-03-27 13:39:43 +01:00
Thomas Roehl
5c5e8b8f2d Remove prebuild LIKWID and add Makefile that prepares LIKWID 2021-03-27 12:44:11 +01:00
Jan Eitzinger
e7d933e60f Reformat. Cosmetic changes in structure 2021-03-26 10:19:54 +01:00
Thomas Roehl
f32ff9d006 Fixing format with gofmt 2021-03-25 17:47:08 +01:00
Thomas Roehl
e9a74a4c9c Add LustreCollector 2021-03-25 16:52:28 +01:00
Thomas Roehl
642314c102 Add InfinibandCollector 2021-03-25 15:55:06 +01:00
Thomas Roehl
5659607993 Add network device to metric names to NetstatCollector 2021-03-25 15:54:17 +01:00
Thomas Roehl
0110547b68 Add load_five and load_fifteen to LoadavgCollector 2021-03-25 15:53:43 +01:00
Thomas Roehl
73fd59537e Add simpler collectors for load, mem_used and network 2021-03-25 14:47:35 +01:00
Thomas Roehl
a6ac0c5373 Add likwid collector 2021-03-25 14:47:10 +01:00
Thomas Roehl
1c7aa6477b Add collectors base type 2021-03-25 14:46:21 +01:00
Thomas Roehl
da7faad595 Delete old files 2021-03-25 14:45:15 +01:00
Jan Eitzinger
4519632ef2 Reformat, remove x bit on memavg Collector 2021-03-19 09:34:12 +01:00
Thomas Roehl
9597798ebc memavg collector in Go 2021-03-18 18:01:59 +01:00
Thomas Roehl
627d268d0b Line protocol wants field keys 2021-03-18 17:30:08 +01:00
Jan Eitzinger
e0b9c9e8b2 Add example for collector 2021-03-18 09:11:25 +01:00
Thomas Roehl
24c5a80585 Script version, no time loop 2021-03-17 18:00:09 +01:00