ClusterCockpit/cc-metric-collector
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-metric-collector.git synced 2026-07-26 18:57:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
275 Commits 37 Branches 23 Tags
acf5db543e1a56f7aa1046cc1316674f17db743a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Thomas GruberandGitHub 1ea63332d3 Update README.md 2022-02-08 13:49:48 +01:00
Thomas GruberandGitHub e1cf682989 Add other collectors to README 2022-02-08 13:22:20 +01:00
Thomas GruberandGitHub d1e66201a6 Add non-blocking InfluxDB sink (#29) 
* Add non-blocking InfluxDB sink

* Add configurable batch size
 2022-02-07 16:51:46 +01:00
Thomas GruberandGitHub 5263a974d1 Split NfsCollector in Nfs3Collector and Nfs4Collector (#28) 
* Split NfsCollector in Nfs3Collector and Nfs4Collector

* Add documentation
 2022-02-07 15:43:01 +01:00
Thomas Roehl bb87046501 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 15:42:33 +01:00
Thomas Roehl 0bd638f211 Add script to help create configs for LikwidCollector 2022-02-07 15:42:25 +01:00
Thomas Roehl b7ee125942 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 13:47:06 +01:00
Thomas Roehl 52458ce5a1 Fix for LustreCollector. Check for root user 2022-02-07 13:27:35 +01:00
Thomas Roehl c313055570 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-07 13:22:34 +01:00
Thomas Roehl 25bb395f02 Fix for NumaDomain getter in ccTopology 2022-02-07 13:22:26 +01:00
Thomas GruberandGitHub fdb58b0be2 Sink specific configuration maps (#25) 
* Use sink-specific configurations to have more flexibility. Adjust sample sink configuration files

* Add documentation

* Add links to individual sink readmes

* Fix link in README

* HTTPS for HttpSink

* If no CPU die id available, use the socket id instead
 2022-02-04 18:12:24 +01:00
Thomas Roehl 66b9a25a88 Prefix metrics from NetstatCollector with 'net' 2022-02-04 12:39:59 +01:00
Thomas Roehl 02cd21abe2 HTTPS for HttpSink 2022-02-04 12:39:25 +01:00
Thomas Roehl db02c89683 Update LustreCollector to use lctl. Sysfs version is commented out 2022-02-03 22:05:16 +01:00
Thomas GruberandGitHub 92d4a9c2b9 Split MetricRouter and MetricAggregator (#24) 
* Split MetricRouter and MetricAggregator

* Missing change in MetricCache

* Add README for MetricAggregator
 2022-02-03 16:52:55 +01:00
Thomas Roehl 2806b1e7cc Remove debugging artifacts 2022-02-02 17:14:29 +01:00
Thomas Roehl e59852be03 Fix LikwidCollector, merge artifact causes problems 2022-02-02 16:55:15 +01:00
Thomas Roehl 6f399d5f08 Add scope guidelines in LikwidCollector page 2022-02-02 16:46:35 +01:00
Thomas Roehl 5bf538bf97 Update LikwidCollector page 2022-02-02 16:40:20 +01:00
Thomas Roehl 1222f7a32f Configuration option to disable MetricCache completely 2022-02-02 15:30:14 +01:00
Thomas Roehl 2c13cecf13 Fix link in MetricRouter README 2022-02-02 14:52:19 +01:00
Thomas Roehl ed62e952ce Use MetricAggregator to calculate metrics in LIKWID collector. 2022-02-02 14:52:07 +01:00
Thomas Roehl 4633c8f58d Merge branch 'likwid_collector_gval_and_fix' into develop 2022-02-01 18:35:03 +01:00
Thomas Roehl af8654d325 Update MetricRouter README 2022-02-01 18:28:20 +01:00
Thomas Roehl a4bd141786 Use the MetricAggregator for all calculations in the MetricRouter 2022-02-01 18:27:59 +01:00
Thomas Roehl 64a12b80bb Add and export SetName() function for CCMetric 2022-02-01 18:27:16 +01:00
Thomas Roehl 8319d3de43 Add some more helper functions to ccTopology 2022-02-01 18:26:54 +01:00
Thomas Roehl e550226416 Use gval in LikwidCollector 2022-02-01 16:01:31 +01:00
Thomas GruberGitHubHolger Obermaier
 		6ff6cb7219 Change CCMetric's internal data structure (#22) 
* package ccmetric rewrite

* Create deep copy in New() to avoid access conflicts

* Renamed TagMap() -> Tags(), MetaMap() -> Meta

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-02-01 14:54:34 +01:00
Thomas GruberandGitHub 862630a218 Extend workflow to test Go 1.16 and 1.17 2022-01-31 14:42:19 +01:00
Thomas GruberandGitHub 1f55aa247f Run rpmbuild workflow only for new tags 2022-01-31 13:29:14 +01:00
Thomas Roehl 3e329c3324 Move defer after checking error 2022-01-30 22:05:27 +01:00
Thomas Roehl 011218ab80 Adjust ganglia sink to CCMetric 2022-01-30 15:25:57 +01:00
Thomas GruberandGitHub d915bcc02d Add sink to add metrics to Ganglia through gmetric (#15) 2022-01-30 15:21:24 +01:00
Thomas Roehl df31df149b Fix for missing math.MaxInt in go 1.16 2022-01-30 15:16:46 +01:00
Thomas Roehl 70ebd2f36d Add gval to go files 2022-01-30 15:13:12 +01:00
Thomas Roehl 18c5d0eb34 Add example interval aggregation to MetricRouter config for CI 2022-01-30 15:04:31 +01:00
Thomas GruberandGitHub cf810b1c0c Add Cache and Aggregator to MetricRouter (#21) 
* Add Cache and Aggregator to MetricRouter

* Close done channel in MetricCache
 2022-01-30 15:03:21 +01:00
Thomas GruberandGitHub 11844d9d5d Add common topology module for MetricCollectors and MetricRouter (#20) 2022-01-30 14:59:26 +01:00
Thomas GruberandGitHub 6abbc5f77e Fix Github Actions (#18) 
* Fix config for Github Actions

* Fix paths

* Add CentOS Latest and AlmaLinux 8.5 to RPM action

* Fix ID

* Reduce min Go version to 1.16 and use time.Unix in gpfsMetric
 2022-01-30 14:54:36 +01:00
Thomas Roehl e4a2927b96 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-01-30 14:31:19 +01:00
Thomas Roehl d3f5611541 Add functions to get the fields of a CCMetric and export some more CCMetric functions 2022-01-30 14:30:06 +01:00
Thomas Roehl 4541e50bea Minor fixes in ccLogger 2022-01-30 14:29:25 +01:00
Thomas Roehl 76884c3380 Prefix Nvidia metrics with 'nv_' 2022-01-26 18:45:23 +01:00
Thomas Roehl 86e9b55bc9 Fix for documentation 2022-01-26 18:41:25 +01:00
Thomas Roehl 78834337b0 Fix for documentation 2022-01-26 18:37:59 +01:00
Thomas GruberGitHubHolger Obermaier
 		200af84c54 Modularize the whole thing (#16) 
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
 2022-01-25 15:37:43 +01:00
Thomas GruberandGitHub 15cf16b46f Add Github Action to build RPM (#14) 
* Update SPEC file

* Add clean target to Makefile

* add rpmbuild action

* Add missing build dependency

* Fix package name of golang for CentOS8

* Change outputs

* Fix file list and build requires

* Fix for file list

* Also upload SRPM
 2021-11-29 16:04:50 +01:00
Thomas Roehl 6d14e66505 Fix missing colon in netstat collector 2021-11-26 19:02:09 +01:00
Thomas Roehl 3a13a41a15 Catch panic in Nvidia Go Bindings 2021-11-26 19:01:47 +01:00
Thomas Roehl 75f5ac8980 Catch panicing collectors (hello Nvidia Go Bindings) late. 2021-11-26 19:01:31 +01:00
Thomas Roehl 7a7ce0c1b3 Catch panicing collectors (hello Nvidia Go Bindings) late. 2021-11-26 19:01:12 +01:00
Thomas Roehl cb8221e73c Read config JSON only if there is any 2021-11-26 18:19:45 +01:00
Thomas Roehl 8d643bb5e8 Set default for topprocs collector to Top2 processes if no config is given 2021-11-26 18:19:26 +01:00
Thomas Roehl f5b7da5046 Use configuration with all supported collectors for CI. Fixes #13 2021-11-26 18:15:04 +01:00
Thomas Roehl bf43f6d975 Updated README.md for collectors. Fix TODO 2021-11-26 15:58:59 +01:00
Thomas Roehl 734d9e5b8f Updated README.md for collectors. Not finished 2021-11-26 15:57:11 +01:00
Thomas Roehl 325d97df96 Updated contributing section for sinks 2021-11-26 15:56:52 +01:00
Thomas Roehl 09e15c9949 Update receiver README.md 2021-11-26 14:19:07 +01:00
Thomas Roehl 32d5a5b98f Update sink README.md 2021-11-26 14:18:59 +01:00
Thomas Roehl 24bf9590ee Update sink README.md 2021-11-26 14:10:22 +01:00
Thomas Roehl fcf8dfb680 Update global README.md 2021-11-26 14:10:12 +01:00
Thomas GruberandGitHub 3997984714 Temp collector from sysfs hwmon (#8) 
* Add collector for hwmon temperature

* Comment out local tag overrides

* Add temperature collector to README

* Update temperature collector with own config parser
 2021-11-25 18:19:09 +01:00
Thomas GruberandGitHub 1e7a75598e Add collector to read data from ipmitool or ipmi-sensors (fallback) (#9) 
* Add collector to read data from ipmitool or ipmi-sensors

* Update IPMI collector to use own config

* Add 'ipmitool sensor' parser
 2021-11-25 18:15:56 +01:00
Thomas GruberandGitHub 5d9ce2bf6c Update rpmbuild.yml 2021-11-22 17:22:05 +01:00
Thomas GruberandGitHub 5ac48350eb Update rpmbuild.yml 2021-11-22 17:19:41 +01:00
Thomas GruberandGitHub 531761d323 Update rpmbuild.yml 2021-11-22 17:04:39 +01:00
Thomas GruberandGitHub 92105602c2 Update rpmbuild.yml 2021-11-22 16:53:53 +01:00
Thomas GruberandGitHub 661558ba6d Update rpmbuild.yml 2021-11-22 16:48:21 +01:00
Thomas GruberandGitHub d4229ecfee Update rpmbuild.yml 2021-11-22 16:34:35 +01:00
Thomas GruberandGitHub 14fdb28d51 Create rpmbuild.yml 2021-11-22 16:33:06 +01:00
Thomas GruberandGitHub f37a9575c1 Merge pull request #5 from ClusterCockpit/system_scripts 
Add init script and unit file for systemd
 2021-11-02 15:15:36 +01:00
Thomas GruberandGitHub 3d47f894fa Merge pull request #7 from ClusterCockpit/alternate_storage 
Alternative storage
 2021-11-02 15:15:14 +01:00
Thomas Roehl dc4b8d13c2 Add collectors for custom commands and /proc/diskstat. Per default add a hostname tag to all measurements 2021-10-08 13:29:57 +02:00
Thomas Roehl f74d5dc51a Reuse parser to avoid overhead in NATS receiver 2021-10-08 13:28:44 +02:00
Thomas Roehl 4688997071 Add two helper functions to collector definition 2021-10-08 13:28:02 +02:00
Thomas Roehl 7f1694adc7 Fix format of infiniband collector 2021-10-08 13:27:36 +02:00
Thomas Roehl cdc253b569 Return proper error for init() of memstat collector 2021-10-08 13:27:15 +02:00
Thomas Roehl 1762e33c36 Adjust README 2021-10-04 16:22:30 +02:00
Thomas GruberandGitHub d7ef32de18 Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00
Thomas Roehl 486df91a9d Fix format 2021-10-04 15:47:03 +02:00
Thomas Roehl b06572c865 Reset buffer in nats sink 2021-10-04 15:46:51 +02:00
Thomas Roehl 558bbaba59 Change storage format 2021-10-04 15:23:43 +02:00
Thomas Roehl 2083906086 Add init script and unit file for systemd 2021-05-29 14:11:44 +02:00
Thomas Roehl 8956ce7ad5 CLI option for PID file and fix for DefaultTags 2021-05-29 03:40:12 +02:00
Thomas Roehl ed2020a490 Update README 2021-05-19 19:24:01 +02:00
Thomas Roehl 99f3f3b815 Update READMEs 2021-05-19 01:46:52 +02:00
Thomas Roehl da11523448 Add default_tags to example config 2021-05-19 01:46:37 +02:00
Thomas Roehl 39c96da409 Add config option for default tags like cluster name 2021-05-19 01:34:30 +02:00
Thomas Roehl 9b59429a93 Add some more memory metrics to memstat collector 2021-05-12 18:09:15 +02:00
Thomas Roehl fd6f19ea07 Add proc_total and proc_run to loadavg collector 2021-05-12 18:08:43 +02:00
Thomas Roehl 14bc29f766 Add collector for the top5 processes in %cpu 2021-05-12 17:47:12 +02:00
Thomas Roehl b84dc08cd6 Enable strings as field values in stdout sink 2021-05-12 17:46:16 +02:00
Thomas Roehl e8b2bbdae1 Fix format for cpustat collector 2021-05-12 17:45:41 +02:00
Thomas Roehl e35a9f6b86 Add Cpustat collector for /proc/stat 2021-05-12 16:57:18 +02:00
Thomas Roehl feb9f948f8 Add section about installation and running 2021-05-12 14:46:50 +02:00
Thomas Roehl 1d3a11016e Update README with infos for InfiniBand and Lustre collector and updates to LIKWID Makefile 2021-05-12 14:41:54 +02:00
Thomas Roehl fc01781ce5 Update Makefile to use central LIKWID installation 2021-05-12 14:28:10 +02:00
Thomas Roehl 47510099cf Add logfile command line option 2021-05-11 13:16:35 +02:00
Thomas Roehl 4ee4e617b2 Read config file path from commandline 2021-05-11 12:41:29 +02:00