* Add cpu_used (all-cpu_idle) to CpustatCollector
* Update cc-metric-collector.init
* Allow selection of timestamp precision in HttpSink
* Add comment about precision requirement for cc-metric-store
* Fix for API changes in gofish@v0.15.0
* Update requirements to latest version
* Read sensors through redfish
* Update golang toolchain to 1.21
* Remove stray error check
* Update main config in configuration.md
* Update Release action to use golang 1.22 stable release, no golang RPMs anymore
* Update runonce action to use golang 1.22 stable release, no golang RPMs anymore
* Update README.md
Use right JSON type in configuration
* Update sink's README
* Test whether ipmitool or ipmi-sensors can be executed without errors
* Little fixes to the prometheus sink (#115)
* Add uint64 to float64 cast option
* Add prometheus sink to the list of available sinks
* Add aggregated counters by gpu for nvlink errors
---------
Co-authored-by: Michael Schwarz <schwarz@uni-paderborn.de>
* Ccmessage migration (#119)
* Add cpu_used (all-cpu_idle) to CpustatCollector
* Update cc-metric-collector.init
* Allow selection of timestamp precision in HttpSink
* Add comment about precision requirement for cc-metric-store
* Fix for API changes in gofish@v0.15.0
* Update requirements to latest version
* Read sensors through redfish
* Update golang toolchain to 1.21
* Remove stray error check
* Update main config in configuration.md
* Update Release action to use golang 1.22 stable release, no golang RPMs anymore
* Update runonce action to use golang 1.22 stable release, no golang RPMs anymore
* Switch to CCMessage for all files.
---------
Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu>
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
* Switch to ccmessage also for latest additions in nvidiaMetric
* New Message processor (#118)
* Add cpu_used (all-cpu_idle) to CpustatCollector
* Update cc-metric-collector.init
* Allow selection of timestamp precision in HttpSink
* Add comment about precision requirement for cc-metric-store
* Fix for API changes in gofish@v0.15.0
* Update requirements to latest version
* Read sensors through redfish
* Update golang toolchain to 1.21
* Remove stray error check
* Update main config in configuration.md
* Update Release action to use golang 1.22 stable release, no golang RPMs anymore
* Update runonce action to use golang 1.22 stable release, no golang RPMs anymore
* New message processor to check whether a message should be dropped or manipulate it in flight
* Create a copy of message before manipulation
---------
Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu>
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
* Update collector's Makefile and go.mod/sum files
* Use message processor in router, all sinks and all receivers
* Add support for credential file (NKEY) to NATS sink and receiver
* Fix JSON keys in message processor configuration
* Update docs for message processor, router and the default router config file
* Add link to expr syntax and fix regex matching docs
* Update sample collectors
* Minor style change in collector manager
* Some helpers for ccTopology
* LIKWID collector: write log owner change only once
* Fix for metrics without units and reduce debugging messages for messageProcessor
* Use shorted hostname for hostname added by router
* Define default port for NATS
* CPUstat collector: only add unit for applicable metrics
* Add precision option to all sinks using Influx's encoder
* Add message processor to all sink documentation
* Add units to documentation of cpustat collector
---------
Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu>
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
Co-authored-by: oscarminus <me@oscarminus.de>
Co-authored-by: Michael Schwarz <schwarz@uni-paderborn.de>
* Add cpu_used (all-cpu_idle) to CpustatCollector
* Update to line-protocol/v2
* Update runonce.yml with Golang 1.20
* Update fsnotify in LIKWID Collector
* Use not a pointer to line-protocol.Encoder
* Simplify Makefile
* Use only as many arguments as required
* Allow sum function to handle non float types
* Allow values to be a slice of type float64, float32, int, int64, int32, bool
* Use generic function to simplify code
* Add missing case for type []int32
* Use generic function to compute minimum
* Use generic function to compute maximum
* Use generic function to compute average
* Add error value to sumAnyType
* Use generic function to compute median
* For older versions of go slices is not part of the installation
* Remove old entries from go.sum
* Use simpler sort function
* Compute metrics ib_total and ib_total_pkts
* Add aggregated metrics.
Add missing units
* Update likwidMetric.go
Fixes a potential bug when `fsnotify.NewWatcher()` fails with an error
* Completly avoid memory allocations in infinibandMetric read()
* Fixed initialization: Initalization and measurements should run in the same thread
* Add safe.directory to Release action
* Fix path after installation to /usr/bin after installation
* ioutil.ReadFile is deprecated: As of Go 1.16, this function simply calls os.ReadFile
* Switch to package slices from the golang 1.21 default library
* Read file line by line
* Read file line by line
* Read file line by line
* Use CamelCase
* Use CamelCase
* Fix function getNumaDomain, it always returned 0
* Avoid type conversion by using Atoi
Avoid copying structs by using pointer access
Increase readability with CamelCase variable names
* Add caching
* Cache CpuData
* Cleanup
* Use init function to initalize cache structure to avoid multi threading problems
* Reuse information from /proc/cpuinfo
* Avoid slice cloning. Directly use the cache
* Add DieList
* Add NumaDomainList and SMTList
* Cleanup
* Add comment
* Lookup core ID from /sys/devices/system/cpu, /proc/cpuinfo is not portable
* Lookup all information from /sys/devices/system/cpu, /proc/cpuinfo is not portable
* Correctly handle lists from /sys
* Add Simultaneous Multithreading siblings
* Replace deprecated thread_siblings_list by core_cpus_list
* Reduce number of required slices
* Allow to send total values per core, socket and node
* Send all metrics with same time stamp
calcEventsetMetrics does only computiation, counter measurement is done before
* Input parameters should be float64 when evaluating to float64
* Send all metrics with same time stamp
calcGlobalMetrics does only computiation, counter measurement is done before
* Remove unused variable gmresults
* Add comments
* Updated go packages
* Add build with golang 1.21
* Switch to checkout action version 4
* Switch to setup-go action version 4
* Add workflow_dispatch to allow manual run of workflow
* Add workflow_dispatch to allow manual run of workflow
* Add release build jobs to runonce.yml
* Switch to golang 1.20 for RHEL based distributions
* Use dnf to download golang
* Remove golang versions before 1.20
* Upgrade Ubuntu focal -> jammy
* Pipe golang tar package directly to tar
* Update golang version
* Fix Ubuntu version number
* Add links to ipmi and redfish receivers
* Fix http server addr format
* github.com/influxdata/line-protocol -> github.com/influxdata/line-protocol/v2/lineprotocol
* Corrected spelling
* Add some comments
* github.com/influxdata/line-protocol -> github.com/influxdata/line-protocol/v2/lineprotocol
* Allow other fields not only field "value"
* Add some basic debugging documentation
* Add some basic debugging documentation
* Use a lock for the flush timer
* Add tags in lexical order as required by AddTag()
* Only access meta data, when it gets used as tag
* Use slice to store lexialicly orderd key value pairs
* Increase golang version requirement to 1.20.
* Avoid package cmp to allow builds with golang v1.20
* Fix: Error NVML library not found did crash
cc-metric-collector with "SIGSEGV: segmentation violation"
* Add config option idle_timeout
* Add basic authentication support
* Add basic authentication support
* Avoid unneccessary memory allocations
* Add documentation for send_*_total values
* Use generic package maps to clone maps
* Reuse flush timer
* Add Influx client options
* Reuse ccTopology functionality
* Do not store unused topology information
* Add batch_size config
* Cleanup
* Use stype and stype-id for the NIC in NetstatCollector
* Wait for concurrent flush operations to finish
* Be more verbose in error messages
* Reverted previous changes.
Made the code to complex without much advantages
* Use line protocol encoder
* Go pkg update
* Stop flush timer, when immediatelly flushing
* Fix: Corrected unlock access to batch slice
* Add config option to specify whether to use GZip compression in influx write requests
* Add asynchron send of encoder metrics
* Use DefaultServeMux instead of github.com/gorilla/mux
* Add config option for HTTP keep-alives
* Be more strict, when parsing json
* Add config option for HTTP request timeout and Retry interval
* Allow more then one background send operation
* Fix %sysusers_create_package args (#108)
%sysusers_create_package requires two arguments. See: https://github.com/systemd/systemd/blob/main/src/rpm/macros.systemd.in#L165
* Add nfsiostat to list of collectors
---------
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
Co-authored-by: Holger Obermaier <holgerob@gmx.de>
Co-authored-by: Obihörnchen <obihoernchende@gmail.com>
* cpustatMetric.go: Use derived values instead of absolute values
The values in /proc/stat are absolute counters related to the boot
time of the system. To obtain a utilization of the CPU, the changes
in the counters must be derived according to time. To take only the
absolute values leads to the fact that changes in the utilization,
straight with larger values, do not become visible.
* Add new collector for /proc/schedstat
The `schedstat` collector reads data from /proc/schedstat and calculates
a load value, separated by hwthread. This might be useful to detect bad
cpu pinning on shared nodes etc.
Co-authored-by: Michael Schwarz <post@michael-schwarz.name>
* Add collector for AMD ROCm SMI metrics
* Fix import path
* Fix imports
* Remove Board Number
* store GPU index explicitly
* Remove board number from description
* Provide info to CollectorManager whether the collector can be executed in parallel with others
* Split serial and parallel collectors. Read in parallel first
* added beegfs collectors to collectors/README.md
* added beegfs collectors and docs
* added new beegfs collectors to AvailableCollectors list
* Feedback implemented
* changed error type
* changed error to only return
* changed beegfs lookup path
* fixed typo in md files
Co-authored-by: Mehmet Soysal <mehmet.soysal@kit.edu>
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally
* Use central timer for collectors and router. Add expressions to router
* Add expression to router config
* Update entry points
* Start with README
* Update README for CCMetric
* Formatting
* Update README.md
* Add README for MultiChanTicker
* Add README for MultiChanTicker
* Update README.md
* Add README to metric router
* Update main README
* Remove SinkEntity type
* Update README for sinks
* Update go files
* Update README for receivers
* Update collectors README
* Update collectors README
* Use seperate page per collector
* Fix for tempstat page
* Add docs for customcmd collector
* Add docs for ipmistat collector
* Add docs for topprocs collector
* Update customCmdMetric.md
* Use seconds when calculating LIKWID metrics
* Add IB metrics ib_recv_pkts and ib_xmit_pkts
* Drop domain part of host name
* Updated to latest stable version of likwid
* Define source code dependencies in Makefile
* Add GPFS / IBM Spectrum Scale collector
* Add vet and staticcheck make targets
* Add vet and staticcheck make targets
* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces
* Add sample collector to README.md
* Add CPU frequency collector
* Avoid staticcheck warning: redundant return statement
* Avoid staticcheck warning: unnecessary assignment to the blank identifier
* Simplified code
* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread
* Add collector for NFS clients
* Move publication of metrics into Flush() for NatsSink
* Update GitHub actions
* Refactoring
* Avoid vet warning: Println arg list ends with redundant newline
* Avoid vet warning struct field commands has json tag but is not exported
* Avoid vet warning: return copies lock value.
* Corrected typo
* Refactoring
* Add go sources in internal/...
* Bad separator in Makefile
* Fix Infiniband collector
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
