On nodes with >300 cores, one interval's burst of per-hwthread metrics
overran the fixed 200-slot channels. With blocking sends at every hop,
sink back-pressure propagated to the collectors, the collection round
exceeded the interval, and time.Ticker silently dropped the missed
ticks - whole intervals were skipped without any log message.
- multiChanTicker: deliver ticks non-blockingly and warn when a
consumer misses a tick instead of stalling all consumers; guard the
channel list with a mutex (data race with AddChannel)
- collectorManager: run the collection round detached from the tick
loop, skip-and-warn when a round is still running, log per-collector
and per-round durations at debug level, close serial collectors on
shutdown
- metricRouter: buffer the interval timestamp channel and drain it
before stamping, so metrics never carry the previous interval's
timestamp; warn when the collector input channel is full at tick time
- main: scale the inter-manager channels to max(200, 24*NumCPU),
overridable with the new optional channel_buffer_size config option
- add first unit tests for ticker, collector manager and router
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add cpu_used (all-cpu_idle) to CpustatCollector
* Update cc-metric-collector.init
* Allow selection of timestamp precision in HttpSink
* Add comment about precision requirement for cc-metric-store
* Fix for API changes in gofish@v0.15.0
* Update requirements to latest version
* Read sensors through redfish
* Update golang toolchain to 1.21
* Remove stray error check
* Update main config in configuration.md
* Update Release action to use golang 1.22 stable release, no golang RPMs anymore
* Update runonce action to use golang 1.22 stable release, no golang RPMs anymore
* Update README.md
Use right JSON type in configuration
* Update sink's README
* Test whether ipmitool or ipmi-sensors can be executed without errors
---------
Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu>
Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
moebiusband
Thomas Roehl
Thomas Roehl