Commit Graph

40 Commits

Author SHA1 Message Date
Thomas Gruber
195d0794b0
Merge develop branch into main (#106)
* Add cpu_used (all-cpu_idle) to CpustatCollector

* Update to line-protocol/v2

* Update runonce.yml with Golang 1.20

* Update fsnotify in LIKWID Collector

* Use not a pointer to line-protocol.Encoder

* Simplify Makefile

* Use only as many arguments as required

* Allow sum function to handle non float types

* Allow values to be a slice of type float64, float32, int, int64, int32, bool

* Use generic function to simplify code

* Add missing case for type []int32

* Use generic function to compute minimum

* Use generic function to compute maximum

* Use generic function to compute average

* Add error value to sumAnyType

* Use generic function to compute median

* For older versions of go slices is not part of the installation

* Remove old entries from go.sum

* Use simpler sort function

* Compute metrics ib_total and ib_total_pkts

* Add aggregated metrics.
Add missing units

* Update likwidMetric.go

Fixes a potential bug when `fsnotify.NewWatcher()` fails with an error

* Completly avoid memory allocations in infinibandMetric read()

* Fixed initialization: Initalization and measurements should run in the same thread

---------

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2023-08-29 14:12:49 +02:00
Thomas Roehl
a0acf01dc3 Build DEB package for Ubuntu 20.04 for releases 2022-09-28 12:19:36 +02:00
Thomas Roehl
a2f0bc37d4 Add runonce job for Golang 1.19 2022-08-03 17:06:28 +02:00
Thomas Roehl
09b740b82e Make wget quiet in Release action to reduce log size 2022-07-12 12:37:10 +02:00
Thomas Gruber
f79b7b5e2b
Merge branch 'main' into develop 2022-07-12 11:36:46 +02:00
Thomas Roehl
b16343e5e2 Use Golang 1.18 in Release action to build RPMs 2022-07-12 11:30:27 +02:00
Thomas Roehl
4fa37a58f2 Remove Golang version 1.16 an 1.17 from Action. Latest commits require Golang 1.18 2022-07-11 16:01:45 +02:00
Thomas Roehl
31a38bc17d Update release action 2022-06-09 14:36:25 +02:00
Thomas Gruber
0d31ec481b
Update Release.yml 2022-06-09 12:42:11 +02:00
Thomas Gruber
8d85bd53f1
Merge latest development changes to main branch (#79)
* Cleanup: Remove unused code

* Use Golang duration parser for 'interval' and 'duration'
 in main config

* Update handling of LIKWID headers. Download only if not already present in the system. Fixes #73

* Units with cc-units (#64)

* Add option to normalize units with cc-unit

* Add unit conversion to router

* Add option to change unit prefix in the router

* Add to MetricRouter README

* Add order of operations in router to README

* Use second add_tags/del_tags only if metric gets renamed

* Skip disks in DiskstatCollector that have size=0

* Check readability of sensor files in TempCollector

* Fix for --once option

* Rename `cpu` type to `hwthread` (#69)

* Rename 'cpu' type to 'hwthread' to avoid naming clashes with MetricStore and CC-Webfrontend

* Collectors in parallel (#74)

* Provide info to CollectorManager whether the collector can be executed in parallel with others

* Split serial and parallel collectors. Read in parallel first

* Update NvidiaCollector with new metrics, MIG and NvLink support (#75)

* CC topology module update (#76)

* Rename CPU to hardware thread, write some comments

* Do renaming in other parts

* Remove CpuList and SocketList function from metricCollector. Available in ccTopology

* Option to use MIG UUID as subtype-id in NvidiaCollector

* Option to use MIG slice name as subtype-id in NvidiaCollector

* MetricRouter: Fix JSON in README

* Fix for Github Action to really use the selected version

* Remove Ganglia installation in runonce Action and add Go 1.18

* Fix daemon options in init script

* Add separate go.mod files to use it with deprecated 1.16

* Minor updates for Makefiles

* fix string comparison

* AMD ROCm SMI collector (#77)

* Add collector for AMD ROCm SMI metrics

* Fix import path

* Fix imports

* Remove Board Number

* store GPU index explicitly

* Remove board number from description

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

* Improved http sink (#78)

* automatic flush in NatsSink

* tweak default options of HttpSink

* shorter cirt. section and retries for HttpSink

* fix error handling

* Remove file added by mistake.

* Use http instead of ftp to download likwid

* Fix serial number in rocmCollector

Co-authored-by: Thomas Roehl <thomas.roehl@fau.de>

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
Co-authored-by: Lou <lou.knauer@gmx.de>
2022-06-08 15:25:40 +02:00
Thomas Roehl
cc84a94647 Remove Ganglia installation in runonce Action and add Go 1.18 2022-05-23 17:37:14 +02:00
Thomas Roehl
838b8d824d Fix for Github Action to really use the selected version 2022-05-23 16:50:58 +02:00
Thomas Gruber
3f76947f54
Merge latest developments into main (#67)
* Update configuration.md

Add an additional receiver to have better alignment of components

* Change default GpfsCollector command to `mmpmon` (#53)

* Set default cmd to 'mmpmon'

* Reuse looked up path

* Cast const to string

* Just download LIKWID to get the headers (#54)

* Just download LIKWID to get the headers

* Remove perl-Data-Dumper from BuildRequires, only required by LIKWID build

* Add HttpReceiver as counterpart to the HttpSink (#49)

* Use GBytes as unit for large memory numbers

* Make maxForward configurable, save old name in meta in rename metrics and make the hostname tag key configurable

* Single release action (#55)

Building all RPMs and releasing in a single workflow

* Makefile target to build binary-only Debian packages (#61)

* Add 'install' and 'DEB' make targets to build binary-only Debian packages

* Add control file for DEB builds

* Use a single line for bash loop in make clean

* Add config options for retry intervals of InfluxDB clients (#59)

* Refactoring of LikwidCollector and metric units (#62)

* Reduce complexity of LikwidCollector and allow metric units

* Add unit to LikwidCollector docu and fix some typos

* Make library path configurable

* Use old metric name in Ganglia if rename has happened in the router (#60)

* Use old metric name if rename has happened in the router

* Also check for Ganglia renames for the oldname

* Derived metrics (#57)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Simplified iota usage

* Move unit tag to meta data tags

* Derived metrics (#65)

* Add time-based derivatived (e.g. bandwidth) to some collectors

* Add documentation

* Add comments

* Fix: Only compute rates with a valid previous state

* Only compute rates with a valid previous state

* Define const values for net/dev fields

* Set default config values

* Add comments

* Refactor: Consolidate data structures

* Refactor: Consolidate data structures

* Refactor: Avoid struct deep copy

* Refactor: Avoid redundant tag maps

* Refactor: Use int64 type for absolut values

* Update LustreCollector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>

* Meta to tags list and map for sinks (#63)

* Change ccMetric->Influx functions

* Use a meta_as_tags string list in config but create a lookup map afterwards

* Add meta as tag logic to sampleSink

* Fix staticcheck warnings (#66)

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-03-15 16:41:11 +01:00
Thomas Gruber
21edca5f88
Single release action (#55)
Building all RPMs and releasing in a single workflow
2022-03-11 13:37:47 +01:00
Thomas Roehl
f7e8b52667 Run RPM build actions only on tag push 2022-03-02 13:21:54 +01:00
Holger Obermaier
33d954f767
Action red hat universal base image (#52)
Add Red Hat Universal Base Image 8 Workflow
2022-03-01 17:15:20 +01:00
Holger Obermaier
a5325a6535
GitHub actions (#51)
Create new GitHub action which uses unmodified AlmaLinux Docker image
2022-03-01 15:39:26 +01:00
Thomas Roehl
5742721cbd Proper tag name in release 2022-02-21 18:58:13 +01:00
Thomas Roehl
1425463a26 Upload RPM not SRPM 2022-02-21 18:49:39 +01:00
Thomas Gruber
5cd7ac2541
Update rpmbuild.yml 2022-02-21 18:28:07 +01:00
Thomas Gruber
afa831aff8
Update rpmbuild.yml 2022-02-21 18:17:12 +01:00
Thomas Roehl
a1328b30e9 Remove CentOS8 from RPMBUILD action. End of Life 2022-02-21 17:16:07 +01:00
Thomas Roehl
3ade75490c Add RHEL UBI 8 to rpmbuild action 2022-02-21 15:09:42 +01:00
Thomas Gruber
e28c1fb30b
Ganglia sink using libganglia.so directly (#35)
* Add sink directly using libganglia.so

* Remove unneeded confuse header

* add submodule init to build action

* add submodule init to runonce action

* add installation og ganglia to runonce

* add installation of ganglia to runonce

* add installation of ganglia to runonce

* libconfuse not required

* Remove ganglia submodule

* Remove ganglia.h

* Add Makefile to help creating the libganglia.so link

* Fix cgo header

* Rename new Ganglia sink to 'libgangliaSink'

* Add documentation for libgangliaSink

* Extend make buildsystem with find&symlink helper for libgangliaSink

* Add metric renaming function

* Add build tag 'ganglia' and create corresponding files
2022-02-16 18:33:46 +01:00
Thomas Gruber
862630a218
Extend workflow to test Go 1.16 and 1.17 2022-01-31 14:42:19 +01:00
Thomas Gruber
1f55aa247f
Run rpmbuild workflow only for new tags 2022-01-31 13:29:14 +01:00
Thomas Gruber
6abbc5f77e
Fix Github Actions (#18)
* Fix config for Github Actions

* Fix paths

* Add CentOS Latest and AlmaLinux 8.5 to RPM action

* Fix ID

* Reduce min Go version to 1.16 and use time.Unix in gpfsMetric
2022-01-30 14:54:36 +01:00
Holger Obermaier
7953629940 Update GitHub actions 2022-01-24 15:55:15 +01:00
Thomas Gruber
15cf16b46f
Add Github Action to build RPM (#14)
* Update SPEC file

* Add clean target to Makefile

* add rpmbuild action

* Add missing build dependency

* Fix package name of golang for CentOS8

* Change outputs

* Fix file list and build requires

* Fix for file list

* Also upload SRPM
2021-11-29 16:04:50 +01:00
Thomas Roehl
f5b7da5046 Use configuration with all supported collectors for CI. Fixes #13 2021-11-26 18:15:04 +01:00
Thomas Roehl
c6edf7aed4 Fix application name 2021-11-25 17:54:58 +01:00
Thomas Roehl
59ac123662 Add Makefile, update LIKWID Makefile and fix Github Action 2021-11-25 17:51:34 +01:00
Thomas Roehl
71e86fe79d Simple run test for Github Action. No RPM build 2021-11-25 16:24:33 +01:00
Thomas Gruber
5d9ce2bf6c
Update rpmbuild.yml 2021-11-22 17:22:05 +01:00
Thomas Gruber
5ac48350eb
Update rpmbuild.yml 2021-11-22 17:19:41 +01:00
Thomas Gruber
531761d323
Update rpmbuild.yml 2021-11-22 17:04:39 +01:00
Thomas Gruber
92105602c2
Update rpmbuild.yml 2021-11-22 16:53:53 +01:00
Thomas Gruber
661558ba6d
Update rpmbuild.yml 2021-11-22 16:48:21 +01:00
Thomas Gruber
d4229ecfee
Update rpmbuild.yml 2021-11-22 16:34:35 +01:00
Thomas Gruber
14fdb28d51
Create rpmbuild.yml 2021-11-22 16:33:06 +01:00