Commit Graph

66 Commits

Author SHA1 Message Date
Holger Obermaier a97c705f4c Do not create link to libganglia.so.
libganglia.so is now loaded during runtime by dlopen
and no longer required during link time
2022-02-21 20:55:14 +01:00
Holger Obermaier 54c693532f Do not create fake libganglia.so.
libganglia.so is now loaded during runtime by dlopen
and no longer required during link time
2022-02-21 17:28:54 +01:00
Thomas Roehl 842395682c Remove -lganglia ldflag and check dl lib 2022-02-21 17:01:13 +01:00
Holger Obermaier e8adf5b909 Create an empty libganglia.so stub if none exists 2022-02-18 15:44:12 +01:00
Thomas Roehl d9a81501e5 Use common functions in both Ganglia sinks 2022-02-18 15:05:45 +01:00
Thomas Roehl e2f78fe1c0 Add linker flag -Wl,--unresolved-symbols=ignore-in-object-files to build without library. Remove build tags 2022-02-18 11:41:15 +01:00
Holger Obermaier bb9c6be741 Do not create libganglia link if libganglia is not installed 2022-02-17 08:03:55 +01:00
Thomas Roehl 3f8a2cb9b8 Fix default gmond config file path for libgangliaSink 2022-02-17 01:33:38 +01:00
Thomas Roehl 7be6d2a338 Use only readable files when searching for libganglia.so 2022-02-16 23:23:07 +01:00
Thomas Roehl e5585eaaa0 Add additional lines for build tags 2022-02-16 19:07:00 +01:00
Thomas Gruber e28c1fb30b Ganglia sink using libganglia.so directly (#35)
* Add sink directly using libganglia.so

* Remove unneeded confuse header

* add submodule init to build action

* add submodule init to runonce action

* add installation og ganglia to runonce

* add installation of ganglia to runonce

* add installation of ganglia to runonce

* libconfuse not required

* Remove ganglia submodule

* Remove ganglia.h

* Add Makefile to help creating the libganglia.so link

* Fix cgo header

* Rename new Ganglia sink to 'libgangliaSink'

* Add documentation for libgangliaSink

* Extend make buildsystem with find&symlink helper for libgangliaSink

* Add metric renaming function

* Add build tag 'ganglia' and create corresponding files
2022-02-16 18:33:46 +01:00
Holger Obermaier a3ad9d0cb0 Move toTheSinks out of for loop 2022-02-14 18:53:37 +01:00
Thomas Roehl 247fb23de1 Try to operate on multiple metrics if channels if filled 2022-02-14 18:12:50 +01:00
Holger Obermaier e4285f02c5 Avoid one append 2022-02-11 17:17:25 +01:00
Thomas Roehl dc1e4f28a0 Check for Ganglia group explicitly in tags and meta info 2022-02-11 16:10:59 +01:00
Lou 442e512f2d Automatically flush batched writes in the HTTP sink (#31)
* Add error handling for Sink.Write

* simplify HttpSink config

* HttpSink: dynamically sized batches flushed after timer

* fix panic if sink type does not exist
2022-02-10 13:12:32 +01:00
Holger Obermaier 2aa8c812a6 Add config option flush_interval 2022-02-10 09:43:02 +01:00
Holger Obermaier acf5db543e Fix: Reset counter 2022-02-09 23:22:54 +01:00
Holger Obermaier a0e97d216a Move all flush operations to the sinks 2022-02-09 19:47:49 +01:00
Holger Obermaier 1d299be3ea Add comments 2022-02-09 11:08:50 +01:00
Holger Obermaier b4d7643c25 Add comments 2022-02-09 10:09:03 +01:00
Thomas Roehl 7f78a5baf2 Add timeout options to httpSink 2022-02-08 18:06:07 +01:00
Thomas Roehl fec3c5981d Fix for gangliaSink 2022-02-08 18:04:08 +01:00
Thomas Roehl 7e4c35e224 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-08 13:46:48 +01:00
Thomas Roehl 8c744617be Remove logging and enable command execution for GangliaSink 2022-02-08 13:45:41 +01:00
Holger Obermaier 377f85111a Refactoring: Use ccmetric function ToLineProtocol() in stdout sink 2022-02-08 13:38:35 +01:00
Holger Obermaier 6d55c376bd Refactoring: Remove all *List() functions from CCMetric 2022-02-08 11:23:19 +01:00
Holger Obermaier d98b678399 Refactoring: Replace FieldList() -> Fields() 2022-02-08 11:05:07 +01:00
Holger Obermaier e1a7379c2e Generate influxDB point for data type ccMetric 2022-02-08 09:31:08 +01:00
Holger Obermaier af051b5e7e Replace FieldList() by Fields() 2022-02-07 22:52:39 +01:00
Holger Obermaier 7b104ebe90 Use cclog.ComponentDebug. Avoid copying point.Fields() 2022-02-07 18:00:17 +01:00
Thomas Gruber d1e66201a6 Add non-blocking InfluxDB sink (#29)
* Add non-blocking InfluxDB sink

* Add configurable batch size
2022-02-07 16:51:46 +01:00
Thomas Gruber fdb58b0be2 Sink specific configuration maps (#25)
* Use sink-specific configurations to have more flexibility. Adjust sample sink configuration files

* Add documentation

* Add links to individual sink readmes

* Fix link in README

* HTTPS for HttpSink

* If no CPU die id available, use the socket id instead
2022-02-04 18:12:24 +01:00
Thomas Roehl 02cd21abe2 HTTPS for HttpSink 2022-02-04 12:39:25 +01:00
Thomas Gruber 6ff6cb7219 Change CCMetric's internal data structure (#22)
* package ccmetric rewrite

* Create deep copy in New() to avoid access conflicts

* Renamed TagMap() -> Tags(), MetaMap() -> Meta

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-02-01 14:54:34 +01:00
Thomas Roehl 011218ab80 Adjust ganglia sink to CCMetric 2022-01-30 15:25:57 +01:00
Thomas Gruber d915bcc02d Add sink to add metrics to Ganglia through gmetric (#15) 2022-01-30 15:21:24 +01:00
Holger Obermaier 9e99e47d73 Wait for close of done channel, to ensure manager finished. 2022-01-30 12:08:33 +01:00
Holger Obermaier 4e408f9490 Add documentation 2022-01-28 15:16:58 +01:00
Holger Obermaier b9236dcc31 Handle shutdown sequentially 2022-01-27 17:43:00 +01:00
Thomas Roehl 3fd77e6887 Use non-blocking send at close, use common done function and remove default case 2022-01-26 16:54:51 +01:00
Thomas Roehl 2925ad9f40 Use ccLogger anywhere 2022-01-25 17:43:10 +01:00
Thomas Gruber 200af84c54 Modularize the whole thing (#16)
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-01-25 15:37:43 +01:00
Holger Obermaier 5dd2af4e8f Avoid staticcheck warning: redundant return statement 2022-01-21 14:35:52 +01:00
Thomas Roehl 325d97df96 Updated contributing section for sinks 2021-11-26 15:56:52 +01:00
Thomas Roehl 32d5a5b98f Update sink README.md 2021-11-26 14:18:59 +01:00
Thomas Roehl 24bf9590ee Update sink README.md 2021-11-26 14:10:22 +01:00
Lou Knauer 485223c590 Add new httpSink
This sink is compatible with the HTTP API of cc-metric-store.
Example config.json section:

```
"sink": {
    "type": "http",
    "host": "localhost",
    "port": "8080",
    "database": "api/write",
    "password": "<JWT>"
},
```

The password/JWT can be omitted.
2021-10-12 13:44:38 +02:00
Lou Knauer cdc1811576 Add Flush method to sink interface 2021-10-12 13:43:58 +02:00
Thomas Gruber d7ef32de18 Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00