Commit Graph

65 Commits

Author SHA1 Message Date
Holger Obermaier
54c693532f Do not create fake libganglia.so.
libganglia.so is now loaded during runtime by dlopen
and no longer required during link time
2022-02-21 17:28:54 +01:00
Thomas Roehl
842395682c Remove -lganglia ldflag and check dl lib 2022-02-21 17:01:13 +01:00
Holger Obermaier
e8adf5b909 Create an empty libganglia.so stub if none exists 2022-02-18 15:44:12 +01:00
Thomas Roehl
d9a81501e5 Use common functions in both Ganglia sinks 2022-02-18 15:05:45 +01:00
Thomas Roehl
e2f78fe1c0 Add linker flag -Wl,--unresolved-symbols=ignore-in-object-files to build without library. Remove build tags 2022-02-18 11:41:15 +01:00
Holger Obermaier
bb9c6be741 Do not create libganglia link if libganglia is not installed 2022-02-17 08:03:55 +01:00
Thomas Roehl
3f8a2cb9b8 Fix default gmond config file path for libgangliaSink 2022-02-17 01:33:38 +01:00
Thomas Roehl
7be6d2a338 Use only readable files when searching for libganglia.so 2022-02-16 23:23:07 +01:00
Thomas Roehl
e5585eaaa0 Add additional lines for build tags 2022-02-16 19:07:00 +01:00
Thomas Gruber
e28c1fb30b
Ganglia sink using libganglia.so directly (#35)
* Add sink directly using libganglia.so

* Remove unneeded confuse header

* add submodule init to build action

* add submodule init to runonce action

* add installation og ganglia to runonce

* add installation of ganglia to runonce

* add installation of ganglia to runonce

* libconfuse not required

* Remove ganglia submodule

* Remove ganglia.h

* Add Makefile to help creating the libganglia.so link

* Fix cgo header

* Rename new Ganglia sink to 'libgangliaSink'

* Add documentation for libgangliaSink

* Extend make buildsystem with find&symlink helper for libgangliaSink

* Add metric renaming function

* Add build tag 'ganglia' and create corresponding files
2022-02-16 18:33:46 +01:00
Holger Obermaier
a3ad9d0cb0 Move toTheSinks out of for loop 2022-02-14 18:53:37 +01:00
Thomas Roehl
247fb23de1 Try to operate on multiple metrics if channels if filled 2022-02-14 18:12:50 +01:00
Holger Obermaier
e4285f02c5 Avoid one append 2022-02-11 17:17:25 +01:00
Thomas Roehl
dc1e4f28a0 Check for Ganglia group explicitly in tags and meta info 2022-02-11 16:10:59 +01:00
Lou
442e512f2d
Automatically flush batched writes in the HTTP sink (#31)
* Add error handling for Sink.Write

* simplify HttpSink config

* HttpSink: dynamically sized batches flushed after timer

* fix panic if sink type does not exist
2022-02-10 13:12:32 +01:00
Holger Obermaier
2aa8c812a6 Add config option flush_interval 2022-02-10 09:43:02 +01:00
Holger Obermaier
acf5db543e Fix: Reset counter 2022-02-09 23:22:54 +01:00
Holger Obermaier
a0e97d216a Move all flush operations to the sinks 2022-02-09 19:47:49 +01:00
Holger Obermaier
1d299be3ea Add comments 2022-02-09 11:08:50 +01:00
Holger Obermaier
b4d7643c25 Add comments 2022-02-09 10:09:03 +01:00
Thomas Roehl
7f78a5baf2 Add timeout options to httpSink 2022-02-08 18:06:07 +01:00
Thomas Roehl
fec3c5981d Fix for gangliaSink 2022-02-08 18:04:08 +01:00
Thomas Roehl
7e4c35e224 Merge branch 'develop' of github.com:ClusterCockpit/cc-metric-collector into develop 2022-02-08 13:46:48 +01:00
Thomas Roehl
8c744617be Remove logging and enable command execution for GangliaSink 2022-02-08 13:45:41 +01:00
Holger Obermaier
377f85111a Refactoring: Use ccmetric function ToLineProtocol() in stdout sink 2022-02-08 13:38:35 +01:00
Holger Obermaier
6d55c376bd Refactoring: Remove all *List() functions from CCMetric 2022-02-08 11:23:19 +01:00
Holger Obermaier
d98b678399 Refactoring: Replace FieldList() -> Fields() 2022-02-08 11:05:07 +01:00
Holger Obermaier
e1a7379c2e Generate influxDB point for data type ccMetric 2022-02-08 09:31:08 +01:00
Holger Obermaier
af051b5e7e Replace FieldList() by Fields() 2022-02-07 22:52:39 +01:00
Holger Obermaier
7b104ebe90 Use cclog.ComponentDebug. Avoid copying point.Fields() 2022-02-07 18:00:17 +01:00
Thomas Gruber
d1e66201a6
Add non-blocking InfluxDB sink (#29)
* Add non-blocking InfluxDB sink

* Add configurable batch size
2022-02-07 16:51:46 +01:00
Thomas Gruber
fdb58b0be2
Sink specific configuration maps (#25)
* Use sink-specific configurations to have more flexibility. Adjust sample sink configuration files

* Add documentation

* Add links to individual sink readmes

* Fix link in README

* HTTPS for HttpSink

* If no CPU die id available, use the socket id instead
2022-02-04 18:12:24 +01:00
Thomas Roehl
02cd21abe2 HTTPS for HttpSink 2022-02-04 12:39:25 +01:00
Thomas Gruber
6ff6cb7219
Change CCMetric's internal data structure (#22)
* package ccmetric rewrite

* Create deep copy in New() to avoid access conflicts

* Renamed TagMap() -> Tags(), MetaMap() -> Meta

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-02-01 14:54:34 +01:00
Thomas Roehl
011218ab80 Adjust ganglia sink to CCMetric 2022-01-30 15:25:57 +01:00
Thomas Gruber
d915bcc02d
Add sink to add metrics to Ganglia through gmetric (#15) 2022-01-30 15:21:24 +01:00
Holger Obermaier
9e99e47d73 Wait for close of done channel, to ensure manager finished. 2022-01-30 12:08:33 +01:00
Holger Obermaier
4e408f9490 Add documentation 2022-01-28 15:16:58 +01:00
Holger Obermaier
b9236dcc31 Handle shutdown sequentially 2022-01-27 17:43:00 +01:00
Thomas Roehl
3fd77e6887 Use non-blocking send at close, use common done function and remove default case 2022-01-26 16:54:51 +01:00
Thomas Roehl
2925ad9f40 Use ccLogger anywhere 2022-01-25 17:43:10 +01:00
Thomas Gruber
200af84c54
Modularize the whole thing (#16)
* Use channels, add a metric router, split up configuration and use extended version of Influx line protocol internally

* Use central timer for collectors and router. Add expressions to router

* Add expression to router config

* Update entry points

* Start with README

* Update README for CCMetric

* Formatting

* Update README.md

* Add README for MultiChanTicker

* Add README for MultiChanTicker

* Update README.md

* Add README to metric router

* Update main README

* Remove SinkEntity type

* Update README for sinks

* Update go files

* Update README for receivers

* Update collectors README

* Update collectors README

* Use seperate page per collector

* Fix for tempstat page

* Add docs for customcmd collector

* Add docs for ipmistat collector

* Add docs for topprocs collector

* Update customCmdMetric.md

* Use seconds when calculating LIKWID metrics

* Add IB metrics ib_recv_pkts and ib_xmit_pkts

* Drop domain part of host name

* Updated to latest stable version of likwid

* Define source code dependencies in Makefile

* Add GPFS / IBM Spectrum Scale collector

* Add vet and staticcheck make targets

* Add vet and staticcheck make targets

* Avoid go vet warning:
struct field tag `json:"..., omitempty"` not compatible with reflect.StructTag.Get: suspicious space in struct tag value
struct field tag `json:"...", omitempty` not compatible with reflect.StructTag.Get: key:"value" pairs not separated by spaces

* Add sample collector to README.md

* Add CPU frequency collector

* Avoid staticcheck warning: redundant return statement

* Avoid staticcheck warning: unnecessary assignment to the blank identifier

* Simplified code

* Add CPUFreqCollectorCpuinfo
a metric collector to measure the current frequency of the CPUs
as obtained from /proc/cpuinfo
Only measure on the first hyperthread

* Add collector for NFS clients

* Move publication of metrics into Flush() for NatsSink

* Update GitHub actions

* Refactoring

* Avoid vet warning: Println arg list ends with redundant newline

* Avoid vet warning struct field commands has json tag but is not exported

* Avoid vet warning: return copies lock value.

* Corrected typo

* Refactoring

* Add go sources in internal/...

* Bad separator in Makefile

* Fix Infiniband collector

Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com>
2022-01-25 15:37:43 +01:00
Holger Obermaier
5dd2af4e8f Avoid staticcheck warning: redundant return statement 2022-01-21 14:35:52 +01:00
Thomas Roehl
325d97df96 Updated contributing section for sinks 2021-11-26 15:56:52 +01:00
Thomas Roehl
32d5a5b98f Update sink README.md 2021-11-26 14:18:59 +01:00
Thomas Roehl
24bf9590ee Update sink README.md 2021-11-26 14:10:22 +01:00
Lou Knauer
485223c590 Add new httpSink
This sink is compatible with the HTTP API of cc-metric-store.
Example config.json section:

```
"sink": {
    "type": "http",
    "host": "localhost",
    "port": "8080",
    "database": "api/write",
    "password": "<JWT>"
},
```

The password/JWT can be omitted.
2021-10-12 13:44:38 +02:00
Lou Knauer
cdc1811576 Add Flush method to sink interface 2021-10-12 13:43:58 +02:00
Thomas Gruber
d7ef32de18
Merge branch 'main' into alternate_storage 2021-10-04 15:49:46 +02:00
Thomas Roehl
b06572c865 Reset buffer in nats sink 2021-10-04 15:46:51 +02:00