cc-metric-collector/sinks
Thomas Gruber e28c1fb30b
Ganglia sink using libganglia.so directly (#35)
* Add sink directly using libganglia.so

* Remove unneeded confuse header

* add submodule init to build action

* add submodule init to runonce action

* add installation og ganglia to runonce

* add installation of ganglia to runonce

* add installation of ganglia to runonce

* libconfuse not required

* Remove ganglia submodule

* Remove ganglia.h

* Add Makefile to help creating the libganglia.so link

* Fix cgo header

* Rename new Ganglia sink to 'libgangliaSink'

* Add documentation for libgangliaSink

* Extend make buildsystem with find&symlink helper for libgangliaSink

* Add metric renaming function

* Add build tag 'ganglia' and create corresponding files
2022-02-16 18:33:46 +01:00
..
gangliaSink_disabled.go Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
gangliaSink.go Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
gangliaSink.md Sink specific configuration maps (#25) 2022-02-04 18:12:24 +01:00
httpSink.go Automatically flush batched writes in the HTTP sink (#31) 2022-02-10 13:12:32 +01:00
httpSink.md Automatically flush batched writes in the HTTP sink (#31) 2022-02-10 13:12:32 +01:00
influxAsyncSink.go Add config option flush_interval 2022-02-10 09:43:02 +01:00
influxAsyncSink.md Add non-blocking InfluxDB sink (#29) 2022-02-07 16:51:46 +01:00
influxSink.go Add comments 2022-02-09 11:08:50 +01:00
influxSink.md Add non-blocking InfluxDB sink (#29) 2022-02-07 16:51:46 +01:00
libgangliaSink_disabled.go Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
libgangliaSink.go Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
libgangliaSink.md Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
Makefile Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
metricSink.go Sink specific configuration maps (#25) 2022-02-04 18:12:24 +01:00
natsSink.go Refactoring: Remove all *List() functions from CCMetric 2022-02-08 11:23:19 +01:00
natsSink.md Sink specific configuration maps (#25) 2022-02-04 18:12:24 +01:00
README.md Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
sinkManager.go Ganglia sink using libganglia.so directly (#35) 2022-02-16 18:33:46 +01:00
stdoutSink.go Refactoring: Use ccmetric function ToLineProtocol() in stdout sink 2022-02-08 13:38:35 +01:00
stdoutSink.md Sink specific configuration maps (#25) 2022-02-04 18:12:24 +01:00

CCMetric sinks

This folder contains the SinkManager and sink implementations for the cc-metric-collector.

Available sinks:

Configuration

The configuration file for the sinks is a list of configurations. The type field in each specifies which sink to initialize.

[
  "mystdout" : {
    "type" : "stdout",
    "meta_as_tags" : false
  },
  "metricstore" : {
    "type" : "http",
    "host" : "localhost",
    "port" : "4123",
    "database" : "ccmetric",
    "password" : "<jwt token>"
  }
]

Contributing own sinks

A sink contains four functions and is derived from the type sink:

  • Init(config json.RawMessage) error
  • Write(point CCMetric) error
  • Flush() error
  • Close()

The data structures should be set up in Init() like opening a file or server connection. The Write() function writes/sends the data. For non-blocking sinks, the Flush() method tells the sink to drain its internal buffers. The Close() function should tear down anything created in Init().

Finally, the sink needs to be registered in the sinkManager.go. There is a list of sinks called AvailableSinks which is a map (sink_type_string -> pointer to sink interface). Add a new entry with a descriptive name and the new sink.

Sample sink

package sinks

import (
	"encoding/json"
	"log"
	lp "github.com/ClusterCockpit/cc-metric-collector/internal/ccMetric"
)

type SampleSinkConfig struct {
	defaultSinkConfig  // defines JSON tags for 'name' and 'meta_as_tags'
}

type SampleSink struct {
	sink              // declarate 'name' and 'meta_as_tags'
	config StdoutSinkConfig // entry point to the SampleSinkConfig
}

// Initialize the sink by giving it a name and reading in the config JSON
func (s *SampleSink) Init(config json.RawMessage) error {
	s.name = "SampleSink"   // Always specify a name here
  // Read in the config JSON
	if len(config) > 0 {
		err := json.Unmarshal(config, &s.config)
		if err != nil {
			return err
		}
	}
	return nil
}

// Code to submit a single CCMetric to the sink
func (s *SampleSink) Write(point lp.CCMetric) error {
	log.Print(point)
	return nil
}

// If the sink uses batched sends internally, you can tell to flush its buffers
func (s *SampleSink) Flush() error {
	return nil
}


// Close sink: close network connection, close files, close libraries, ...
func (s *SampleSink) Close() {}