mirror of
https://github.com/ClusterCockpit/cc-metric-collector.git
synced 2025-04-16 09:55:56 +02:00
Update README for sinks
This commit is contained in:
Thomas Roehl
parent
50d24915cc
commit
7af8393626
126
sinks/README.md
126
sinks/README.md
@ -1,65 +1,99 @@
|
||||
This folder contains the sinks for the cc-metric-collector.
|
||||
# CCMetric sinks
|
||||
|
||||
# `metricSink.go`
|
||||
The base class/configuration is located in `metricSink.go`.
|
||||
This folder contains the SinkManager and sink implementations for the cc-metric-collector.
|
||||
|
||||
# Sinks
|
||||
* `stdoutSink.go`: Writes all metrics to `stdout` in InfluxDB line protocol. The sink does not use https://github.com/influxdata/line-protocol to reduce the executed code for debugging
|
||||
* `influxSink.go`: Writes all metrics to an InfluxDB database instance using a blocking writer. It uses https://github.com/influxdata/influxdb-client-go . Configuration for the server, port, ssl, password, database name and organisation are in the global configuration file. The 'password' is used for the token and the 'database' for the bucket. It uses the v2 API of Influx.
|
||||
* `natsSink.go`: Sends all metrics to an NATS server using the InfluxDB line protocol as encoding. It uses https://github.com/nats-io/nats.go . Configuration for the server, port, user, password and database name are in the global configuration file. The database name is used as subject for the NATS messages.
|
||||
* `httpSink.go`: Sends all metrics to an HTTP endpoint `http://<host>:<port>/<database>` using a POST request. The body of the request will consist of lines in the InfluxDB line protocol. In case password is specified, that password is used as a JWT in the 'Authorization' header.
|
||||
# Configuration
|
||||
|
||||
# Installation
|
||||
Nothing to do, all sinks are pure Go code
|
||||
|
||||
# Sink configuration
|
||||
The configuration file for the sinks is a list of configurations. The `type` field in each specifies which sink to initialize.
|
||||
|
||||
```json
|
||||
"sink": {
|
||||
"user": "testuser",
|
||||
"password": "testpass",
|
||||
"host": "127.0.0.1",
|
||||
"port": "9090",
|
||||
"database": "testdb",
|
||||
"organization": "testorg",
|
||||
"ssl": false
|
||||
"type": "stdout"
|
||||
[
|
||||
{
|
||||
"type" : "stdout",
|
||||
"meta_as_tags" : false
|
||||
},
|
||||
{
|
||||
"type" : "http",
|
||||
"host" : "localhost",
|
||||
"port" : "4123",
|
||||
"database" : "ccmetric",
|
||||
"password" : "<jwt token>"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## `stdout`
|
||||
When configuring `type = stdout`, all metrics are printed to stdout. No further configuration is required or touched, so you can leave your other-sink-config in there and just change the `type` for debugging purposes
|
||||
This example initializes two sinks, the `stdout` sink printing all metrics to the STDOUT and the `http` sink with the given `host`, `port`, `database` and `password`.
|
||||
|
||||
## `influxdb`
|
||||
The InfluxDB sink uses blocking write operations to write to an InfluxDB database using the v2 API. It uses the following configuration options:
|
||||
* `host`: Hostname of the database instance
|
||||
* `port`: Portnumber (as string) of the database
|
||||
* `database`: Name of the database, called 'bucket' in InfluxDB v2
|
||||
* `organization`: The InfluxDB v2 API uses organizations to separate database instances running on the same host
|
||||
* `ssl`: Boolean to activate SSL/TLS
|
||||
* `user`: Although the v2 API uses API keys instead of username and password, this field can be used if the sink should authentificate with `username:password`. If you want to use an API key, leave this field empty.
|
||||
* `password`: API key for the InfluxDB v2 API or password if `user` is set
|
||||
If `meta_as_tags` is set, all meta information attached to CCMetric are printed out as tags.
|
||||
|
||||
## `nats`
|
||||
* `host`: Hostname of the NATS server
|
||||
* `port`: Portnumber (as string) of the NATS server
|
||||
* `user`: Username for authentification in the NATS transport system
|
||||
* `password`: Password for authentification in the NATS transport system
|
||||
## Type `stdout`
|
||||
|
||||
```json
|
||||
{
|
||||
"type" : "stdout",
|
||||
"meta_as_tags" : <true|false>
|
||||
}
|
||||
```
|
||||
|
||||
The `stdout` sink dumps all metrics to the STDOUT.
|
||||
|
||||
## Type `http`
|
||||
|
||||
```json
|
||||
{
|
||||
"type" : "http",
|
||||
"host" : "<hostname>",
|
||||
"port" : "<portnumber>",
|
||||
"database" : "<database name>",
|
||||
"password" : "<jwt token>",
|
||||
"meta_as_tags" : <true|false>
|
||||
}
|
||||
```
|
||||
The sink uses POST requests to send metrics to `http://<host>:<port>/<database>` using the JWT token as a JWT in the 'Authorization' header.
|
||||
|
||||
## Type `nats`
|
||||
|
||||
```json
|
||||
{
|
||||
"type" : "nats",
|
||||
"host" : "<hostname>",
|
||||
"port" : "<portnumber>",
|
||||
"user" : "<username>",
|
||||
"password" : "<password>",
|
||||
"database" : "<database name>"
|
||||
"meta_as_tags" : <true|false>
|
||||
}
|
||||
```
|
||||
|
||||
This sink publishes the CCMetric in a NATS environment using `host`, `port`, `user` and `password` for connecting. The metrics are published using the topic `database`.
|
||||
|
||||
## Type `influxdb`
|
||||
|
||||
```json
|
||||
{
|
||||
"type" : "influxdb",
|
||||
"host" : "<hostname>",
|
||||
"port" : "<portnumber>",
|
||||
"user" : "<username>",
|
||||
"password" : "<password or API key>",
|
||||
"database" : "<database name>"
|
||||
"organization": "<InfluxDB v2 organization>",
|
||||
"ssl" : <true|false>,
|
||||
"meta_as_tags" : <true|false>
|
||||
}
|
||||
```
|
||||
|
||||
This sink submits the CCMetrics to an InfluxDB time-series database. It uses `host`, `port` and `ssl` for connecting. For authentification, it uses either `user:password` if `user` is set and only `password` as API key. The `organization` and `database` are used for writing to the correct database.
|
||||
|
||||
## `http`
|
||||
* `host`: Hostname of the HTTP server
|
||||
* `port`: Portnumber (as string) of the HTTP server
|
||||
* `database`: Endpoint to write to. HTTP POST requests are performed on `http://<host>:<port>/<database>`
|
||||
* `password`: JSON Web token used for authentification
|
||||
|
||||
|
||||
# Contributing own sinks
|
||||
A sink contains three functions and is derived from the type `Sink` (in `metricSink.go`):
|
||||
A sink contains three functions and is derived from the type `Sink`:
|
||||
* `Init(config SinkConfig) error`
|
||||
* `Write(measurement string, tags map[string]string, fields map[string]interface{}, t time.Time) error`
|
||||
* `Write(point CCMetric) error`
|
||||
* `Flush() error`
|
||||
* `Close()`
|
||||
|
||||
The data structures should be set up in `Init()` like opening a file or server connection. The `Write()` function takes a measurement, tags, fields and a timestamp and writes/sends the data. For non-blocking sinks, the `Flush()` method tells the sink to drain its internal buffers. The `Close()` function should tear down anything created in `Init()`.
|
||||
The data structures should be set up in `Init()` like opening a file or server connection. The `Write()` function writes/sends the data. For non-blocking sinks, the `Flush()` method tells the sink to drain its internal buffers. The `Close()` function should tear down anything created in `Init()`.
|
||||
|
||||
Finally, the sink needs to be registered in the `metric-collector.go`. There is a list of sinks called `Sinks` which is a map (sink_type_string -> pointer to sink). Add a new entry with a descriptive name and the new sink.
|
||||
Finally, the sink needs to be registered in the `sinkManager.go`. There is a list of sinks called `AvailableSinks` which is a map (`sink_type_string` -> `pointer to sink interface`). Add a new entry with a descriptive name and the new sink.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user