A node agent for measuring, processing and forwarding node level metrics
Go to file
2021-05-14 19:21:16 +02:00
collectors Return error at Init for skip collectors early 2021-05-14 19:21:16 +02:00
sinks Enable strings as field values in stdout sink 2021-05-12 17:46:16 +02:00
.gitignore Initial commit 2021-02-16 16:24:11 +01:00
config.json Add collector for the top5 processes in %cpu 2021-05-12 17:47:12 +02:00
go.mod Add sink for InfluxDB (with the original InfluxDB client) 2021-03-26 16:48:09 +01:00
go.sum Add sink for InfluxDB (with the original InfluxDB client) 2021-03-26 16:48:09 +01:00
LICENSE Initial commit 2021-02-16 16:24:11 +01:00
metric-collector.go Add collector for the top5 processes in %cpu 2021-05-12 17:47:12 +02:00
README.md Add section about installation and running 2021-05-12 14:46:50 +02:00

cc-metric-collector

A node agent for measuring, processing and forwarding node level metrics.

Open questions:

  • Are hostname unique with a computing center or is it required to store the cluster name in addition to the hostname?
  • What about memory domain granularity?

Configuration

Configuration is implemented using a single json document that is distributed over network and may be persisted as file. Supported metrics are documented here.

{
    "sink": {
        "user": "admin",
        "password": "12345",
        "host": "localhost",
        "port": "8080",
        "database": "testdb",
        "type": "stdout"
    },
    "interval" : 3,
    "duration" : 1,
    "collectors": [
        "memstat",
        "likwid",
        "loadavg",
        "netstat",
        "ibstat",
        "lustrestat"
    ]
}

All available collectors are listed in the above JSON. There are currently three sinks supported influxdb, nats and stdout. The interval defines how often the metrics should be read and send to the sink. The duration tells collectors how long one measurement has to take. An example for this is the likwid collector which starts the hardware performance counter, waits for duration seconds and stops the counters again. For most systems, the likwid collector has to do two measurements, thus interval must be larger than two times duration.

Installation

$ git clone git@github.com:ClusterCockpit/cc-metric-collector.git
$ cd cc-metric-collector/collectors
$ edit Makefile (for LIKWID collector)
$ make (downloads LIKWID, builds it as static library and copies all required files for the collector)
$ cd ..
$ go get (requires at least golang 1.13)
$ go build metric-collector

Running

$ ./metric-collector --help
Usage of metric-collector:
  -config string
    	Path to configuration file (default "./config.json")
  -log string
    	Path for logfile (default "stderr")