mirror of
https://github.com/ClusterCockpit/cc-metric-collector.git
synced 2024-12-25 15:09:05 +01:00
94b086acf0
* InfiniBandCollector: Scale raw readings from octets to bytes * Fix clock frequency coming from LikwidCollector and update docs * Build DEB package for Ubuntu 20.04 for releases * Fix memstat collector with numa_stats option * Remove useless prints from MemstatCollector * Replace ioutils with os and io (#87) * Use lower case for error strings in RocmSmiCollector * move maybe-usable-by-other-cc-components to pkg. Fix all files to use the new paths (#88) * Add collector for monitoring the execution of cc-metric-collector itself (#81) * Add collector to monitor execution of cc-metric-collector itself * Register SelfCollector * Fix import paths for moved packages * Check if at least one CPU with frequency information was detected * Correct type: /proc/stats -> /proc/stat * Update README.md * Run ipmitool asynchron. Improved error handling. * Corrected some typos * Add running average power limit (RAPL) metric collector * Add running average power limit (RAPL) metric collector * Do not mess up with the orignal configuration * * Corrected json config in numastatsMetric.md * Added some debug output to numastatsMetric.go * Fixed computing number of physical packages for non continous physical package IDs (e.g. on Ampere Altra Q80-30) * Fix kernel panic for receiver config with missing receiver type * Add receiver to gather remote IPMI sensor metrics * Added config option to add ipmi-sensors command line options * Add documentaion for IPMI receiver * Update to latest version of included go modules * Add go.mod to App dependency * Try to use common metric tags across hardware vendors * Add IPMI metric: current * remove prefix enumeration like 01-... * Add IPMI receiver example configuration to receivers.json * Minimal formating changes * Add hostlist package * Added tests for hostlist Expand() * Use package hostlist to expand a host list * Use package hostlist to expand a host list * Some servers return "ConsumedPowerWatt":65535 instead of "ConsumedPowerWatt":null * Updated to latest package versions * Do not allow unknown fields in JSON configuration file * Add workflow to customize packages to docs * NFS I/O Stats Collector (#91) * Initial version * Delete values for vanished mount points and comments * Fix for Likwid collector (#95) * Run LIKWID in separate thread and check metric type * Change LIKWID collector documentation to use 'type' instead of 'scope' * Re-initialize LIKWID after one read is missing due to lock toggle * Register cc-metric-collector at Zenodo (#93) * Add initial version of Zenodo project file * Orcid ID added * Update .zenodo.json Co-authored-by: Holger Obermaier <holger.obermaier@kit.edu> * Update ipmiMetric.go * Use latest LIKWID version for builds * Update README.md * Remove development stuff from Makefile * Add Requires(pre) to RPM SPEC file * Use curly brackets in packaging make targets * Fix for LIKWID collector with separate measurement thread and inotify watcher on the LIKWID lock (#97) * Debian does not like underscores in the version * Update cc-metric-collector.service Remove dependency services not used by cc-metric-collector * Add new requirements to module file * Use customcmd commands if they did not error. (#101) * Merge develop and main (#99) * InfiniBandCollector: Scale raw readings from octets to bytes * Fix clock frequency coming from LikwidCollector and update docs * Build DEB package for Ubuntu 20.04 for releases * Fix memstat collector with numa_stats option * Remove useless prints from MemstatCollector * Replace ioutils with os and io (#87) * Use lower case for error strings in RocmSmiCollector * move maybe-usable-by-other-cc-components to pkg. Fix all files to use the new paths (#88) * Add collector for monitoring the execution of cc-metric-collector itself (#81) * Add collector to monitor execution of cc-metric-collector itself * Register SelfCollector * Fix import paths for moved packages * Check if at least one CPU with frequency information was detected * Correct type: /proc/stats -> /proc/stat * Update README.md * Run ipmitool asynchron. Improved error handling. * Corrected some typos * Add running average power limit (RAPL) metric collector * Add running average power limit (RAPL) metric collector * Do not mess up with the orignal configuration * * Corrected json config in numastatsMetric.md * Added some debug output to numastatsMetric.go * Fixed computing number of physical packages for non continous physical package IDs (e.g. on Ampere Altra Q80-30) * Fix kernel panic for receiver config with missing receiver type * Add receiver to gather remote IPMI sensor metrics * Added config option to add ipmi-sensors command line options * Add documentaion for IPMI receiver * Update to latest version of included go modules * Add go.mod to App dependency * Try to use common metric tags across hardware vendors * Add IPMI metric: current * remove prefix enumeration like 01-... * Add IPMI receiver example configuration to receivers.json * Minimal formating changes * Add hostlist package * Added tests for hostlist Expand() * Use package hostlist to expand a host list * Use package hostlist to expand a host list * Some servers return "ConsumedPowerWatt":65535 instead of "ConsumedPowerWatt":null * Updated to latest package versions * Do not allow unknown fields in JSON configuration file * Add workflow to customize packages to docs * NFS I/O Stats Collector (#91) * Initial version * Delete values for vanished mount points and comments * Fix for Likwid collector (#95) * Run LIKWID in separate thread and check metric type * Change LIKWID collector documentation to use 'type' instead of 'scope' * Re-initialize LIKWID after one read is missing due to lock toggle * Register cc-metric-collector at Zenodo (#93) * Add initial version of Zenodo project file * Orcid ID added * Update .zenodo.json Co-authored-by: Holger Obermaier <holger.obermaier@kit.edu> * Update ipmiMetric.go * Use latest LIKWID version for builds * Update README.md * Remove development stuff from Makefile * Add Requires(pre) to RPM SPEC file * Use curly brackets in packaging make targets * Fix for LIKWID collector with separate measurement thread and inotify watcher on the LIKWID lock (#97) Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> * Update likwid_perfgroup_to_cc_config.py * Use customcmd commands if they did not error. --------- Co-authored-by: Thomas Gruber <Thomas.Roehl@googlemail.com> Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> --------- Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> Co-authored-by: Holger Obermaier <Holger.Obermaier@kit.edu> Co-authored-by: fodinabor <5982050+fodinabor@users.noreply.github.com>
130 lines
5.2 KiB
Makefile
130 lines
5.2 KiB
Makefile
APP = cc-metric-collector
|
|
GOSRC_APP := cc-metric-collector.go
|
|
GOSRC_COLLECTORS := $(wildcard collectors/*.go)
|
|
GOSRC_SINKS := $(wildcard sinks/*.go)
|
|
GOSRC_RECEIVERS := $(wildcard receivers/*.go)
|
|
GOSRC_INTERNAL := $(wildcard internal/*/*.go)
|
|
GOSRC := $(GOSRC_APP) $(GOSRC_COLLECTORS) $(GOSRC_SINKS) $(GOSRC_RECEIVERS) $(GOSRC_INTERNAL)
|
|
COMPONENT_DIRS := collectors \
|
|
sinks \
|
|
receivers \
|
|
internal/metricRouter \
|
|
internal/ccMetric \
|
|
internal/metricAggregator \
|
|
internal/ccLogger \
|
|
internal/ccTopology \
|
|
internal/multiChanTicker
|
|
|
|
BINDIR = bin
|
|
GOBIN = $(shell which go)
|
|
|
|
|
|
.PHONY: all
|
|
all: $(APP)
|
|
|
|
$(APP): $(GOSRC) go.mod
|
|
make -C collectors
|
|
$(GOBIN) get
|
|
$(GOBIN) build -o $(APP) $(GOSRC_APP)
|
|
|
|
install: $(APP)
|
|
@WORKSPACE=$(PREFIX)
|
|
@if [ -z "$${WORKSPACE}" ]; then exit 1; fi
|
|
@mkdir --parents --verbose $${WORKSPACE}/usr/$(BINDIR)
|
|
@install -Dpm 755 $(APP) $${WORKSPACE}/usr/$(BINDIR)/$(APP)
|
|
@mkdir --parents --verbose $${WORKSPACE}/etc/cc-metric-collector $${WORKSPACE}/etc/default $${WORKSPACE}/etc/systemd/system $${WORKSPACE}/etc/init.d
|
|
@install -Dpm 600 config.json $${WORKSPACE}/etc/cc-metric-collector/cc-metric-collector.json
|
|
@sed -i -e s+"\"./"+"\"/etc/cc-metric-collector/"+g $${WORKSPACE}/etc/cc-metric-collector/cc-metric-collector.json
|
|
@install -Dpm 600 sinks.json $${WORKSPACE}/etc/cc-metric-collector/sinks.json
|
|
@install -Dpm 600 collectors.json $${WORKSPACE}/etc/cc-metric-collector/collectors.json
|
|
@install -Dpm 600 router.json $${WORKSPACE}/etc/cc-metric-collector/router.json
|
|
@install -Dpm 600 receivers.json $${WORKSPACE}/etc/cc-metric-collector/receivers.json
|
|
@install -Dpm 600 scripts/cc-metric-collector.config $${WORKSPACE}/etc/default/cc-metric-collector
|
|
@install -Dpm 644 scripts/cc-metric-collector.service $${WORKSPACE}/etc/systemd/system/cc-metric-collector.service
|
|
@install -Dpm 644 scripts/cc-metric-collector.init $${WORKSPACE}/etc/init.d/cc-metric-collector
|
|
|
|
|
|
.PHONY: clean
|
|
.ONESHELL:
|
|
clean:
|
|
@for COMP in $(COMPONENT_DIRS); do if [ -e $$COMP/Makefile ]; then make -C $$COMP clean; fi; done
|
|
rm -f $(APP)
|
|
|
|
.PHONY: fmt
|
|
fmt:
|
|
$(GOBIN) fmt $(GOSRC_COLLECTORS)
|
|
$(GOBIN) fmt $(GOSRC_SINKS)
|
|
$(GOBIN) fmt $(GOSRC_RECEIVERS)
|
|
$(GOBIN) fmt $(GOSRC_APP)
|
|
@for F in $(GOSRC_INTERNAL); do $(GOBIN) fmt $$F; done
|
|
|
|
|
|
# Examine Go source code and reports suspicious constructs
|
|
.PHONY: vet
|
|
vet:
|
|
$(GOBIN) vet ./...
|
|
|
|
|
|
# Run linter for the Go programming language.
|
|
# Using static analysis, it finds bugs and performance issues, offers simplifications, and enforces style rules
|
|
.PHONY: staticcheck
|
|
staticcheck:
|
|
$(GOBIN) install honnef.co/go/tools/cmd/staticcheck@latest
|
|
$$($(GOBIN) env GOPATH)/bin/staticcheck ./...
|
|
|
|
.ONESHELL:
|
|
.PHONY: RPM
|
|
RPM: scripts/cc-metric-collector.spec
|
|
@WORKSPACE="$${PWD}"
|
|
@SPECFILE="$${WORKSPACE}/scripts/cc-metric-collector.spec"
|
|
# Setup RPM build tree
|
|
@eval $$(rpm --eval "ARCH='%{_arch}' RPMDIR='%{_rpmdir}' SOURCEDIR='%{_sourcedir}' SPECDIR='%{_specdir}' SRPMDIR='%{_srcrpmdir}' BUILDDIR='%{_builddir}'")
|
|
@mkdir --parents --verbose "$${RPMDIR}" "$${SOURCEDIR}" "$${SPECDIR}" "$${SRPMDIR}" "$${BUILDDIR}"
|
|
# Create source tarball
|
|
@COMMITISH="HEAD"
|
|
@VERS=$$(git describe --tags $${COMMITISH})
|
|
@VERS=$${VERS#v}
|
|
@VERS=$$(echo $${VERS} | sed -e s+'-'+'_'+g)
|
|
@eval $$(rpmspec --query --queryformat "NAME='%{name}' VERSION='%{version}' RELEASE='%{release}' NVR='%{NVR}' NVRA='%{NVRA}'" --define="VERS $${VERS}" "$${SPECFILE}")
|
|
@PREFIX="$${NAME}-$${VERSION}"
|
|
@FORMAT="tar.gz"
|
|
@SRCFILE="$${SOURCEDIR}/$${PREFIX}.$${FORMAT}"
|
|
@git archive --verbose --format "$${FORMAT}" --prefix="$${PREFIX}/" --output="$${SRCFILE}" $${COMMITISH}
|
|
# Build RPM and SRPM
|
|
@rpmbuild -ba --define="VERS $${VERS}" --rmsource --clean "$${SPECFILE}"
|
|
# Report RPMs and SRPMs when in GitHub Workflow
|
|
@if [[ "$${GITHUB_ACTIONS}" == true ]]; then
|
|
@ RPMFILE="$${RPMDIR}/$${ARCH}/$${NVRA}.rpm"
|
|
@ SRPMFILE="$${SRPMDIR}/$${NVR}.src.rpm"
|
|
@ echo "SRPM=$${SRPMFILE}" >> $${GITHUB_OUTPUT}
|
|
@ echo "RPM=$${RPMFILE}" >> $${GITHUB_OUTPUT}
|
|
@fi
|
|
|
|
.PHONY: DEB
|
|
DEB: scripts/cc-metric-collector.deb.control $(APP)
|
|
@BASEDIR=$${PWD}
|
|
@WORKSPACE=$${PWD}/.dpkgbuild
|
|
@DEBIANDIR=$${WORKSPACE}/debian
|
|
@DEBIANBINDIR=$${WORKSPACE}/DEBIAN
|
|
@mkdir --parents --verbose $${WORKSPACE} $${DEBIANBINDIR}
|
|
#@mkdir --parents --verbose $$DEBIANDIR
|
|
@CONTROLFILE="$${BASEDIR}/scripts/cc-metric-collector.deb.control"
|
|
@COMMITISH="HEAD"
|
|
@VERS=$$(git describe --tags --abbrev=0 $${COMMITISH})
|
|
@if [ -z "$${VERS}" ]; then VERS=${GITHUB_REF_NAME}; fi
|
|
@VERS=$${VERS#v}
|
|
@ARCH=$$(uname -m)
|
|
@ARCH=$$(echo $${ARCH} | sed -e s+'_'+'-'+g)
|
|
@if [ "$${ARCH}" = "x86-64" ]; then ARCH=amd64; fi
|
|
@PREFIX="$${NAME}-$${VERSION}_$${ARCH}"
|
|
@SIZE_BYTES=$$(du -bcs --exclude=.dpkgbuild "$${WORKSPACE}"/ | awk '{print $$1}' | head -1 | sed -e 's/^0\+//')
|
|
@SIZE="$$(awk -v size="$${SIZE_BYTES}" 'BEGIN {print (size/1024)+1}' | awk '{print int($$0)}')"
|
|
@sed -e s+"{VERSION}"+"$${VERS}"+g -e s+"{INSTALLED_SIZE}"+"$${SIZE}"+g -e s+"{ARCH}"+"$${ARCH}"+g $${CONTROLFILE} > $${DEBIANBINDIR}/control
|
|
@make PREFIX=$${WORKSPACE} install
|
|
@DEB_FILE="cc-metric-collector_$${VERS}_$${ARCH}.deb"
|
|
@dpkg-deb -b $${WORKSPACE} "$${DEB_FILE}"
|
|
@if [ "$${GITHUB_ACTIONS}" = "true" ]; then
|
|
@ echo "DEB=$${DEB_FILE}" >> $${GITHUB_OUTPUT}
|
|
@fi
|
|
@rm -r "$${WORKSPACE}"
|