mirror of
https://github.com/ClusterCockpit/cc-metric-collector.git
synced 2024-11-10 12:37:25 +01:00
b3c27e0af5
* Cleanup: Remove unused code * Use Golang duration parser for 'interval' and 'duration' in main config * Update handling of LIKWID headers. Download only if not already present in the system. Fixes #73 * Units with cc-units (#64) * Add option to normalize units with cc-unit * Add unit conversion to router * Add option to change unit prefix in the router * Add to MetricRouter README * Add order of operations in router to README * Use second add_tags/del_tags only if metric gets renamed * Skip disks in DiskstatCollector that have size=0 * Check readability of sensor files in TempCollector * Fix for --once option * Rename `cpu` type to `hwthread` (#69) * Rename 'cpu' type to 'hwthread' to avoid naming clashes with MetricStore and CC-Webfrontend * Collectors in parallel (#74) * Provide info to CollectorManager whether the collector can be executed in parallel with others * Split serial and parallel collectors. Read in parallel first * Update NvidiaCollector with new metrics, MIG and NvLink support (#75) * CC topology module update (#76) * Rename CPU to hardware thread, write some comments * Do renaming in other parts * Remove CpuList and SocketList function from metricCollector. Available in ccTopology * Option to use MIG UUID as subtype-id in NvidiaCollector * Option to use MIG slice name as subtype-id in NvidiaCollector * MetricRouter: Fix JSON in README * Fix for Github Action to really use the selected version * Remove Ganglia installation in runonce Action and add Go 1.18 * Fix daemon options in init script * Add separate go.mod files to use it with deprecated 1.16 * Minor updates for Makefiles * fix string comparison * AMD ROCm SMI collector (#77) * Add collector for AMD ROCm SMI metrics * Fix import path * Fix imports * Remove Board Number * store GPU index explicitly * Remove board number from description * Use http instead of ftp to download likwid * Fix serial number in rocmCollector * Improved http sink (#78) * automatic flush in NatsSink * tweak default options of HttpSink * shorter cirt. section and retries for HttpSink * fix error handling * Remove file added by mistake. * Use http instead of ftp to download likwid * Fix serial number in rocmCollector Co-authored-by: Thomas Roehl <thomas.roehl@fau.de> * Fix: When sending metrics failed the batch size could be exceeded * Improved dropping of metrics failed to send * Add memstats and topprocs metric * Updated to latest modules * Check that at least one sink is running * Add drop rate, when send buffer is full * Allow only one timer at a time * Use mutex to ensure only on flush timer is running * Fix for NvidiaCollector when devices are not in MiG mode * Remove Golang version 1.16 an 1.17 from Action. Latest commits require Golang 1.18 * Use Golang 1.18 in Release action to build RPMs * Change unit of CpufreqCollector to Hz. That's what the sysfs outputs * Make wget quiet in Release action to reduce log size Co-authored-by: Holger Obermaier <40787752+ho-ob@users.noreply.github.com> Co-authored-by: Lou <lou.knauer@gmx.de>
197 lines
7.6 KiB
YAML
197 lines
7.6 KiB
YAML
# See: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions
|
|
|
|
# Workflow name
|
|
name: Release
|
|
|
|
# Run on tag push
|
|
on:
|
|
push:
|
|
tags:
|
|
- '**'
|
|
|
|
jobs:
|
|
|
|
#
|
|
# Build on AlmaLinux 8.5 using go-toolset
|
|
#
|
|
AlmaLinux-RPM-build:
|
|
runs-on: ubuntu-latest
|
|
# See: https://hub.docker.com/_/almalinux
|
|
container: almalinux:8.5
|
|
# The job outputs link to the outputs of the 'rpmrename' step
|
|
# Only job outputs can be used in child jobs
|
|
outputs:
|
|
rpm : ${{steps.rpmrename.outputs.RPM}}
|
|
srpm : ${{steps.rpmrename.outputs.SRPM}}
|
|
steps:
|
|
|
|
# Use dnf to install development packages
|
|
- name: Install development packages
|
|
run: |
|
|
dnf --assumeyes group install "Development Tools" "RPM Development Tools"
|
|
dnf --assumeyes install wget openssl-devel diffutils delve which
|
|
|
|
# Checkout git repository and submodules
|
|
# fetch-depth must be 0 to use git describe
|
|
# See: https://github.com/marketplace/actions/checkout
|
|
- name: Checkout
|
|
uses: actions/checkout@v2
|
|
with:
|
|
submodules: recursive
|
|
fetch-depth: 0
|
|
|
|
# Use dnf to install build dependencies
|
|
- name: Install build dependencies
|
|
run: |
|
|
wget -q http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-bin-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-src-1.18.2-1.module_el8.7.0+1173+5d37c0fd.noarch.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/go-toolset-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm
|
|
rpm -i go*.rpm
|
|
|
|
- name: RPM build MetricCollector
|
|
id: rpmbuild
|
|
run: make RPM
|
|
|
|
# AlmaLinux 8.5 is a derivate of RedHat Enterprise Linux 8 (UBI8),
|
|
# so the created RPM both contain the substring 'el8' in the RPM file names
|
|
# This step replaces the substring 'el8' to 'alma85'. It uses the move operation
|
|
# because it is unclear whether the default AlmaLinux 8.5 container contains the
|
|
# 'rename' command. This way we also get the new names for output.
|
|
- name: Rename RPMs (s/el8/alma85/)
|
|
id: rpmrename
|
|
run: |
|
|
OLD_RPM="${{steps.rpmbuild.outputs.RPM}}"
|
|
OLD_SRPM="${{steps.rpmbuild.outputs.SRPM}}"
|
|
NEW_RPM="${OLD_RPM/el8/alma85}"
|
|
NEW_SRPM=${OLD_SRPM/el8/alma85}
|
|
mv "${OLD_RPM}" "${NEW_RPM}"
|
|
mv "${OLD_SRPM}" "${NEW_SRPM}"
|
|
echo "::set-output name=SRPM::${NEW_SRPM}"
|
|
echo "::set-output name=RPM::${NEW_RPM}"
|
|
|
|
# See: https://github.com/actions/upload-artifact
|
|
- name: Save RPM as artifact
|
|
uses: actions/upload-artifact@v2
|
|
with:
|
|
name: cc-metric-collector RPM for AlmaLinux 8.5
|
|
path: ${{ steps.rpmrename.outputs.RPM }}
|
|
- name: Save SRPM as artifact
|
|
uses: actions/upload-artifact@v2
|
|
with:
|
|
name: cc-metric-collector SRPM for AlmaLinux 8.5
|
|
path: ${{ steps.rpmrename.outputs.SRPM }}
|
|
|
|
#
|
|
# Build on UBI 8 using go-toolset
|
|
#
|
|
UBI-8-RPM-build:
|
|
runs-on: ubuntu-latest
|
|
# See: https://catalog.redhat.com/software/containers/ubi8/ubi/5c359854d70cc534b3a3784e?container-tabs=gti
|
|
container: registry.access.redhat.com/ubi8/ubi:8.5-226.1645809065
|
|
# The job outputs link to the outputs of the 'rpmbuild' step
|
|
outputs:
|
|
rpm : ${{steps.rpmbuild.outputs.RPM}}
|
|
srpm : ${{steps.rpmbuild.outputs.SRPM}}
|
|
steps:
|
|
|
|
# Use dnf to install development packages
|
|
- name: Install development packages
|
|
run: dnf --assumeyes --disableplugin=subscription-manager install rpm-build go-srpm-macros rpm-build-libs rpm-libs gcc make python38 git wget openssl-devel diffutils delve which
|
|
|
|
# Checkout git repository and submodules
|
|
# fetch-depth must be 0 to use git describe
|
|
# See: https://github.com/marketplace/actions/checkout
|
|
- name: Checkout
|
|
uses: actions/checkout@v2
|
|
with:
|
|
submodules: recursive
|
|
fetch-depth: 0
|
|
|
|
# Use dnf to install build dependencies
|
|
- name: Install build dependencies
|
|
run: |
|
|
wget -q http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-bin-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/golang-src-1.18.2-1.module_el8.7.0+1173+5d37c0fd.noarch.rpm \
|
|
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/go-toolset-1.18.2-1.module_el8.7.0+1173+5d37c0fd.x86_64.rpm
|
|
rpm -i go*.rpm
|
|
|
|
- name: RPM build MetricCollector
|
|
id: rpmbuild
|
|
run: make RPM
|
|
|
|
# See: https://github.com/actions/upload-artifact
|
|
- name: Save RPM as artifact
|
|
uses: actions/upload-artifact@v2
|
|
with:
|
|
name: cc-metric-collector RPM for UBI 8
|
|
path: ${{ steps.rpmbuild.outputs.RPM }}
|
|
- name: Save SRPM as artifact
|
|
uses: actions/upload-artifact@v2
|
|
with:
|
|
name: cc-metric-collector SRPM for UBI 8
|
|
path: ${{ steps.rpmbuild.outputs.SRPM }}
|
|
|
|
#
|
|
# Create release with fresh RPMs
|
|
#
|
|
Release:
|
|
runs-on: ubuntu-latest
|
|
# We need the RPMs, so add dependency
|
|
needs: [AlmaLinux-RPM-build, UBI-8-RPM-build]
|
|
|
|
steps:
|
|
# See: https://github.com/actions/download-artifact
|
|
- name: Download AlmaLinux 8.5 RPM
|
|
uses: actions/download-artifact@v2
|
|
with:
|
|
name: cc-metric-collector RPM for AlmaLinux 8.5
|
|
- name: Download AlmaLinux 8.5 SRPM
|
|
uses: actions/download-artifact@v2
|
|
with:
|
|
name: cc-metric-collector SRPM for AlmaLinux 8.5
|
|
|
|
- name: Download UBI 8 RPM
|
|
uses: actions/download-artifact@v2
|
|
with:
|
|
name: cc-metric-collector RPM for UBI 8
|
|
- name: Download UBI 8 SRPM
|
|
uses: actions/download-artifact@v2
|
|
with:
|
|
name: cc-metric-collector SRPM for UBI 8
|
|
|
|
# The download actions do not publish the name of the downloaded file,
|
|
# so we re-use the job outputs of the parent jobs. The files are all
|
|
# downloaded to the current folder.
|
|
# The gh-release action afterwards does not accept file lists but all
|
|
# files have to be listed at 'files'. The step creates one output per
|
|
# RPM package (2 per distro)
|
|
- name: Set RPM variables
|
|
id: files
|
|
run: |
|
|
ALMA_85_RPM=$(basename "${{ needs.AlmaLinux-RPM-build.outputs.rpm}}")
|
|
ALMA_85_SRPM=$(basename "${{ needs.AlmaLinux-RPM-build.outputs.srpm}}")
|
|
UBI_8_RPM=$(basename "${{ needs.UBI-8-RPM-build.outputs.rpm}}")
|
|
UBI_8_SRPM=$(basename "${{ needs.UBI-8-RPM-build.outputs.srpm}}")
|
|
echo "ALMA_85_RPM::${ALMA_85_RPM}"
|
|
echo "ALMA_85_SRPM::${ALMA_85_SRPM}"
|
|
echo "UBI_8_RPM::${UBI_8_RPM}"
|
|
echo "UBI_8_SRPM::${UBI_8_SRPM}"
|
|
echo "::set-output name=ALMA_85_RPM::${ALMA_85_RPM}"
|
|
echo "::set-output name=ALMA_85_SRPM::${ALMA_85_SRPM}"
|
|
echo "::set-output name=UBI_8_RPM::${UBI_8_RPM}"
|
|
echo "::set-output name=UBI_8_SRPM::${UBI_8_SRPM}"
|
|
|
|
# See: https://github.com/softprops/action-gh-release
|
|
- name: Release
|
|
uses: softprops/action-gh-release@v1
|
|
if: startsWith(github.ref, 'refs/tags/')
|
|
with:
|
|
name: cc-metric-collector-${{github.ref_name}}
|
|
files: |
|
|
${{ steps.files.outputs.ALMA_85_RPM }}
|
|
${{ steps.files.outputs.ALMA_85_SRPM }}
|
|
${{ steps.files.outputs.UBI_8_RPM }}
|
|
${{ steps.files.outputs.UBI_8_SRPM }}
|