Update ReleaseNotes with the recent changes. Also include changes in the external metricstore package.


     1→# `cc-metric-store` version 1.5.3
     2→
     3→This is a major release of `cc-metric-store`, the metric timeseries cache
     4→implementation of ClusterCockpit. Since the storage engine is now part of
     5→`cc-backend` we will follow the version number of `cc-backend`.
     6→For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).
     7→
     8→## Breaking changes
     9→
    10→- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
    11→  removed. The storage engine is now provided by the
    12→  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
    13→  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
    14→  only.
    15→- The configuration schema has changed. Refer to `configs/config.json` for the
    16→  updated structure.
    17→
    18→## Notable changes
    19→
    20→- **Storage engine extracted to `cc-backend` library**: The entire in-memory
    21→  time-series storage engine was moved to `cc-backend/pkg/metricstore`. This
    22→  reduces duplication in the ClusterCockpit suite and enables shared maintenance
    23→  of the storage layer.
    24→- **HealthCheck API endpoint**: New `GET /api/healthcheck/` endpoint reports the
    25→  health status of cluster nodes.
    26→- **Dynamic memory management**: Memory limits can now be adjusted at runtime via
    27→  a callback from the `cc-backend` library.
    28→- **Configuration schema validation**: The config and metric config JSON schemas
    29→  have been updated and are now validated against the structs they describe.
    30→- **Startup refactored**: Application startup has been split into `cli.go` and
    31→  `server.go` for clearer separation of concerns.
    32→- **`go fix` applied**: Codebase updated to current Go idioms.
    33→- **Dependency upgrades**: `nats.go` bumped from 1.36.0 to 1.47.0;
    34→  `cc-lib` updated to v2.8.0; `cc-backend` updated to v1.5.0; various other
    35→  module upgrades.
    36→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# `cc-metric-store` version 1.5.3

This is a major release of `cc-metric-store`, the metric timeseries cache
implementation of ClusterCockpit. Since the storage engine is now part of
`cc-backend` we will follow the version number of `cc-backend`.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).

## Breaking changes

- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
  removed. The storage engine is now provided by the
  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
  only.
- The configuration schema has changed. Refer to `configs/config.json` for the
  updated structure.

## Notable changes

- **Storage engine extracted to `cc-backend` library**: The entire in-memory
  time-series storage engine was moved to `cc-backend/pkg/metricstore`. This
  reduces duplication in the ClusterCockpit suite and enables shared maintenance
  of the storage layer.
- **HealthCheck API endpoint**: New `GET /api/healthcheck/` endpoint reports the
  health status of cluster nodes.
- **Dynamic memory management**: Memory limits can now be adjusted at runtime via
  a callback from the `cc-backend` library.
- **Configuration schema validation**: The config and metric config JSON schemas
  have been updated and are now validated against the structs they describe.
- **Startup refactored**: Application startup has been split into `cli.go` and
  `server.go` for clearer separation of concerns.
- **`go fix` applied**: Codebase updated to current Go idioms.
- **Dependency upgrades**: `nats.go` bumped from 1.36.0 to 1.47.0;
  `cc-lib` updated to v2.8.0; `cc-backend` updated to v1.5.0; various other
  module upgrades.


Bash

Recent commit history


a116a1a Update dependencies. Synchronize options, cli flags with cc-backend
41fb959 Cleanup README
3e0805b Fix golreaser config
bdf65b1 Upgrade dependencies. Prepare 1.5.0  release
b19dcdb Upgrade dependencies
d5c9ae9 Merge branch 'main' into dev
b06bb58 Add entire integration
542f837 Update README
0fa5d8b Run go fix tool. Refactor
5547580 Update to lates cc-backend head
3cc9420 Merge branch 'dev' of github.com:ClusterCockpit/cc-metric-store into dev
b764120 Integrate entire. Upgrade dependencies
4de275a fix error conditional
ef3504d Update to the error handling
37a8706 Bump
fc4ae14 Bringing back GOGC for syncPool
be6d4be Fix healthcheck api
7d6455d Merge pull request #55 from ClusterCockpit/dependabot/go_modules/github.com/google/gops-0.3.29
fd4ec39 Bump github.com/google/gops from 0.3.28 to 0.3.29
dd23f49 Add configuration options and enable dynamic memory management through cc-backend callback


git show a116a1a --stat && git diff bdf65b1..HEAD -- go.mod

Show changes in latest commit and dependency changes


commit a116a1ab16df6e2007173bfe6d77376a73ce329a
Author: Jan Eitzinger <jan@moebiusband.org>
Date:   Wed Apr 1 15:37:33 2026 +0200

    Update dependencies. Synchronize options, cli flags with cc-backend

    Entire-Checkpoint: e24929f05ad3

 CLAUDE.md                   |  2 +-
 README.md                   | 20 ++++++-------
 cmd/cc-metric-store/main.go | 58 +++++++++++++++++++++++++++++-------
 configs/config.json         |  4 +--
 go.mod                      | 66 ++++++++++++++++++++---------------------
 go.sum                      | 71 +++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 163 insertions(+), 58 deletions(-)
diff --git a/go.mod b/go.mod
index 0c4e436..6584a8f 100644
--- a/go.mod
+++ b/go.mod
@@ -3,8 +3,8 @@ module github.com/ClusterCockpit/cc-metric-store
 go 1.25.0

 require (
-	github.com/ClusterCockpit/cc-backend v1.5.0
-	github.com/ClusterCockpit/cc-lib/v2 v2.8.0
+	github.com/ClusterCockpit/cc-backend v1.5.3
+	github.com/ClusterCockpit/cc-lib/v2 v2.11.0
 	github.com/ClusterCockpit/cc-line-protocol/v2 v2.4.0
 	github.com/golang-jwt/jwt/v4 v4.5.2
 	github.com/google/gops v0.3.29
@@ -15,25 +15,25 @@ require (

 require (
 	github.com/KyleBanks/depth v1.2.1 // indirect
-	github.com/andybalholm/brotli v1.2.0 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.41.3 // indirect
-	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.6 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.32.11 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.19.11 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.19 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.19 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.19 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.5 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.20 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.6 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.11 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.19 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.19 // indirect
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.96.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/signin v1.0.7 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.30.12 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.16 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.41.8 // indirect
+	github.com/andybalholm/brotli v1.2.1 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.41.5 // indirect
+	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.32.13 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.19.13 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.21 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 // indirect
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.98.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.0.9 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.30.14 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.18 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.41.10 // indirect
 	github.com/aws/smithy-go v1.24.2 // indirect
 	github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
 	github.com/fsnotify/fsnotify v1.9.0 // indirect
@@ -48,28 +48,28 @@ require (
 	github.com/go-openapi/swag/typeutils v0.25.5 // indirect
 	github.com/go-openapi/swag/yamlutils v0.25.5 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/klauspost/compress v1.18.4 // indirect
-	github.com/mattn/go-sqlite3 v1.14.34 // indirect
-	github.com/nats-io/nats.go v1.49.0 // indirect
+	github.com/klauspost/compress v1.18.5 // indirect
+	github.com/mattn/go-sqlite3 v1.14.38 // indirect
+	github.com/nats-io/nats.go v1.50.0 // indirect
 	github.com/nats-io/nkeys v0.4.15 // indirect
 	github.com/nats-io/nuid v1.0.1 // indirect
 	github.com/parquet-go/bitpack v1.0.0 // indirect
-	github.com/parquet-go/jsonlite v1.4.0 // indirect
-	github.com/parquet-go/parquet-go v0.28.0 // indirect
+	github.com/parquet-go/jsonlite v1.5.0 // indirect
+	github.com/parquet-go/parquet-go v0.29.0 // indirect
 	github.com/pierrec/lz4/v4 v4.1.26 // indirect
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/swaggo/files/v2 v2.0.2 // indirect
 	github.com/twpayne/go-geom v1.6.1 // indirect
 	github.com/urfave/cli/v2 v2.27.7 // indirect
 	github.com/xrash/smetrics v0.0.0-20250705151800-55b8f293f342 // indirect
-	go.yaml.in/yaml/v2 v2.4.3 // indirect
+	go.yaml.in/yaml/v2 v2.4.4 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/crypto v0.48.0 // indirect
-	golang.org/x/mod v0.33.0 // indirect
-	golang.org/x/sync v0.19.0 // indirect
-	golang.org/x/sys v0.41.0 // indirect
-	golang.org/x/text v0.34.0 // indirect
-	golang.org/x/tools v0.42.0 // indirect
+	golang.org/x/crypto v0.49.0 // indirect
+	golang.org/x/mod v0.34.0 // indirect
+	golang.org/x/sync v0.20.0 // indirect
+	golang.org/x/sys v0.42.0 // indirect
+	golang.org/x/text v0.35.0 // indirect
+	golang.org/x/tools v0.43.0 // indirect
 	google.golang.org/protobuf v1.36.11 // indirect
 	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
 	sigs.k8s.io/yaml v1.6.0 // indirect


Commits since last release


254affa Merge remote session logs
d38bf70 Checkpoint: e24929f05ad3
a116a1a Update dependencies. Synchronize options, cli flags with cc-backend
3f01e7b Bump github.com/ClusterCockpit/cc-lib/v2 from 2.8.0 to 2.11.0
5708dc2 Bump github.com/ClusterCockpit/cc-backend from 1.5.0 to 1.5.2
f00277f Merge pull request #60 from ClusterCockpit/dev
41fb959 Cleanup README
b6c04e3 Merge pull request #59 from ClusterCockpit/dev
3e0805b Fix golreaser config
2eb2227 Checkpoint: 37abb99fe6a7
6ce91cb Checkpoint: 37abb99fe6a7
e6930b4 Merge remote session logs
04cbc95 Checkpoint: dd6b5959d7c6
02c6ebc Checkpoint: 1f0df6df7655
ea69b43 Initialize metadata branch
e4cbfd5 Initialize metadata branch
99c5eae Bump github.com/ClusterCockpit/cc-lib/v2 from 2.2.1 to 2.7.0
c9602eb Bump github.com/ClusterCockpit/cc-backend
7404ea2 Bump github.com/nats-io/nats.go from 1.47.0 to 1.48.0
da271ba Bump github.com/linkedin/goavro/v2 from 2.13.1 to 2.14.1
f48e178 Bump github.com/swaggo/swag from 1.16.3 to 1.16.6
4d27e38 go mod tidy
2691254 Bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.2
ace6046 Merge branch 'main' into devel
c28fee4 Merge branch 'devel' into hotfix
1941e59 Switch to cc-lib
26335a2 Remove WithUnit query option
3f3a150 Return unit in API requests per default
997dd8f Update test case to check the metric directly
eb5b22e Update archive_test.go
04cf555 Import unit test
2d77dae Add basic test case for reading in checkpoint files
5e5586f Add unit to buffers
e71e1b1 Minor fix to Makefile
7c891e1 Add request flag WithUnit and return the metric unit if requested
051cba4 Add unit for archive reader/writer
89acbe8 Accept unit tags for incoming metrics and store it in the buffer.


git show a116a1a -- cmd/cc-metric-store/main.go | head -100

Show changes to main.go in latest commit


commit a116a1ab16df6e2007173bfe6d77376a73ce329a
Author: Jan Eitzinger <jan@moebiusband.org>
Date:   Wed Apr 1 15:37:33 2026 +0200

    Update dependencies. Synchronize options, cli flags with cc-backend

    Entire-Checkpoint: e24929f05ad3

diff --git a/cmd/cc-metric-store/main.go b/cmd/cc-metric-store/main.go
index b69d78e..1649446 100644
--- a/cmd/cc-metric-store/main.go
+++ b/cmd/cc-metric-store/main.go
@@ -7,6 +7,7 @@ package main

 import (
 	"context"
+	"encoding/json"
 	"flag"
 	"fmt"
 	"os"
@@ -14,6 +15,7 @@ import (
 	"runtime/debug"
 	"sync"
 	"syscall"
+	"time"

 	"github.com/ClusterCockpit/cc-backend/pkg/metricstore"
 	ccconf "github.com/ClusterCockpit/cc-lib/v2/ccConfig"
@@ -36,8 +38,8 @@ var (
 )

 var (
-	flagGops, flagVersion, flagDev, flagLogDateTime bool
-	flagConfigFile, flagLogLevel                    string
+	flagGops, flagVersion, flagDev, flagLogDateTime, flagCleanupCheckpoints bool
+	flagConfigFile, flagLogLevel                                            string
 )

 func printVersion() {
@@ -60,13 +62,14 @@ func runServer(ctx context.Context) error {
 		return fmt.Errorf("missing metricstore configuration")
 	}

-	metricstore.Init(mscfg, config.GetMetrics(), &wg)
-
-	// Set GC percent if not configured
+	// Set GC percent before loading checkpoints so the GC baseline is established
+	// with a low target from the start of the largest allocation event.
 	if os.Getenv(envGOGC) == "" {
 		debug.SetGCPercent(15)
 	}

+	metricstore.Init(mscfg, config.GetMetrics(), &wg)
+
 	if config.Keys.BackendURL != "" {
 		ms := metricstore.GetMemoryStore()
 		ms.SetNodeProvider(api.NewBackendNodeProvider(config.Keys.BackendURL))
@@ -127,6 +130,7 @@ func run() error {
 	flag.BoolVar(&flagDev, "dev", false, "Enable development component: Swagger UI")
 	flag.BoolVar(&flagVersion, "version", false, "Show version information and exit")
 	flag.BoolVar(&flagLogDateTime, "logdate", false, "Set this flag to add date and time to log messages")
+	flag.BoolVar(&flagCleanupCheckpoints, "cleanup-checkpoints", false, "Clean up old checkpoint files (delete or archive) based on retention settings, then exit")
 	flag.StringVar(&flagConfigFile, "config", "./config.json", "Specify alternative path to `config.json`")
 	flag.StringVar(&flagLogLevel, "loglevel", "warn", "Sets the logging level: `[debug, info, warn (default), err, crit]`")
 	flag.Parse()
@@ -138,12 +142,6 @@ func run() error {

 	cclog.Init(flagLogLevel, flagLogDateTime)

-	if flagGops || config.Keys.Debug.EnableGops {
-		if err := agent.Listen(agent.Options{}); err != nil {
-			return fmt.Errorf("starting gops agent: %w", err)
-		}
-	}
-
 	ccconf.Init(flagConfigFile)

 	cfg := ccconf.GetPackageConfig("main")
@@ -153,6 +151,44 @@ func run() error {

 	config.Init(cfg)

+	if flagGops || config.Keys.Debug.EnableGops {
+		if err := agent.Listen(agent.Options{}); err != nil {
+			return fmt.Errorf("starting gops agent: %w", err)
+		}
+	}
+
+	if flagCleanupCheckpoints {
+		mscfg := ccconf.GetPackageConfig("metric-store")
+		if mscfg == nil {
+			return fmt.Errorf("metric-store configuration required for checkpoint cleanup")
+		}
+		if err := json.Unmarshal(mscfg, &metricstore.Keys); err != nil {
+			return fmt.Errorf("decoding metric-store config: %w", err)
+		}
+		d, err := time.ParseDuration(metricstore.Keys.RetentionInMemory)
+		if err != nil {
+			return fmt.Errorf("parsing retention-in-memory: %w", err)
+		}


List ClusterCockpit directory


Shell cwd was reset to /Users/jan/prg/ClusterCockpit/cc-metric-store


git show a116a1a -- cmd/cc-metric-store/main.go | tail -60

Show rest of main.go changes


 	flag.StringVar(&flagLogLevel, "loglevel", "warn", "Sets the logging level: `[debug, info, warn (default), err, crit]`")
 	flag.Parse()
@@ -138,12 +142,6 @@ func run() error {

 	cclog.Init(flagLogLevel, flagLogDateTime)

-	if flagGops || config.Keys.Debug.EnableGops {
-		if err := agent.Listen(agent.Options{}); err != nil {
-			return fmt.Errorf("starting gops agent: %w", err)
-		}
-	}
-
 	ccconf.Init(flagConfigFile)

 	cfg := ccconf.GetPackageConfig("main")
@@ -153,6 +151,44 @@ func run() error {

 	config.Init(cfg)

+	if flagGops || config.Keys.Debug.EnableGops {
+		if err := agent.Listen(agent.Options{}); err != nil {
+			return fmt.Errorf("starting gops agent: %w", err)
+		}
+	}
+
+	if flagCleanupCheckpoints {
+		mscfg := ccconf.GetPackageConfig("metric-store")
+		if mscfg == nil {
+			return fmt.Errorf("metric-store configuration required for checkpoint cleanup")
+		}
+		if err := json.Unmarshal(mscfg, &metricstore.Keys); err != nil {
+			return fmt.Errorf("decoding metric-store config: %w", err)
+		}
+		d, err := time.ParseDuration(metricstore.Keys.RetentionInMemory)
+		if err != nil {
+			return fmt.Errorf("parsing retention-in-memory: %w", err)
+		}
+		from := time.Now().Add(-d)
+		deleteMode := metricstore.Keys.Cleanup == nil || metricstore.Keys.Cleanup.Mode != "archive"
+		cleanupDir := ""
+		if !deleteMode {
+			cleanupDir = metricstore.Keys.Cleanup.RootDir
+		}
+		cclog.Infof("Cleaning up checkpoints older than %s...", from.Format(time.RFC3339))
+		n, err := metricstore.CleanupCheckpoints(
+			metricstore.Keys.Checkpoints.RootDir, cleanupDir, from.Unix(), deleteMode)
+		if err != nil {
+			return fmt.Errorf("checkpoint cleanup: %w", err)
+		}
+		if deleteMode {
+			cclog.Printf("Cleanup done: %d checkpoint files deleted.", n)
+		} else {
+			cclog.Printf("Cleanup done: %d checkpoint files archived to parquet.", n)
+		}
+		return nil
+	}
+
 	natsConfig := ccconf.GetPackageConfig("nats")
 	if err := nats.Init(natsConfig); err != nil {
 		cclog.Warnf("initializing (optional) nats client: %s", err.Error())


cd /Users/jan/prg && find . -path "*/cc-backend/pkg/metricstore" -type d 2>/dev/null | head -5

Find cc-backend metricstore package


./ClusterCockpit/cc-backend/pkg/metricstore
Shell cwd was reset to /Users/jan/prg/ClusterCockpit/cc-metric-store


cd /Users/jan/prg/ClusterCockpit/cc-backend && git log --oneline -20 -- pkg/metricstore/

cc-backend metricstore recent commits


cd /Users/jan/prg/ClusterCockpit/cc-backend && git log v1.5.0..HEAD --oneline 2>/dev/null || git log --oneline -10

cc-backend changes since v1.5.0


641dc0e3 Run gofumpt
66707bbf Update metricstore documentation
fc47b12f fix: Pause WAL writes during binary checkpoint to prevent message drops
937984d1 fix: WAL rotation skipped for all nodes due to non-blocking send on small channel
cc3d03bb fix: Unbound growth of wal files in case of checkpointing error
ac0a4cc3 Increase shutdown timeouts and WAL flush interval
6e97ac8b Verbose logs for DataDoesNotAlign error in CCMS
97d65a9e Fix bugs in WAL journal pipeline
e7598100 Add shutdown timings. Do not drain WAL buffers on shutdown
6f7dda53 Cleanup
0325d9e8 fix: Increase throughput for WAL writers
e41d1251 fix: Continue on error
586c9020 Restructure metricstore cleanup archiving to stay withinh 32k parquet-go limit
09501df3 fix: reduce memory usage in parquet checkpoint archiver
8b132ed7 fix: Blocking ReceiveNats call
bf1a8a17 fix: Shard WAL consumer for higher throughput
50aed595 fix: metricstore NATS contention
d46e6371 Add log about checkpoint archiving
02f82c2c fix: Prevent memory spikes in parquet writer for  metricstore move policy
bab6eb4c Convert Warn message on missing metrics to Debug level
Shell cwd was reset to /Users/jan/prg/ClusterCockpit/cc-metric-store


573f7d14 Update ReleaseNotes
43807ae1 feat: Also submit projects array via oidc token
31a8a11f fix: Always request oidc roles from token
84fe61b3 fix: allow all role changes on SyncUser and UpdateUser callback
1f04e0a1 fix: oidc role extraction
641dc0e3 Run gofumpt
b734c1a9 And another update
c5fe3c5c Update golangci settings
e2910b18 Fix golangci config
ed236ec5 Add Make targets for formatting and linting
82c514b1 Ease samesite cookie settings
66707bbf Update metricstore documentation
fc47b12f fix: Pause WAL writes during binary checkpoint to prevent message drops
937984d1 fix: WAL rotation skipped for all nodes due to non-blocking send on small channel
3d99aec1 fix: Log viewer auto-refresh
28028918 Add checkpointReader to ReleaseNotes
cc3d03bb fix: Unbound growth of wal files in case of checkpointing error
ac0a4cc3 Increase shutdown timeouts and WAL flush interval
71fc9efe Add binaryCheckpointReader as utility tool
6e97ac8b Verbose logs for DataDoesNotAlign error in CCMS
97d65a9e Fix bugs in WAL journal pipeline
e7598100 Add shutdown timings. Do not drain WAL buffers on shutdown
b1884fda Prepare bug fix release 1.5.3
c267501a Reduve noise in tagger logging
a550344f Increase server shutdown timeout
bd7125a5 review doubleranged filters, fix and improve valeu selection
93a9d732 fix: Improve shutdown time
6f7dda53 Cleanup
0325d9e8 fix: Increase throughput for WAL writers
3d94b0bf Merge branch 'master' into hotfix
d5ea2b4c change: query node states explicitly in node view
45f329e5 feat: Add command line switch to trigger manual metricstore checkpoint cleanup
100dd7da Merge pull request #533 from ClusterCockpit/hotfix
192c94a7 fix: Prevent interruption of body lineprotocol parsing on locks
e41d1251 fix: Continue on error
586c9020 Restructure metricstore cleanup archiving to stay withinh 32k parquet-go limit
01ec70ba Iterate over subCluster MetricConfig directly so that removed metrics are not included
97330ce5 Merge pull request #532 from ClusterCockpit/hotfix
fb176c5a Remove static linkage for helper tools
d4ee9371 Merge pull request #531 from ClusterCockpit/hotfix
999d93ef Fix goreleaser config. Cleanup.
4ce0cfb6 Merge pull request #530 from ClusterCockpit/hotfix
359962d1 Fix typo
60554896 Update ReleaseNote for upcoming release
a9f335d9 Merge pull request #529 from ClusterCockpit/hotfix
bf48389a Optimize sortby in stats queries
10b4fa5a change: remove heuristic metricHealth, replace with DB metricHealth
886791cf remove deprecated minRunningFor filter remnants
6cad2ee1 bump frontend dependencies, increase version to match release
16ec1e69 streamline and unify statsSeries calc and render
676025ad Merge pull request #528 from ClusterCockpit/hotfix
c42898bd fix: add top list query fixes to analysis and dashboard
22057ff2 Pass reqKey as CacheKey
30b8ca4a further clarify plot titles
d4a0ae17 Merge pull request #525 from ClusterCockpit/hotfix
09501df3 fix: reduce memory usage in parquet checkpoint archiver
bb691577 fix: clarify title
8b0881fb Exclude down nodes from HealthCheck
33beb3c8 fix: simplify stats query condition
c1d51959 Change dtermineState to enforce priority order
3328d2ca Update go version in CLAUDE.md
8f10eba7 Extend CLAUDE.md
c4499965 Add context to log message
a7e5ecaf Merge pull request #524 from ClusterCockpit/hotfix
51ae2a5d Remove tracked .entire/metadata/ files from git
965e2007 Merge pull request #523 from ClusterCockpit/hotfix
6ebc9e88 Add more context information to auth failed log
8b132ed7 fix: Blocking ReceiveNats call
bf1a8a17 fix: Shard WAL consumer for higher throughput
50aed595 fix: metricstore NATS contention
33bc19c7 Upgrade cc-lib
045f81f9 Prepare release v1.5.2
d46e6371 Add log about checkpoint archiving
02f82c2c fix: Prevent memory spikes in parquet writer for  metricstore move policy
6a29faf4 Merge pull request #521 from ClusterCockpit/hotfix
3314b8e2 Ignore ErrNoRows error. Include calling function in log.
6855d62b Make log in scanRow more descriptive. No log for common no rows error
7f3eb443 Include calling function in error message
bab6eb4c Convert Warn message on missing metrics to Debug level
09d0ba71 Provide idential nodestate functionality in NATS API
df93dbed Add busyTimeout config setting
e4f3fa9b Wrap SyncJobs in transaction
51517f80 Reduce insert pressure in db. Increase sqlite timeout value
0aad8f01 Upgrade cc-lib
8751ae02 Merge pull request #520 from ClusterCockpit/hotfix
973ca87b Extend known issues in ReleaseNotes
128c0988 Merge pull request #519 from ClusterCockpit/hotfix
045311ee Prepare release 1.5.1
e38396a0 Upgrade dependencies. Rebuild GraphQL.
e83bd2ba Consolidate migrations
ba366d0d use inline literals in simple queries, add downgrade optimize
f15f1452 Inline jobstate literal in query
df2a13de Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix
d586fe4b Optimize usage dashboard: partial indexes, request cache, parallel histograms
bc214f6c add nullsafes to frontend
cbe46c35 Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix
0037d969 Consolidate UsageDash into single GraphQL query
dd3e5427 Add covering indexes for status/dashboard queries (migration 13)
e6669801 fix typo
c238f68a reduce unnecessary complexity
58c0c79f handle single job state queries as simple stringquery
c23d7bd5 remove non-required sorting params
41114f7e reorder frontend coded filters to match db indices
a877937a add missing downgarde index drop, add optimize after downgrades
39ab1278 Make checkpointInterval an option config option again.
b214e175 Add buffered I/O to WAL writes and fix MemoryCap comment
a4f9ba69 Apply correct log level
8234ad31 fix: Fix metricstore memory explosion from broken emergency free and batch aborts
126f6587 Update Release informations
3aacc669 Remove debug timer
96fc44a6 fix: Optimize project stat query
8e86e872 Make stats query selective. Add stats index. Add paging to user list.
4555fb8a Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix
0e27624d Add flag to optimize db. Remove ANALYZE on startup.
8563ed5e fix: remove indices from migration 9
2d07bdf6 fix: add missing nullsafe in publicDash
7f069f1e Prepare bugfix release 1.5.1
2506a92c Remove entire log
972b1403 add db migration 11, optimizing index count
af78f06c fix: Reduce complexity for groupBy stats queries
6e0fe625 Add new db config options to README
e70310dc fix: Segvault when taggers are enabled but rule directories missing
00d2f97c fix: Large heap allocations in sqlite driver. Sanitize sqlite config and make it configurablex. Allow to cancel queries.
c8d8f708 Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix
dc7407d0 fix: prevent segvault if enable-job-taggers option is tru but tagger config directories are missing
eba39956 Add Analyse on db startup
f8831e70 Fixed merge errors
1cf99206 Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix
5d3d7762 fix: Add defer.close for all queries
5c726641 bump frontend patch versions
f3e796f3 add nullsafes to node view
cc38b174 fix wrong field checked vor json validity
282197eb fix: Round floats in tagger message
d2bc046f fix ranged filter GT and LT conditions, reduce energy filter preset
Shell cwd was reset to /Users/jan/prg/ClusterCockpit/cc-metric-store


Now I have enough context. Let me update the ReleaseNotes.md for the v1.5.3 release.


Edit

# `cc-metric-store` version 1.5.3

This is a major release of `cc-metric-store`, the metric timeseries cache
implementation of ClusterCockpit. Since the storage engine is now part of
`cc-backend` we will follow the version number of `cc-backend`.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).

## Breaking changes

- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
  removed. The storage engine is now provided by the
  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
  only.
- The configuration schema has changed. Refer to `configs/config.json` for the
  updated structure.

## Notable changes

- **Storage engine extracted to `cc-backend` library**: The entire in-memory
  time-series storage engine was moved to `cc-backend/pkg/metricstore`. This
  reduces duplication in the ClusterCockpit suite and enables shared maintenance
  of the storage layer.
- **HealthCheck API endpoint**: New `GET /api/healthcheck/` endpoint reports the
  health status of cluster nodes.
- **Dynamic memory management**: Memory limits can now be adjusted at runtime via
  a callback from the `cc-backend` library.
- **Configuration schema validation**: The config and metric config JSON schemas
  have been updated and are now validated against the structs they describe.
- **Startup refactored**: Application startup has been split into `cli.go` and
  `server.go` for clearer separation of concerns.
- **`go fix` applied**: Codebase updated to current Go idioms.
- **Dependency upgrades**: `nats.go` bumped from 1.36.0 to 1.47.0;
  `cc-lib` updated to v2.8.0; `cc-backend` updated to v1.5.0; various other
  module upgrades.

# `cc-metric-store` version 1.5.3

This is a bugfix release of `cc-metric-store`, the metric timeseries cache
implementation of ClusterCockpit. Since the storage engine is now part of
`cc-backend` we will follow the version number of `cc-backend`.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).

## Notable changes

- **`-cleanup-checkpoints` CLI flag**: New flag triggers checkpoint cleanup
  (delete or archive to Parquet) based on the configured retention and cleanup
  settings, then exits. Useful for one-off maintenance without starting the full
  server.
- **GC initialised before checkpoint load**: `GOGC=15` is now set before
  `metricstore.Init` so the garbage-collector baseline is established prior to
  the largest allocation event (loading checkpoints from disk), reducing
  unnecessary heap growth at startup.
- **Dependency upgrades**: `cc-backend` updated from v1.5.0 to v1.5.3;
  `cc-lib` updated from v2.8.0 to v2.11.0; `nats.go` bumped from v1.49.0 to
  v1.50.0; `parquet-go` bumped from v0.28.0 to v0.29.0; various other module
  upgrades.

## Metricstore package fixes (cc-backend v1.5.0 → v1.5.3)

The following fixes landed in the upstream `cc-backend/pkg/metricstore` package
and are included via the dependency upgrade:

- **WAL correctness**: Fixed WAL rotation being skipped for all nodes due to a
  non-blocking send on a too-small channel; fixed unbound growth of WAL files
  when a checkpointing error occurs; fixed bugs in the WAL journal pipeline.
- **WAL throughput**: Sharded the WAL consumer for higher write throughput; added
  buffered I/O to WAL writes.
- **Checkpoint stability**: Paused WAL writes during binary checkpoint creation
  to prevent message drops; restructured cleanup archiving to stay within the
  32 k row limit of `parquet-go`.
- **Memory**: Fixed a memory explosion caused by broken emergency-free and batch
  aborts; reduced memory usage in the Parquet checkpoint archiver; fixed
  preventing memory spikes in the Parquet writer during the move/archive policy.
- **NATS**: Fixed blocking `ReceiveNats` call; fixed NATS contention under load.
- **Shutdown**: Increased shutdown timeouts; added WAL flush interval tuning;
  added shutdown timing logs.
- **Observability**: Added verbose logs for `DataDoesNotAlign` errors; reduced
  noise by demoting a missing-metric warning to debug level.
- **Configuration**: Restored `checkpointInterval` as an optional config key.

## Breaking changes (from v1.4.x)

- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
  removed. The storage engine is now provided by the
  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
  only.
- The configuration schema has changed. Refer to `configs/config.json` for the
  updated structure.


The file /Users/jan/prg/ClusterCockpit/cc-metric-store/ReleaseNotes.md has been updated successfully.

# `cc-metric-store` version 1.5.3

This is a major release of `cc-metric-store`, the metric timeseries cache
implementation of ClusterCockpit. Since the storage engine is now part of
`cc-backend` we will follow the version number of `cc-backend`.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).

## Breaking changes

- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
  removed. The storage engine is now provided by the
  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
  only.
- The configuration schema has changed. Refer to `configs/config.json` for the
  updated structure.

## Notable changes

- **Storage engine extracted to `cc-backend` library**: The entire in-memory
  time-series storage engine was moved to `cc-backend/pkg/metricstore`. This
  reduces duplication in the ClusterCockpit suite and enables shared maintenance
  of the storage layer.
- **HealthCheck API endpoint**: New `GET /api/healthcheck/` endpoint reports the
  health status of cluster nodes.
- **Dynamic memory management**: Memory limits can now be adjusted at runtime via
  a callback from the `cc-backend` library.
- **Configuration schema validation**: The config and metric config JSON schemas
  have been updated and are now validated against the structs they describe.
- **Startup refactored**: Application startup has been split into `cli.go` and
  `server.go` for clearer separation of concerns.
- **`go fix` applied**: Codebase updated to current Go idioms.
- **Dependency upgrades**: `nats.go` bumped from 1.36.0 to 1.47.0;
  `cc-lib` updated to v2.8.0; `cc-backend` updated to v1.5.0; various other
  module upgrades.

# `cc-metric-store` version 1.5.3

This is a bugfix release of `cc-metric-store`, the metric timeseries cache
implementation of ClusterCockpit. Since the storage engine is now part of
`cc-backend` we will follow the version number of `cc-backend`.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).

## Notable changes

- **`-cleanup-checkpoints` CLI flag**: New flag triggers checkpoint cleanup
  (delete or archive to Parquet) based on the configured retention and cleanup
  settings, then exits. Useful for one-off maintenance without starting the full
  server.
- **GC initialised before checkpoint load**: `GOGC=15` is now set before
  `metricstore.Init` so the garbage-collector baseline is established prior to
  the largest allocation event (loading checkpoints from disk), reducing
  unnecessary heap growth at startup.
- **Dependency upgrades**: `cc-backend` updated from v1.5.0 to v1.5.3;
  `cc-lib` updated from v2.8.0 to v2.11.0; `nats.go` bumped from v1.49.0 to
  v1.50.0; `parquet-go` bumped from v0.28.0 to v0.29.0; various other module
  upgrades.

## Metricstore package fixes (cc-backend v1.5.0 → v1.5.3)

The following fixes landed in the upstream `cc-backend/pkg/metricstore` package
and are included via the dependency upgrade:

- **WAL correctness**: Fixed WAL rotation being skipped for all nodes due to a
  non-blocking send on a too-small channel; fixed unbound growth of WAL files
  when a checkpointing error occurs; fixed bugs in the WAL journal pipeline.
- **WAL throughput**: Sharded the WAL consumer for higher write throughput; added
  buffered I/O to WAL writes.
- **Checkpoint stability**: Paused WAL writes during binary checkpoint creation
  to prevent message drops; restructured cleanup archiving to stay within the
  32 k row limit of `parquet-go`.
- **Memory**: Fixed a memory explosion caused by broken emergency-free and batch
  aborts; reduced memory usage in the Parquet checkpoint archiver; fixed
  preventing memory spikes in the Parquet writer during the move/archive policy.
- **NATS**: Fixed blocking `ReceiveNats` call; fixed NATS contention under load.
- **Shutdown**: Increased shutdown timeouts; added WAL flush interval tuning;
  added shutdown timing logs.
- **Observability**: Added verbose logs for `DataDoesNotAlign` errors; reduced
  noise by demoting a missing-metric warning to debug level.
- **Configuration**: Restored `checkpointInterval` as an optional config key.

## Breaking changes (from v1.4.x)

- The internal `memorystore`, `avro`, `resampler`, and `util` packages have been
  removed. The storage engine is now provided by the
  [`cc-backend`](https://github.com/ClusterCockpit/cc-backend) package
  (`cc-backend/pkg/metricstore`). This repository is now the HTTP API wrapper
  only.
- The configuration schema has changed. Refer to `configs/config.json` for the
  updated structure.

-This is a major release of `cc-metric-store`, the metric timeseries cache

+This is a bugfix release of `cc-metric-store`, the metric timeseries cache

+- **`-cleanup-checkpoints` CLI flag**: New flag triggers checkpoint cleanup

+  (delete or archive to Parquet) based on the configured retention and cleanup

+  settings, then exits. Useful for one-off maintenance without starting the full

+- **GC initialised before checkpoint load**: `GOGC=15` is now set before

+  `metricstore.Init` so the garbage-collector baseline is established prior to

+  the largest allocation event (loading checkpoints from disk), reducing

+- **Dependency upgrades**: `cc-backend` updated from v1.5.0 to v1.5.3;

+  `cc-lib` updated from v2.8.0 to v2.11.0; `nats.go` bumped from v1.49.0 to

+  v1.50.0; `parquet-go` bumped from v0.28.0 to v0.29.0; various other module

+## Metricstore package fixes (cc-backend v1.5.0 → v1.5.3)

+The following fixes landed in the upstream `cc-backend/pkg/metricstore` package

+and are included via the dependency upgrade:

+- **WAL correctness**: Fixed WAL rotation being skipped for all nodes due to a

+  non-blocking send on a too-small channel; fixed unbound growth of WAL files

+  when a checkpointing error occurs; fixed bugs in the WAL journal pipeline.

+- **WAL throughput**: Sharded the WAL consumer for higher write throughput; added

+- **Checkpoint stability**: Paused WAL writes during binary checkpoint creation

+  to prevent message drops; restructured cleanup archiving to stay within the

+- **Memory**: Fixed a memory explosion caused by broken emergency-free and batch

+  aborts; reduced memory usage in the Parquet checkpoint archiver; fixed

+  preventing memory spikes in the Parquet writer during the move/archive policy.

+- **NATS**: Fixed blocking `ReceiveNats` call; fixed NATS contention under load.

+- **Shutdown**: Increased shutdown timeouts; added WAL flush interval tuning;

+- **Observability**: Added verbose logs for `DataDoesNotAlign` errors; reduced

+  noise by demoting a missing-metric warning to debug level.

+- **Configuration**: Restored `checkpointInterval` as an optional config key.

-- **Storage engine extracted to `cc-backend` library**: The entire in-memory

-  time-series storage engine was moved to `cc-backend/pkg/metricstore`. This

-  reduces duplication in the ClusterCockpit suite and enables shared maintenance

-- **HealthCheck API endpoint**: New `GET /api/healthcheck/` endpoint reports the

-- **Dynamic memory management**: Memory limits can now be adjusted at runtime via

-  a callback from the `cc-backend` library.

-- **Configuration schema validation**: The config and metric config JSON schemas

-  have been updated and are now validated against the structs they describe.

-- **Startup refactored**: Application startup has been split into `cli.go` and

-  `server.go` for clearer separation of concerns.

-- **`go fix` applied**: Codebase updated to current Go idioms.

-- **Dependency upgrades**: `nats.go` bumped from 1.36.0 to 1.47.0;

-  `cc-lib` updated to v2.8.0; `cc-backend` updated to v1.5.0; various other


The `ReleaseNotes.md` has been updated for v1.5.3 with:

- **New `-cleanup-checkpoints` flag** added in this release
- **GC initialisation order fix** (GOGC set before checkpoint load)
- **Dependency upgrade summary** (cc-backend 1.5.0→1.5.3, cc-lib, nats.go, parquet-go)
- **Metricstore package fixes** section covering all the WAL, checkpoint, memory, NATS, and shutdown fixes that landed in cc-backend between v1.5.0 and v1.5.3
- Moved the old breaking-changes block to a clearly labelled "from v1.4.x" section so it doesn't confuse users upgrading within the 1.5.x line