Commit Graph
243 Commits
Author SHA1 Message Date
moebiusbandandClaude Opus 5 a5c1fdf21f fix: forward resample algorithm to metric store for running jobs
Only archived job data and internal-store node-list data honoured the
selected resample algorithm. For running jobs the algorithm was dropped:
MetricDataRepository.LoadData had no such parameter, so the memory store
fell back to an empty string, which cc-lib's GetResampler maps to LTTB.
The external store client was worse - its APIQueryRequest had no
ResampleAlgo field at all, and LoadNodeListData accepted the parameter
without using it.

Add resampleAlgo to the LoadData interface (mirroring LoadNodeListData),
forward it from metricdispatch, and set it on both stores' requests. The
field is tagged omitempty, so the wire format is unchanged when empty -
verify the deployed cc-metric-store accepts it before relying on it there.

The REST job endpoints pass a non-zero resolution and therefore do
resample, so they now request the configured default instead of an empty
string. Add config.ResampleAlgo() for that, since "" is not a neutral
value at this layer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 12:17:03 +02:00
moebiusbandandClaude Opus 5 fca7b81e3d chore: apply gofumpt formatting
Pre-existing formatting drift picked up by `make fmt`. Whitespace and
comment layout only, no semantic change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 10:28:22 +02:00
moebiusbandandClaude Opus 5 4cfbecd818 Merge branch 'feat/565-add-metric-tooltip' into metric-store-tickets
Resolve conflicts in the generated GraphQL code by regenerating it against
the merged schema. cc-lib v2.13.0 adds Tooltip to schema.MetricConfig and
schema.GlobalMetricListItem, so gqlgen now binds the tooltip field directly
and the hand-written globalMetricListItem/metricConfig resolvers introduced
on the tooltip branch are no longer needed.

Also migrate to the cc-lib v2.13.0 metric container types, which changed
from bare maps to structs carrying array-valued metric groups:

  schema.JobData          map -> {Metrics, Groups}
  schema.ScopedJobStats   map -> {Metrics, Groups}
  job.Statistics          map -> schema.JobStatisticsSet{Metrics, Groups}

Callers index .Metrics, return the zero struct instead of nil, and
deepCopy/DecodeJobStats now also carry the Groups payload through.
archive.GetStatistics returns the full JobStatisticsSet so group
statistics survive the round trip.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 10:22:22 +02:00
moebiusband ffbe171327 Merge branch 'main' into feature/526-average-resample 2026-06-18 07:17:41 +02:00
moebiusbandandClaude Opus 4.8 2b01b57495 feat: replace gorilla/sessions with alexedwards/scs/v2
Browser sessions are now server-side, stored in the SQLite database via
scs/sqlite3store (new `sessions` table, DB migration to version 12) instead
of gorilla/sessions client-side cookie storage. Only an opaque random token
is kept in the cookie; session data lives server-side and survives restarts.

Session middleware is wired as a hybrid to avoid buffering large responses:
scs.LoadAndSave on the login/logout write paths, and a non-buffering
read-only LoadSession middleware on the secured/config/frontend read paths
so the large GraphQL /query responses stream unbuffered. JWT-only APIs
(/api, /userapi, /api/metricstore) and static files are left unwrapped.

The session cookie Secure flag is now derived from the server config (set
when cc-backend terminates TLS itself); previously it was effectively never
set. The SESSION_KEY env var is removed as server-side tokens need no
signing secret. The dormant Bearer-JWT branch in the frontend urql client
is removed; the web UI authenticates GraphQL via the session cookie.

Closes #558

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b51075f43cc7
2026-06-17 07:54:26 +02:00
moebiusband 5d8d4e228e Merge branch 'main' into feature/526-average-resample 2026-06-17 06:30:54 +02:00
Jan EitzingerandGitHub c94f5918f3 Merge pull request #556 from ClusterCockpit/release/v1.5
Fix critical/severe issues in init, startup and shutdown
2026-06-07 07:31:02 +02:00
moebiusbandandClaude Opus 4.8 1b72b0b5ad Fix critical/severe issues in init, startup and shutdown
- auth: do not abort the server when authentication is disabled. auth.Init
  is now always called; with disable-authentication it sets up an ephemeral
  session store (SESSION_KEY not required) and registers no authenticators,
  so the unconditional auth.GetAuthInstance() callers (server init,
  api.New()) always get a valid instance.
- main: run the graceful-shutdown sequence on the startup-error path. runServer
  derives a cancelable context and, on a server-start failure, cancels it and
  waits so the metricstore final checkpoint / WAL rotation, archiver flush and
  taskmanager shutdown actually run before exit.
- server: log the :80 HTTP->HTTPS redirect listener error instead of dropping it.
- archiver: guard Shutdown against being called when Start never ran
  (avoids close(nil) panic / blocking on a nil workerDone).
- nats API: stop worker goroutines on shutdown via a stop channel + idempotent
  Shutdown(); workers and subscription callbacks select on stop and the
  channels are never closed, so no send-on-closed-channel can occur. Wired
  into Server.Shutdown after the NATS client is closed.
- metricstore: make Shutdown idempotent (nil shutdownFunc, early return) and
  release shutdownFuncMu before the checkpoint write.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3c179f9caa8f
2026-06-05 10:16:28 +02:00
Jan EitzingerandGitHub d74ae77c8e Merge pull request #553 from ClusterCockpit/release/v1.5
Release/v1.5
2026-06-04 20:33:22 +02:00
moebiusbandandClaude Opus 4.8 16942f55a0 Fix medium-severity issues from follow-up security audit
Addresses the remaining medium findings from the second-pass audit:

- DoS hardening: bound GraphQL query cost with FixedComplexityLimit, and
  reject non-positive items-per-page / page values so uint64 conversion
  cannot underflow into an unbounded LIMIT/OFFSET. The -1 "load all"
  sentinel stays valid for dashboards; REST now returns 400 for bad input.

- Security headers: add X-Content-Type-Options, X-Frame-Options,
  Referrer-Policy and a conservative CSP (frame-ancestors/object-src/
  base-uri) that hardens against clickjacking and base-tag injection
  without restricting the self-hosted SPA's inline scripts.

- Stored XSS: render job.metaData.message as escaped text instead of
  {@html ...} in Job.root and JobFootprint, preserving line breaks via
  white-space: pre-wrap.

- SQL injection hardening: parameterize the tag-scope IN list and the
  manager project subquery in CountTags instead of interpolating
  user.Username / user.Projects (externally sourced via OIDC/LDAP).

- CSRF defense-in-depth: reject cross-site state-changing requests via
  Sec-Fetch-Site, failing open for non-browser clients, on top of the
  existing SameSite=Lax session cookie.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: de7d47a85c7c
2026-06-04 20:08:41 +02:00
moebiusband 6f7e262f3f Fix issues after security audit
Entire-Checkpoint: bc18358a9343
2026-06-04 18:33:30 +02:00
moebiusband 0020f63582 Rebuild Swagger 2026-06-04 17:56:32 +02:00
Christoph Kluge 1ebde74774 Adapt swagger definitions of user update endpoint 2026-06-02 17:58:15 +02:00
Christoph Kluge 40722d72f5 fix name in doc comment 2026-06-02 16:48:23 +02:00
Christoph Kluge f4384668e5 fix name comment 2026-06-02 16:47:29 +02:00
Christoph Kluge e06982db00 reintroduce user update api path 2026-06-02 16:34:46 +02:00
moebiusband c0d2d65f96 Introduce average resampler support
Fixes #526

Entire-Checkpoint: 893a1de325b5
2026-03-19 21:16:48 +01:00
moebiusband 8b0881fb17 Exclude down nodes from HealthCheck
Entire-Checkpoint: 0c3347168c79
2026-03-18 11:20:12 +01:00
moebiusband c1d51959d5 Change dtermineState to enforce priority order
Make exception if node is idle + down, then final state is idle

Entire-Checkpoint: 92c797737df8
2026-03-18 10:57:06 +01:00
moebiusband 7f3eb443d9 Include calling function in error message
Entire-Checkpoint: a4948d0fe7a3
2026-03-16 15:42:38 +01:00
moebiusband 09d0ba71d2 Provide idential nodestate functionality in NATS API
Entire-Checkpoint: 3a40b75edd68
2026-03-16 12:13:14 +01:00
moebiusband 51517f8031 Reduce insert pressure in db. Increase sqlite timeout value
Entire-Checkpoint: a1e2931d4deb
2026-03-16 11:17:47 +01:00
moebiusband a418abc7d5 Run go fix 2026-02-27 14:40:26 +01:00
moebiusband a1db8263d7 Document line protocol. Optimize REST writeMetric path 2026-02-27 12:30:27 +01:00
moebiusband 6ecb934967 Switch to CC line-protocol package. Update cc-lib. 2026-02-27 08:55:33 +01:00
Jan EitzingerandGitHub 12c01655c3 Merge pull request #496 from ClusterCockpit/add_editmetabyrequest_2026
Manually migrate editMetaByRequest to current state of 2026
2026-02-23 12:53:07 +01:00
moebiusband 03c65e06f6 Allow finer control for omit tagged jobs in retention policies 2026-02-23 08:46:47 +01:00
moebiusband 8ee6c09e9b Enable to run taggers from within admin web interface 2026-02-22 09:42:53 +01:00
moebiusband fc1ba1f5b3 Merge branch 'dev' of github.com:ClusterCockpit/cc-backend into dev 2026-02-21 13:52:14 +01:00
moebiusband 82e79b074a Reverse Lookup order in stop job request 2026-02-21 13:51:31 +01:00
Christoph Kluge 2d90fd05d6 Manually migrate editMetaByRequest to current state of 2026
- originally PR #400
2026-02-20 10:38:07 +01:00
moebiusband e1c1148160 Fix more bugs related to job_cache ids used in job table 2026-02-20 09:20:18 +01:00
moebiusband f00f9fcee0 Cleanup debug printf 2026-02-19 17:42:45 +01:00
Aditya Ujeniya 2eeefc2720 Add healthCheck support for external CCMS 2026-02-16 16:57:17 +01:00
moebiusband 25ff094bdf Fix log endpoint 2026-02-16 09:10:08 +01:00
moebiusband 98f9c18f72 Merge branch 'dev' into log-aggregator 2026-02-15 19:53:34 +01:00
moebiusband 865cd3db54 Prersist faulty nodestate metric lists to db 2026-02-12 08:48:15 +01:00
moebiusband 8d6c6b819b Update and port to cc-lib 2026-02-11 07:06:06 +01:00
moebiusband ac7eb93141 fix: Transfer always to main job table before archiving 2026-02-09 19:57:46 +01:00
moebiusband 2b395a94e6 Fix setup issue with chi router 2026-02-07 18:02:48 +01:00
moebiusband f6aa40d927 Migrate from gorilla to chi web framework. add 404 handler 2026-02-07 17:48:12 +01:00
moebiusband 363e839c49 Add simple log viewer in web frontend 2026-02-07 07:05:33 +01:00
moebiusband a85f72fccd Change log level to debug for nodestate API endpoint 2026-02-06 14:30:04 +01:00
moebiusband fa7727c6ca Print job db id instead of its address 2026-02-06 14:06:56 +01:00
Christoph Kluge 276559d120 revert endpoint change 2026-02-05 15:25:49 +01:00
Christoph Kluge e3148b16eb add timers to updateNodeStates 2026-02-05 15:24:11 +01:00
Aditya Ujeniya 39b8356683 Optimized CCMS healthcheck 2026-02-04 10:24:45 +01:00
moebiusband 0d62a300e7 Intermediate state of node Healthcheck
TODOS:
* Remove error handling from routine and simplify API call
* Use map for hardware level metrics
2026-02-03 18:35:17 +01:00
moebiusband 248f11f4f8 Change API of Node HealthState 2026-02-03 14:55:12 +01:00
moebiusband 00a41373e8 Add monitoring healthstate support in nodestate API. 2026-02-03 12:23:24 +01:00