Commit Graph
238 Commits
Author SHA1 Message Date
moebiusbandandClaude Opus 4.8 0724b2dc60 Adopt cc-lib hierarchical JobData/Statistics structs
cc-lib changed JobData, ScopedJobStats and Job.Statistics from flat maps
to structs (a .Metrics map plus array-valued Groups) to represent
filesystem (and future interconnect) metric groups. Migrate all
construction, indexing and iteration to the new API across the archive
backends, metricstore query path, metric dispatcher, archiver, importer,
tagger, taskmanager, repository and API layers.

Semantics: DecodeJobStats now projects JobData.Groups into
ScopedJobStats.Groups, and the archiver derives per-filesystem node-scope
statistics into Job.Statistics.Groups. deepCopy, resampling and the
metric/scope filter are group-aware. The metricstore internal storage
(buffers, selector tree, checkpoint/parquet) is unchanged; all conversion
stays at the LoadData/archive-codec seam.

Note: requires the corresponding cc-lib release; bump the cc-lib
dependency version once tagged (a local go.mod replace was used during
development and is intentionally not committed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 19:14:12 +02:00
moebiusbandandClaude Opus 4.8 2b01b57495 feat: replace gorilla/sessions with alexedwards/scs/v2
Browser sessions are now server-side, stored in the SQLite database via
scs/sqlite3store (new `sessions` table, DB migration to version 12) instead
of gorilla/sessions client-side cookie storage. Only an opaque random token
is kept in the cookie; session data lives server-side and survives restarts.

Session middleware is wired as a hybrid to avoid buffering large responses:
scs.LoadAndSave on the login/logout write paths, and a non-buffering
read-only LoadSession middleware on the secured/config/frontend read paths
so the large GraphQL /query responses stream unbuffered. JWT-only APIs
(/api, /userapi, /api/metricstore) and static files are left unwrapped.

The session cookie Secure flag is now derived from the server config (set
when cc-backend terminates TLS itself); previously it was effectively never
set. The SESSION_KEY env var is removed as server-side tokens need no
signing secret. The dormant Bearer-JWT branch in the frontend urql client
is removed; the web UI authenticates GraphQL via the session cookie.

Closes #558

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b51075f43cc7
2026-06-17 07:54:26 +02:00
Jan EitzingerandGitHub c94f5918f3 Merge pull request #556 from ClusterCockpit/release/v1.5
Fix critical/severe issues in init, startup and shutdown
2026-06-07 07:31:02 +02:00
moebiusbandandClaude Opus 4.8 1b72b0b5ad Fix critical/severe issues in init, startup and shutdown
- auth: do not abort the server when authentication is disabled. auth.Init
  is now always called; with disable-authentication it sets up an ephemeral
  session store (SESSION_KEY not required) and registers no authenticators,
  so the unconditional auth.GetAuthInstance() callers (server init,
  api.New()) always get a valid instance.
- main: run the graceful-shutdown sequence on the startup-error path. runServer
  derives a cancelable context and, on a server-start failure, cancels it and
  waits so the metricstore final checkpoint / WAL rotation, archiver flush and
  taskmanager shutdown actually run before exit.
- server: log the :80 HTTP->HTTPS redirect listener error instead of dropping it.
- archiver: guard Shutdown against being called when Start never ran
  (avoids close(nil) panic / blocking on a nil workerDone).
- nats API: stop worker goroutines on shutdown via a stop channel + idempotent
  Shutdown(); workers and subscription callbacks select on stop and the
  channels are never closed, so no send-on-closed-channel can occur. Wired
  into Server.Shutdown after the NATS client is closed.
- metricstore: make Shutdown idempotent (nil shutdownFunc, early return) and
  release shutdownFuncMu before the checkpoint write.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3c179f9caa8f
2026-06-05 10:16:28 +02:00
Jan EitzingerandGitHub d74ae77c8e Merge pull request #553 from ClusterCockpit/release/v1.5
Release/v1.5
2026-06-04 20:33:22 +02:00
moebiusbandandClaude Opus 4.8 16942f55a0 Fix medium-severity issues from follow-up security audit
Addresses the remaining medium findings from the second-pass audit:

- DoS hardening: bound GraphQL query cost with FixedComplexityLimit, and
  reject non-positive items-per-page / page values so uint64 conversion
  cannot underflow into an unbounded LIMIT/OFFSET. The -1 "load all"
  sentinel stays valid for dashboards; REST now returns 400 for bad input.

- Security headers: add X-Content-Type-Options, X-Frame-Options,
  Referrer-Policy and a conservative CSP (frame-ancestors/object-src/
  base-uri) that hardens against clickjacking and base-tag injection
  without restricting the self-hosted SPA's inline scripts.

- Stored XSS: render job.metaData.message as escaped text instead of
  {@html ...} in Job.root and JobFootprint, preserving line breaks via
  white-space: pre-wrap.

- SQL injection hardening: parameterize the tag-scope IN list and the
  manager project subquery in CountTags instead of interpolating
  user.Username / user.Projects (externally sourced via OIDC/LDAP).

- CSRF defense-in-depth: reject cross-site state-changing requests via
  Sec-Fetch-Site, failing open for non-browser clients, on top of the
  existing SameSite=Lax session cookie.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: de7d47a85c7c
2026-06-04 20:08:41 +02:00
moebiusband 6f7e262f3f Fix issues after security audit
Entire-Checkpoint: bc18358a9343
2026-06-04 18:33:30 +02:00
moebiusband 0020f63582 Rebuild Swagger 2026-06-04 17:56:32 +02:00
Christoph Kluge 1ebde74774 Adapt swagger definitions of user update endpoint 2026-06-02 17:58:15 +02:00
Christoph Kluge 40722d72f5 fix name in doc comment 2026-06-02 16:48:23 +02:00
Christoph Kluge f4384668e5 fix name comment 2026-06-02 16:47:29 +02:00
Christoph Kluge e06982db00 reintroduce user update api path 2026-06-02 16:34:46 +02:00
moebiusband 8b0881fb17 Exclude down nodes from HealthCheck
Entire-Checkpoint: 0c3347168c79
2026-03-18 11:20:12 +01:00
moebiusband c1d51959d5 Change dtermineState to enforce priority order
Make exception if node is idle + down, then final state is idle

Entire-Checkpoint: 92c797737df8
2026-03-18 10:57:06 +01:00
moebiusband 7f3eb443d9 Include calling function in error message
Entire-Checkpoint: a4948d0fe7a3
2026-03-16 15:42:38 +01:00
moebiusband 09d0ba71d2 Provide idential nodestate functionality in NATS API
Entire-Checkpoint: 3a40b75edd68
2026-03-16 12:13:14 +01:00
moebiusband 51517f8031 Reduce insert pressure in db. Increase sqlite timeout value
Entire-Checkpoint: a1e2931d4deb
2026-03-16 11:17:47 +01:00
moebiusband a418abc7d5 Run go fix 2026-02-27 14:40:26 +01:00
moebiusband a1db8263d7 Document line protocol. Optimize REST writeMetric path 2026-02-27 12:30:27 +01:00
moebiusband 6ecb934967 Switch to CC line-protocol package. Update cc-lib. 2026-02-27 08:55:33 +01:00
Jan EitzingerandGitHub 12c01655c3 Merge pull request #496 from ClusterCockpit/add_editmetabyrequest_2026
Manually migrate editMetaByRequest to current state of 2026
2026-02-23 12:53:07 +01:00
moebiusband 03c65e06f6 Allow finer control for omit tagged jobs in retention policies 2026-02-23 08:46:47 +01:00
moebiusband 8ee6c09e9b Enable to run taggers from within admin web interface 2026-02-22 09:42:53 +01:00
moebiusband fc1ba1f5b3 Merge branch 'dev' of github.com:ClusterCockpit/cc-backend into dev 2026-02-21 13:52:14 +01:00
moebiusband 82e79b074a Reverse Lookup order in stop job request 2026-02-21 13:51:31 +01:00
Christoph Kluge 2d90fd05d6 Manually migrate editMetaByRequest to current state of 2026
- originally PR #400
2026-02-20 10:38:07 +01:00
moebiusband e1c1148160 Fix more bugs related to job_cache ids used in job table 2026-02-20 09:20:18 +01:00
moebiusband f00f9fcee0 Cleanup debug printf 2026-02-19 17:42:45 +01:00
Aditya Ujeniya 2eeefc2720 Add healthCheck support for external CCMS 2026-02-16 16:57:17 +01:00
moebiusband 25ff094bdf Fix log endpoint 2026-02-16 09:10:08 +01:00
moebiusband 98f9c18f72 Merge branch 'dev' into log-aggregator 2026-02-15 19:53:34 +01:00
moebiusband 865cd3db54 Prersist faulty nodestate metric lists to db 2026-02-12 08:48:15 +01:00
moebiusband 8d6c6b819b Update and port to cc-lib 2026-02-11 07:06:06 +01:00
moebiusband ac7eb93141 fix: Transfer always to main job table before archiving 2026-02-09 19:57:46 +01:00
moebiusband 2b395a94e6 Fix setup issue with chi router 2026-02-07 18:02:48 +01:00
moebiusband f6aa40d927 Migrate from gorilla to chi web framework. add 404 handler 2026-02-07 17:48:12 +01:00
moebiusband 363e839c49 Add simple log viewer in web frontend 2026-02-07 07:05:33 +01:00
moebiusband a85f72fccd Change log level to debug for nodestate API endpoint 2026-02-06 14:30:04 +01:00
moebiusband fa7727c6ca Print job db id instead of its address 2026-02-06 14:06:56 +01:00
Christoph Kluge 276559d120 revert endpoint change 2026-02-05 15:25:49 +01:00
Christoph Kluge e3148b16eb add timers to updateNodeStates 2026-02-05 15:24:11 +01:00
Aditya Ujeniya 39b8356683 Optimized CCMS healthcheck 2026-02-04 10:24:45 +01:00
moebiusband 0d62a300e7 Intermediate state of node Healthcheck
TODOS:
* Remove error handling from routine and simplify API call
* Use map for hardware level metrics
2026-02-03 18:35:17 +01:00
moebiusband 248f11f4f8 Change API of Node HealthState 2026-02-03 14:55:12 +01:00
moebiusband 00a41373e8 Add monitoring healthstate support in nodestate API. 2026-02-03 12:23:24 +01:00
Christoph Kluge e9cd6b4225 set updateNodeStates timeStamp once per request
-prevents per-host timestamp mismatches due to handler iteration duration
2026-02-02 17:51:41 +01:00
Aditya Ujeniya a71341064e Update to MetricStore HealthCheck API 2026-01-30 23:24:16 +01:00
moebiusband 95689e3c99 Add API endpoint for getUsedNodes
Needed by dynamic memory management for external ccms
2026-01-28 07:05:29 +01:00
moebiusband b307e885ce feat: Add support for multiple external metric stores 2026-01-27 10:02:07 +01:00
moebiusband 0ea836c69c Revert metricstore api paths 2026-01-26 13:17:36 +01:00