Commit Graph

2847 Commits

Author SHA1 Message Date
Aditya Ujeniya 650db33b22 docs(metricstore): fix misleading comment in stats fast-path test 2026-07-20 10:47:33 +02:00
Aditya Ujeniya 9d5268a828 fix(metricstore): guard stats fast path against empty buffer
Add len(b.data) > 0 guard to prevent infinite loop when an empty linked
buffer is reachable. With the guard false, the normal path handles empty
buffers safely via the existing t < b.start || idx >= len(b.data) check.

Add regression test TestStatsFastPathThenPartialTail to pin the boundary:
three-buffer chain where earlier buffers are fully covered (fast path) and
the last buffer is partially covered (normal path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 10:43:43 +02:00
Aditya Ujeniya 9eb6d0e83e feat(metricstore): serve full-buffer stats from cached aggregate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 10:30:42 +02:00
Aditya Ujeniya b6a9f7ed3c feat(metricstore): add recomputeStats full-scan helper 2026-07-20 10:26:30 +02:00
Aditya Ujeniya 5ff376c742 style(metricstore): gofmt stats_test comment alignment 2026-07-20 10:25:16 +02:00
Aditya Ujeniya 931a2a6e79 feat(metricstore): maintain running buffer stats on write 2026-07-20 10:22:16 +02:00
Aditya Ujeniya aa61ccb88b feat(metricstore): add running-stat fields to buffer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 10:17:12 +02:00
Aditya Ujeniya c509a0104b change resampling policy points 2026-07-20 09:43:18 +02:00
Aditya Ujeniya 662e86b45d chore: ignore local agent and docs working directories 2026-07-08 15:45:24 +02:00
Aditya Ujeniya 94799aca36 fix: drop unconditional force-free of buffers retained for running jobs
MemoryUsageTracker freed all buffers older than the retention cutoff on
every tick whenever the last retention pass had excluded used nodes,
regardless of memory pressure — discarding the history kept in memory
for long-running jobs. Buffers are now only reclaimed by the existing
emergency path once memory usage exceeds the configured cap. The
now-unused selectorsExcluded state is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:45:24 +02:00
Aditya Ujeniya dd2f5968b1 feat: pass NodeProvider to metricstore Init
Setting the provider after Init left the checkpoint restore inside Init
without job information and forced callers to pre-call InitMetrics and
SetNodeProvider in the right order. Init now receives the provider as a
parameter and injects it before the restore, so the ordering is enforced
by the signature. SetNodeProvider remains for callers that do not run
Init (tests, -cleanup-checkpoints).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:44:54 +02:00
Aditya Ujeniya ea2bfbb6aa fix: wire NodeProvider into CLI checkpoint cleanup path
The -cleanup-checkpoints CLI flag called CleanupCheckpoints without
initializing the MemoryStore singleton or setting its NodeProvider,
so it silently fell back to legacy clean-everything behavior and
could delete checkpoints for hosts with running jobs. Inject the job
repository as NodeProvider the same way runServer does, so the CLI
path also skips used hosts. Also update SetNodeProvider's doc comment,
which only mentioned Free but is now also consulted by FromCheckpoint
and CleanupCheckpoints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 14:31:26 +02:00
Aditya Ujeniya 33a1bd3529 fix: set NodeProvider before metricstore Init so checkpoint load sees it 2026-07-08 14:18:03 +02:00
Aditya Ujeniya 58c71da0b8 feat: skip checkpoint cleanup for nodes with running jobs 2026-07-08 14:13:15 +02:00
Aditya Ujeniya 1e416a6dd0 feat: load full checkpoint history for nodes with running jobs 2026-07-08 14:05:56 +02:00
Aditya Ujeniya 97ab828d60 feat: add isNodeUsed helper for NodeProvider lookups
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 14:01:37 +02:00
moebiusband 84841a7006 fix(deps): update Go dependencies
Bump gqlgen, aws-sdk-go-v2, go-oidc, go-sqlite3, gqlparser, go-openapi,
golang.org/x/* and other transitive dependencies to their latest patch
and minor releases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 964ca6774882
2026-06-18 10:29:20 +02:00
moebiusband 6f1c36099b fix(web): adapt metric plots to policy-based resample config
The backend resample config changed from {trigger, resolutions[]} to a
policy/targetPoints model, but several components still spread
resampleConfig.resolutions, throwing "resolutions is not iterable" and
breaking the job list render. Default the initial resolution to null (the
backend resolves it from the configured policy; zoom overrides it) and
drop the now-obsolete resolution selector and admin display fields.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 27516ce259fd
2026-06-18 10:29:12 +02:00
moebiusband d89f526eb2 feat(config): warn on unrecognized top-level config sections
A setting nested at the wrong level (e.g. "resampling" placed next to
"main" instead of inside it) was silently ignored. Emit a startup warning
for any top-level config section not consumed by the backend so such
misconfigurations surface instead of failing silently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 69d1b85c8db3
2026-06-18 10:28:59 +02:00
moebiusband ffbe171327 Merge branch 'main' into feature/526-average-resample 2026-06-18 07:17:41 +02:00
Jan Eitzinger 1bd3f25371 Merge pull request #562 from ClusterCockpit/feature/517-make-legal-links-configurable
feat(web): make footer legal links configurable
2026-06-17 18:35:25 +02:00
moebiusband 411bc9b317 feat(web): make footer legal links configurable
Add a "main.footer-links" config option so the footer Imprint and
Privacy Policy links can point at internal pages (default) or external
URLs. External http(s) targets open in a new tab; empty/unset values
fall back to the built-in /imprint and /privacy routes, keeping the
existing ./var/*.tmpl override mechanism intact.

Closes #517

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: c5dca72c848f
2026-06-17 17:29:50 +02:00
Jan Eitzinger 9d1efcd55d Merge pull request #561 from ClusterCockpit/feature-283-remove-env-support-alt
feat(auth): replace .env/godotenv secret handling with config-based s…
2026-06-17 15:28:40 +02:00
moebiusband 3562bfa3aa Rewording of init README 2026-06-17 15:24:13 +02:00
moebiusband 83d04dff17 feat(auth): replace .env/godotenv secret handling with config-based secrets
Secrets (JWT keys, LDAP sync password, OIDC client id/secret, cross-login
keys) are now configured directly in config.json under the auth section
where they are used. Each secret can still be supplied via its existing
environment variable, which takes precedence over the config value.

The godotenv dependency, the .env file, configs/env-template.txt and the
loadEnvironment() bootstrap step are removed. -init now writes the demo
JWT keys into config.json instead of a .env file.

Closes #283

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3a7cb814c53f
2026-06-17 12:28:17 +02:00
Jan Eitzinger 07b9a57479 Merge pull request #559 from ClusterCockpit/feature/558-replace-gorilla-sessions
Feature/558 replace gorilla sessions
2026-06-17 10:05:07 +02:00
moebiusband b7f597bb7d Update entire config 2026-06-17 07:58:29 +02:00
moebiusband 2b01b57495 feat: replace gorilla/sessions with alexedwards/scs/v2
Browser sessions are now server-side, stored in the SQLite database via
scs/sqlite3store (new `sessions` table, DB migration to version 12) instead
of gorilla/sessions client-side cookie storage. Only an opaque random token
is kept in the cookie; session data lives server-side and survives restarts.

Session middleware is wired as a hybrid to avoid buffering large responses:
scs.LoadAndSave on the login/logout write paths, and a non-buffering
read-only LoadSession middleware on the secured/config/frontend read paths
so the large GraphQL /query responses stream unbuffered. JWT-only APIs
(/api, /userapi, /api/metricstore) and static files are left unwrapped.

The session cookie Secure flag is now derived from the server config (set
when cc-backend terminates TLS itself); previously it was effectively never
set. The SESSION_KEY env var is removed as server-side tokens need no
signing secret. The dormant Bearer-JWT branch in the frontend urql client
is removed; the web UI authenticates GraphQL via the session cookie.

Closes #558

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b51075f43cc7
2026-06-17 07:54:26 +02:00
moebiusband 5d8d4e228e Merge branch 'main' into feature/526-average-resample 2026-06-17 06:30:54 +02:00
Jan Eitzinger 3bfd3d06ca Merge pull request #557 from ClusterCockpit/release/v1.5
Release/v1.5
2026-06-07 08:21:14 +02:00
moebiusband 9c6075ebb5 Update README to reflect main branch naming 2026-06-07 08:18:46 +02:00
moebiusband af7528c8b2 Update CLAUDE.md
Entire-Checkpoint: 306db138cb4c
2026-06-07 08:16:10 +02:00
moebiusband 01fb4d53f1 Fix broken link in README 2026-06-07 08:12:25 +02:00
Jan Eitzinger c94f5918f3 Merge pull request #556 from ClusterCockpit/release/v1.5
Fix critical/severe issues in init, startup and shutdown
v1.5.4
2026-06-07 07:31:02 +02:00
moebiusband 1b72b0b5ad Fix critical/severe issues in init, startup and shutdown
- auth: do not abort the server when authentication is disabled. auth.Init
  is now always called; with disable-authentication it sets up an ephemeral
  session store (SESSION_KEY not required) and registers no authenticators,
  so the unconditional auth.GetAuthInstance() callers (server init,
  api.New()) always get a valid instance.
- main: run the graceful-shutdown sequence on the startup-error path. runServer
  derives a cancelable context and, on a server-start failure, cancels it and
  waits so the metricstore final checkpoint / WAL rotation, archiver flush and
  taskmanager shutdown actually run before exit.
- server: log the :80 HTTP->HTTPS redirect listener error instead of dropping it.
- archiver: guard Shutdown against being called when Start never ran
  (avoids close(nil) panic / blocking on a nil workerDone).
- nats API: stop worker goroutines on shutdown via a stop channel + idempotent
  Shutdown(); workers and subscription callbacks select on stop and the
  channels are never closed, so no send-on-closed-channel can occur. Wired
  into Server.Shutdown after the NATS client is closed.
- metricstore: make Shutdown idempotent (nil shutdownFunc, early return) and
  release shutdownFuncMu before the checkpoint write.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3c179f9caa8f
2026-06-05 10:16:28 +02:00
Jan Eitzinger dd8c0e1b9f Merge pull request #555 from ClusterCockpit/release/v1.5
Update ReleaseNotes
2026-06-05 09:05:17 +02:00
moebiusband 56ae1de011 Update ReleaseNotes
Entire-Checkpoint: 051ff3c0d253
2026-06-05 09:01:47 +02:00
Jan Eitzinger 879f730876 Merge pull request #554 from ClusterCockpit/release/v1.5
Fix bug that prevents WAL cleanup on shutdown
2026-06-05 08:42:38 +02:00
moebiusband 8733497223 Fix bug that prevents WAL cleanup on shutdown
Entire-Checkpoint: 5b79628feea4
2026-06-05 08:39:49 +02:00
Jan Eitzinger d74ae77c8e Merge pull request #553 from ClusterCockpit/release/v1.5
Release/v1.5
2026-06-04 20:33:22 +02:00
moebiusband 6cfa511348 Update release notes
Entire-Checkpoint: 14328c112325
2026-06-04 20:19:09 +02:00
moebiusband 3bef199cbe Regenerate GraphQL 2026-06-04 20:12:27 +02:00
moebiusband 16942f55a0 Fix medium-severity issues from follow-up security audit
Addresses the remaining medium findings from the second-pass audit:

- DoS hardening: bound GraphQL query cost with FixedComplexityLimit, and
  reject non-positive items-per-page / page values so uint64 conversion
  cannot underflow into an unbounded LIMIT/OFFSET. The -1 "load all"
  sentinel stays valid for dashboards; REST now returns 400 for bad input.

- Security headers: add X-Content-Type-Options, X-Frame-Options,
  Referrer-Policy and a conservative CSP (frame-ancestors/object-src/
  base-uri) that hardens against clickjacking and base-tag injection
  without restricting the self-hosted SPA's inline scripts.

- Stored XSS: render job.metaData.message as escaped text instead of
  {@html ...} in Job.root and JobFootprint, preserving line breaks via
  white-space: pre-wrap.

- SQL injection hardening: parameterize the tag-scope IN list and the
  manager project subquery in CountTags instead of interpolating
  user.Username / user.Projects (externally sourced via OIDC/LDAP).

- CSRF defense-in-depth: reject cross-site state-changing requests via
  Sec-Fetch-Site, failing open for non-browser clients, on top of the
  existing SameSite=Lax session cookie.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: de7d47a85c7c
2026-06-04 20:08:41 +02:00
moebiusband 6d86690c76 Fix critical issues from follow-up security audit
A second-pass audit surfaced three severe issues missed by the previous
review, each a sibling code path of a bug class that was only partially
fixed before:

- auth: JWT session login (jwtSession.go) registered its authenticator
  even when CROSS_LOGIN_JWT_HS512_KEY was unset, leaving an empty HMAC
  key. golang-jwt verifies any HS256/HS512 signature against an empty
  key, allowing unauthenticated admin token forgery. Init() now refuses
  to register without a key, with a defense-in-depth empty-key guard in
  the keyfunc.

- repository: metric names from GraphQL ([String!]) were interpolated
  raw into json_extract(footprint, "$.<name>") SQL. SQLite parses
  double-quoted strings as literals, enabling SQL injection by any
  authenticated user. Validate metric names against ^[a-zA-Z0-9_]+$ in
  jobsMetricStatisticsHistogram and buildFloatJSONCondition.

- metricstore: cluster/host line-protocol tags flowed unvalidated into
  path.Join(RootDir, cluster, host) for checkpoint/WAL files, allowing
  arbitrary file write outside the checkpoint root via NATS
  (unauthenticated) or POST /api/write. Reject path-traversal sequences
  in DecodeLine before the tags become path components.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b57246993ec1
2026-06-04 19:07:20 +02:00
moebiusband 6f7e262f3f Fix issues after security audit
Entire-Checkpoint: bc18358a9343
2026-06-04 18:33:30 +02:00
moebiusband 58ead40112 Merge branch 'main' into release/v1.5 2026-06-04 17:56:41 +02:00
moebiusband 0020f63582 Rebuild Swagger 2026-06-04 17:56:32 +02:00
Jan Eitzinger cdb4f06fea Merge pull request #552 from ClusterCockpit/fix/add-user-edit-api
Reintroduce user update api path
2026-06-04 08:27:23 +02:00
Christoph Kluge 1ebde74774 Adapt swagger definitions of user update endpoint 2026-06-02 17:58:15 +02:00
Christoph Kluge 40722d72f5 fix name in doc comment 2026-06-02 16:48:23 +02:00