6 Commits

Author SHA1 Message Date
Jan Eitzinger
9d1efcd55d Merge pull request #561 from ClusterCockpit/feature-283-remove-env-support-alt
feat(auth): replace .env/godotenv secret handling with config-based s…
2026-06-17 15:28:40 +02:00
3562bfa3aa Rewording of init README 2026-06-17 15:24:13 +02:00
83d04dff17 feat(auth): replace .env/godotenv secret handling with config-based secrets
Secrets (JWT keys, LDAP sync password, OIDC client id/secret, cross-login
keys) are now configured directly in config.json under the auth section
where they are used. Each secret can still be supplied via its existing
environment variable, which takes precedence over the config value.

The godotenv dependency, the .env file, configs/env-template.txt and the
loadEnvironment() bootstrap step are removed. -init now writes the demo
JWT keys into config.json instead of a .env file.

Closes #283

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3a7cb814c53f
2026-06-17 12:28:17 +02:00
Jan Eitzinger
07b9a57479 Merge pull request #559 from ClusterCockpit/feature/558-replace-gorilla-sessions
Feature/558 replace gorilla sessions
2026-06-17 10:05:07 +02:00
b7f597bb7d Update entire config 2026-06-17 07:58:29 +02:00
2b01b57495 feat: replace gorilla/sessions with alexedwards/scs/v2
Browser sessions are now server-side, stored in the SQLite database via
scs/sqlite3store (new `sessions` table, DB migration to version 12) instead
of gorilla/sessions client-side cookie storage. Only an opaque random token
is kept in the cookie; session data lives server-side and survives restarts.

Session middleware is wired as a hybrid to avoid buffering large responses:
scs.LoadAndSave on the login/logout write paths, and a non-buffering
read-only LoadSession middleware on the secured/config/frontend read paths
so the large GraphQL /query responses stream unbuffered. JWT-only APIs
(/api, /userapi, /api/metricstore) and static files are left unwrapped.

The session cookie Secure flag is now derived from the server config (set
when cc-backend terminates TLS itself); previously it was effectively never
set. The SESSION_KEY env var is removed as server-side tokens need no
signing secret. The dormant Bearer-JWT branch in the frontend urql client
is removed; the web UI authenticates GraphQL via the session cookie.

Closes #558

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b51075f43cc7
2026-06-17 07:54:26 +02:00
54 changed files with 454 additions and 896 deletions

1
.entire/.gitignore vendored
View File

@@ -2,3 +2,4 @@ tmp/
settings.local.json
metadata/
logs/
redactors/local/

View File

@@ -161,9 +161,15 @@ applied automatically on startup. Version tracking in `version` table.
- `username`: Authentication username (optional)
- `password`: Authentication password (optional)
- `creds-file-path`: Path to NATS credentials file (optional)
- **.env**: Environment variables (secrets like JWT keys)
- Copy from `configs/env-template.txt`
- NEVER commit this file
- **Secrets** (JWT keys, LDAP sync password, OIDC client id/secret, cross-login
keys): configured directly in `config.json` under the `auth` section where they
are used (e.g. `auth.jwts.public-key`, `auth.jwts.private-key`,
`auth.ldap.sync-password`, `auth.oidc.client-id`/`client-secret`).
- Each secret may also be supplied via its environment variable
(`JWT_PUBLIC_KEY`, `JWT_PRIVATE_KEY`, `LDAP_ADMIN_PASSWORD`, `OID_CLIENT_ID`,
`OID_CLIENT_SECRET`, `CROSS_LOGIN_JWT_PUBLIC_KEY`, `CROSS_LOGIN_JWT_HS512_KEY`).
- The environment variable takes precedence over the value in `config.json`.
- The former `.env`/godotenv mechanism has been removed.
- **cluster.json**: Cluster topology and metric definitions (loaded from archive or config)
## Database

View File

@@ -1,6 +1,6 @@
TARGET = ./cc-backend
FRONTEND = ./web/frontend
VERSION = 1.5.4
VERSION = 1.6.0
GIT_HASH := $(shell git rev-parse --short HEAD || echo 'development')
CURRENT_TIME = $(shell date +"%Y-%m-%d:T%H:%M:%S")
LD_FLAGS = '-s -X main.date=${CURRENT_TIME} -X main.version=${VERSION} -X main.commit=${GIT_HASH}'

View File

@@ -129,12 +129,11 @@ git clone https://github.com/ClusterCockpit/cc-backend.git
cd ./cc-backend/
make
# EDIT THE .env FILE BEFORE YOU DEPLOY (Change the secrets)!
# If authentication is disabled, it can be empty.
cp configs/env-template.txt .env
vim .env
cp configs/config.json .
# EDIT config.json BEFORE YOU DEPLOY: change the secrets under "auth.jwts"
# ("public-key"/"private-key"). Each secret can also be supplied via an
# environment variable (e.g. JWT_PUBLIC_KEY), which takes precedence over the
# value in config.json.
vim config.json
#Optional: Link an existing job archive:
@@ -152,6 +151,27 @@ ln -s <your-existing-job-archive> ./var/job-archive
./cc-backend -help
```
### Authentication and sessions
Browser sessions are stored server-side in the SQLite database (the `sessions`
table) using [`alexedwards/scs`](https://github.com/alexedwards/scs); only an
opaque random token is kept in the session cookie. No cookie-signing secret is
required, so the former `SESSION_KEY` environment variable is no longer used.
Secrets (JWT keys, LDAP sync password, OIDC client id/secret, cross-login keys)
are configured directly in `config.json` under the `auth` section. Each secret
may also be supplied via its environment variable (e.g. `JWT_PUBLIC_KEY`,
`JWT_PRIVATE_KEY`, `LDAP_ADMIN_PASSWORD`, `OID_CLIENT_ID`, `OID_CLIENT_SECRET`,
`CROSS_LOGIN_JWT_PUBLIC_KEY`, `CROSS_LOGIN_JWT_HS512_KEY`); the environment
variable takes precedence when set. The previous `.env` file is no longer used.
The session cookie's `Secure` flag is set automatically when cc-backend serves
HTTPS itself (i.e. `https-cert-file` and `https-key-file` are configured in
`config.json`); otherwise it is left unset so that plain-HTTP development works.
For production deployments, serve cc-backend over HTTPS so the session cookie is
marked `Secure`. If you terminate TLS at a reverse proxy, prefer letting
cc-backend serve HTTPS directly for now so the flag is applied.
## Database Configuration
cc-backend uses SQLite as its database. For large installations, SQLite memory

View File

@@ -1,10 +1,48 @@
# `cc-backend` version 1.5.4
# `cc-backend` version 1.6.0
Supports job archive version 3 and database version 12.
This release replaces the browser session implementation and requires a database
migration to version 12. Run `./cc-backend -migrate-db` after upgrading; the new
`sessions` table is created automatically (a fresh `-init-db` also creates it).
Existing login sessions are invalidated by the upgrade, so users have to log in
again once. See the behavior changes below regarding the session cookie `Secure`
flag and the removal of the `SESSION_KEY` environment variable.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).
## Changes in 1.6.0
### Behavior changes
- **Session backend replaced (`gorilla/sessions``alexedwards/scs`)**: Browser
sessions are now managed by `github.com/alexedwards/scs/v2` with server-side
storage in the SQLite database (new `sessions` table, database version 12)
instead of `gorilla/sessions` client-side cookie storage. Only an opaque random
token is stored in the cookie; the session payload lives server-side. Sessions
still survive backend restarts. This requires the database migration noted above
and invalidates existing sessions.
- **Session cookie `Secure` flag**: The `Secure` attribute on the session cookie
is now derived from the server configuration — it is set when cc-backend
terminates TLS itself (`https-cert-file` and `https-key-file` configured) and
unset otherwise. Previously the flag was effectively never set. Deployments that
terminate TLS at a reverse proxy and want the `Secure` flag should serve
cc-backend over HTTPS directly for now.
- **`SESSION_KEY` removed**: The `SESSION_KEY` environment variable is no longer
used and should be removed from your `.env`. Server-side sessions use random
tokens, so no cookie-signing secret is required. It is ignored if left in place.
### Dependencies
- **Added** `github.com/alexedwards/scs/v2` and
`github.com/alexedwards/scs/sqlite3store`.
- **Removed** `github.com/gorilla/sessions` (and `github.com/gorilla/securecookie`).
## Changes in 1.5.4
Supports job archive version 3 and database version 11.
This is a security and bugfix release of `cc-backend`, the API backend and
This was a security and bugfix release of `cc-backend`, the API backend and
frontend implementation of ClusterCockpit.
For release specific notes visit the [ClusterCockpit Documentation](https://clusterockpit.org/docs/release/).
If you are upgrading from v1.5.1 no database migration is required.
If you are upgrading from v1.5.0 you need to do another DB migration. This
should not take long. For optimal database performance after the migration it is
@@ -15,8 +53,6 @@ While we are confident that the memory issue with the metricstore cleanup move
policy is fixed, it is still recommended to use delete policy for cleanup.
This is also the default.
## Changes in 1.5.4
### Security fixes
- **JWT HMAC empty-key bypass (critical)**: `jwtSession.go` now refuses to

View File

@@ -250,12 +250,6 @@ type TimeWeights {
coreHours: [NullableFloat!]!
}
enum ResampleAlgo {
LTTB
AVERAGE
SIMPLE
}
enum Aggregate {
USER
PROJECT
@@ -346,7 +340,6 @@ type Query {
metrics: [String!]
scopes: [MetricScope!]
resolution: Int
resampleAlgo: ResampleAlgo
): [JobMetricWithName!]!
jobStats(id: ID!, metrics: [String!]): [NamedStats!]!
@@ -406,7 +399,6 @@ type Query {
to: Time!
page: PageRequest
resolution: Int
resampleAlgo: ResampleAlgo
): NodesResultList!
clusterMetrics(

View File

@@ -17,7 +17,7 @@ var (
)
func cliInit() {
flag.BoolVar(&flagInit, "init", false, "Setup var directory, initialize sqlite database file, config.json and .env")
flag.BoolVar(&flagInit, "init", false, "Setup var directory, initialize sqlite database file and config.json")
flag.BoolVar(&flagReinitDB, "init-db", false, "Go through job-archive and re-initialize the 'job', 'tag', and 'jobtag' tables (all running jobs will be lost!)")
flag.BoolVar(&flagSyncLDAP, "sync-ldap", false, "Sync the 'hpc_user' table with ldap")
flag.BoolVar(&flagServer, "server", false, "Start a server, continues listening on port after initialization and argument handling")

View File

@@ -18,24 +18,18 @@ import (
"github.com/ClusterCockpit/cc-lib/v2/util"
)
const envString = `
# Base64 encoded Ed25519 keys (DO NOT USE THESE TWO IN PRODUCTION!)
# You can generate your own keypair using the gen-keypair tool
JWT_PUBLIC_KEY="kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
JWT_PRIVATE_KEY="dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
# Some random bytes used as secret for cookie-based sessions (DO NOT USE THIS ONE IN PRODUCTION)
SESSION_KEY="67d829bf61dc5f87a73fd814e2c9f629"
`
const configString = `
{
"main": {
"addr": "127.0.0.1:8080",
"short-running-jobs-duration": 300,
"resampling": {
"default-policy": "medium",
"default-algo": "lttb"
"minimum-points": 600,
"trigger": 300,
"resolutions": [
240,
60
]
},
"api-allowed-ips": [
"*"
@@ -53,7 +47,9 @@ const configString = `
},
"auth": {
"jwts": {
"max-age": "2000h"
"max-age": "2000h",
"public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0=",
"private-key": "dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
}
}
}
@@ -68,10 +64,6 @@ func initEnv() {
cclog.Abortf("Could not write default ./config.json with permissions '0o666'. Application initialization failed, exited.\nError: %s\n", err.Error())
}
if err := os.WriteFile(".env", []byte(envString), 0o666); err != nil {
cclog.Abortf("Could not write default ./.env file with permissions '0o666'. Application initialization failed, exited.\nError: %s\n", err.Error())
}
if err := os.Mkdir("var", 0o777); err != nil {
cclog.Abortf("Could not create default ./var folder with permissions '0o777'. Application initialization failed, exited.\nError: %s\n", err.Error())
}

View File

@@ -39,7 +39,6 @@ import (
"github.com/ClusterCockpit/cc-lib/v2/schema"
"github.com/ClusterCockpit/cc-lib/v2/util"
"github.com/google/gops/agent"
"github.com/joho/godotenv"
_ "github.com/mattn/go-sqlite3"
)
@@ -89,13 +88,6 @@ func initGops() error {
return nil
}
func loadEnvironment() error {
if err := godotenv.Load(); err != nil {
return fmt.Errorf("loading .env file: %w", err)
}
return nil
}
func initConfiguration() error {
ccconf.Init(flagConfigFile)
@@ -224,7 +216,14 @@ func checkDefaultSecurityKeys() {
// Default JWT public key from init.go
defaultJWTPublic := "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
if os.Getenv("JWT_PUBLIC_KEY") == defaultJWTPublic {
// Resolve the public key the same way the authenticators do: environment
// variable takes precedence over the value configured in config.json.
pubKey := os.Getenv("JWT_PUBLIC_KEY")
if pubKey == "" && auth.Keys.JwtConfig != nil {
pubKey = auth.Keys.JwtConfig.PublicKey
}
if pubKey == defaultJWTPublic {
cclog.Warn("Using default JWT keys - not recommended for production environments")
}
}
@@ -495,7 +494,7 @@ func run() error {
if flagInit {
initEnv()
cclog.Exit("Successfully setup environment!\n" +
"Please review config.json and .env and adjust it to your needs.\n" +
"Please review config.json and adjust it to your needs.\n" +
"Add your job-archive at ./var/job-archive.")
}
@@ -505,17 +504,12 @@ func run() error {
}
// Initialize subsystems in dependency order:
// 1. Load environment variables from .env file (contains sensitive configuration)
// 2. Load configuration from config.json (may reference environment variables)
// 3. Handle database migration commands if requested
// 4. Initialize database connection (requires config for connection string)
// 5. Handle user commands if requested (requires database and authentication config)
// 6. Initialize subsystems like archive and metrics (require config and database)
// Load environment and configuration
if err := loadEnvironment(); err != nil {
return err
}
// 1. Load configuration from config.json (secrets live in config; individual
// secrets may be overridden via environment variables)
// 2. Handle database migration commands if requested
// 3. Initialize database connection (requires config for connection string)
// 4. Handle user commands if requested (requires database and authentication config)
// 5. Initialize subsystems like archive and metrics (require config and database)
if err := initConfiguration(); err != nil {
return err

View File

@@ -121,6 +121,7 @@ func (s *Server) init() error {
}
authHandle := auth.GetAuthInstance()
sessionManager := authHandle.SessionManager()
// Middleware must be defined before routes in chi
s.router.Use(func(next http.Handler) http.Handler {
@@ -220,10 +221,12 @@ func (s *Server) init() error {
})
}
s.router.Post("/login", authHandle.Login(loginFailureHandler).ServeHTTP)
s.router.HandleFunc("/jwt-login", authHandle.Login(loginFailureHandler).ServeHTTP)
// Login/logout mutate the session, so they are wrapped with
// scs.LoadAndSave, which commits the session and writes the cookie.
s.router.Post("/login", sessionManager.LoadAndSave(authHandle.Login(loginFailureHandler)).ServeHTTP)
s.router.Handle("/jwt-login", sessionManager.LoadAndSave(authHandle.Login(loginFailureHandler)))
s.router.Post("/logout", authHandle.Logout(
s.router.Post("/logout", sessionManager.LoadAndSave(authHandle.Logout(
http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
rw.Header().Add("Content-Type", "text/html; charset=utf-8")
rw.WriteHeader(http.StatusOK)
@@ -234,7 +237,7 @@ func (s *Server) init() error {
Build: buildInfo,
Infos: info,
})
})).ServeHTTP)
}))).ServeHTTP)
}
if flagDev {
@@ -246,6 +249,10 @@ func (s *Server) init() error {
// Secured routes (require authentication)
s.router.Group(func(secured chi.Router) {
if !config.Keys.DisableAuthentication {
// Non-buffering session load: makes the session available to
// AuthViaSession without wrapping/buffering the (potentially large,
// e.g. GraphQL /query) response.
secured.Use(authHandle.LoadSession)
secured.Use(func(next http.Handler) http.Handler {
return authHandle.Auth(
next,
@@ -309,6 +316,7 @@ func (s *Server) init() error {
// the /config page route that is registered in the secured group)
s.router.Group(func(configapi chi.Router) {
if !config.Keys.DisableAuthentication {
configapi.Use(authHandle.LoadSession)
configapi.Use(func(next http.Handler) http.Handler {
return authHandle.AuthConfigAPI(next, onFailureResponse)
})
@@ -319,6 +327,7 @@ func (s *Server) init() error {
// Frontend API routes
s.router.Route("/frontend", func(frontendapi chi.Router) {
if !config.Keys.DisableAuthentication {
frontendapi.Use(authHandle.LoadSession)
frontendapi.Use(func(next http.Handler) http.Handler {
return authHandle.AuthFrontendAPI(next, onFailureResponse)
})

View File

@@ -9,7 +9,9 @@
},
"auth": {
"jwts": {
"max-age": "2000h"
"max-age": "2000h",
"public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0=",
"private-key": "dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
}
},
"metric-store-external": [

View File

@@ -14,8 +14,9 @@
"target-path": "./var/nodestate-archive"
},
"resampling": {
"default-policy": "medium",
"default-algo": "lttb"
"minimum-points": 600,
"trigger": 180,
"resolutions": [240, 60]
},
"api-subjects": {
"subject-job-event": "cc.job.event",
@@ -29,7 +30,9 @@
},
"auth": {
"jwts": {
"max-age": "2000h"
"max-age": "2000h",
"public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0=",
"private-key": "dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
}
},
"cron": {

View File

@@ -1,14 +0,0 @@
# Base64 encoded Ed25519 keys (DO NOT USE THESE TWO IN PRODUCTION!)
# You can generate your own keypair using `go run tools/gen-keypair/main.go`
JWT_PUBLIC_KEY="kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
JWT_PRIVATE_KEY="dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
# Base64 encoded Ed25519 public key for accepting externally generated JWTs
# Keys in PEM format can be converted, see `tools/convert-pem-pubkey/Readme.md`
CROSS_LOGIN_JWT_PUBLIC_KEY=""
# Some random bytes used as secret for cookie-based sessions (DO NOT USE THIS ONE IN PRODUCTION)
SESSION_KEY="67d829bf61dc5f87a73fd814e2c9f629"
# Password for the ldap server (optional)
LDAP_ADMIN_PASSWORD="mashup"

5
go.mod
View File

@@ -12,6 +12,8 @@ require (
github.com/ClusterCockpit/cc-lib/v2 v2.12.0
github.com/ClusterCockpit/cc-line-protocol/v2 v2.4.0
github.com/Masterminds/squirrel v1.5.4
github.com/alexedwards/scs/sqlite3store v0.0.0-20251002162104-209de6e426de
github.com/alexedwards/scs/v2 v2.9.0
github.com/aws/aws-sdk-go-v2 v1.41.7
github.com/aws/aws-sdk-go-v2/config v1.32.18
github.com/aws/aws-sdk-go-v2/credentials v1.19.17
@@ -25,9 +27,7 @@ require (
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/golang-migrate/migrate/v4 v4.19.1
github.com/google/gops v0.3.29
github.com/gorilla/sessions v1.4.0
github.com/jmoiron/sqlx v1.4.0
github.com/joho/godotenv v1.5.1
github.com/mattn/go-sqlite3 v1.14.44
github.com/parquet-go/parquet-go v0.30.1
github.com/qustavo/sqlhooks/v2 v2.1.0
@@ -80,7 +80,6 @@ require (
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/influxdata/influxdb-client-go/v2 v2.14.0 // indirect

13
go.sum
View File

@@ -27,6 +27,10 @@ github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e h1:4dAU9FXIyQktpoUAgOJK3OTFc/xug0PCXYCqU0FgDKI=
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
github.com/alexedwards/scs/sqlite3store v0.0.0-20251002162104-209de6e426de h1:c72K9HLu6K442et0j3BUL/9HEYaUJouLkkVANdmqTOo=
github.com/alexedwards/scs/sqlite3store v0.0.0-20251002162104-209de6e426de/go.mod h1:Iyk7S76cxGaiEX/mSYmTZzYehp4KfyylcLaV3OnToss=
github.com/alexedwards/scs/v2 v2.9.0 h1:xa05mVpwTBm1iLeTMNFfAWpKUm4fXAW7CeAViqBVS90=
github.com/alexedwards/scs/v2 v2.9.0/go.mod h1:ToaROZxyKukJKT/xLcVQAChi5k6+Pn1Gvmdl7h3RRj8=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNgfBlViaCIJKLlCJ6/fmUseuG0wVQ=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
@@ -153,18 +157,12 @@ github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gops v0.3.29 h1:n98J2qSOK1NJvRjdLDcjgDryjpIBGhbaqph1mXKL0rY=
github.com/google/gops v0.3.29/go.mod h1:8N3jZftuPazvUwtYY/ncG4iPrjp15ysNKLfq+QQPiwc=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA=
github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ=
github.com/gorilla/sessions v1.4.0/go.mod h1:FLWm50oby91+hl7p/wRxDth9bWSuk0qVL2emc7lT5ik=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
@@ -193,8 +191,6 @@ github.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZ
github.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=
github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o=
github.com/jmoiron/sqlx v1.4.0/go.mod h1:ZrZ7UsYB/weZdl2Bxg6jCRO9c3YHl8r3ahlKmRT4JLY=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I=
github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60=
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
@@ -212,6 +208,7 @@ github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/mattn/go-sqlite3 v1.14.6/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mattn/go-sqlite3 v1.14.44 h1:3VSe+xafpbzsLbdr2AWlAZk9yRHiBhTBakioXaCKTF8=
github.com/mattn/go-sqlite3 v1.14.44/go.mod h1:pjEuOr8IwzLJP2MfGeTb0A35jauH+C2kbHKBr7yXKVQ=

View File

@@ -1,19 +1,19 @@
# How to run `cc-backend` as a systemd service.
# How to run `cc-backend` as a systemd service
The files in this directory assume that you install ClusterCockpit to
`/opt/monitoring/cc-backend`.
Of course you can choose any other location, but make sure you replace all paths
starting with `/opt/monitoring/cc-backend` in the `clustercockpit.service` file!
The `config.json` may contain the optional fields *user* and *group*. If
The `config.json` may contain the optional fields _user_ and _group_. If
specified, the application will call
[setuid](https://man7.org/linux/man-pages/man2/setuid.2.html) and
[setgid](https://man7.org/linux/man-pages/man2/setgid.2.html) after reading the
config file and binding to a TCP port (so it can take a privileged port), but
before it starts accepting any connections. This is good for security, but also
means that the `var/` directory must be readable and writeable by this user.
The `.env` and `config.json` files may contain secrets and should not be
readable by this user. If these files are changed, the server must be restarted.
means that the `var/` directory must be readable and writeable by this user. The
`config.json` file may contain secrets. If this file is changed, the server must
be restarted.
```sh
# 1. Clone this repository somewhere in your home
@@ -25,11 +25,9 @@ make
sudo mkdir -p /opt/monitoring/cc-backend/
cp ./cc-backend /opt/monitoring/cc-backend/
# 3. Modify the `./config.json` and env-template.txt file from the configs directory to your liking and put it in the target directory
# 3. Modify the `./config.json` file from the configs directory to your liking and put it in the target directory
cp ./configs/config.json /opt/monitoring/config.json
cp ./configs/env-template.txt /opt/monitoring/.env
vim /opt/monitoring/config.json # do your thing...
vim /opt/monitoring/.env # do your thing...
vim /opt/monitoring/config.json # do your thing (including the secrets under "auth")...
# 4. (Optional) Customization: Add your versions of the login view, legal texts, and logo image.
# You may use the templates in `./web/templates` as blueprint. Every overwrite separate.
@@ -57,8 +55,9 @@ It is recommended to install all ClusterCockpit components in a common directory
In the following we use `/opt/monitoring`.
Two systemd services run on the central monitoring server:
* clustercockpit : binary cc-backend in `/opt/monitoring/cc-backend`.
* cc-metric-store : Binary cc-metric-store in `/opt/monitoring/cc-metric-store`.
- clustercockpit : binary cc-backend in `/opt/monitoring/cc-backend`.
- cc-metric-store : Binary cc-metric-store in `/opt/monitoring/cc-metric-store`.
ClusterCockpit is deployed as a single binary that embeds all static assets.
We recommend keeping all `cc-backend` binary versions in a folder `archive` and
@@ -68,12 +67,13 @@ This allows for easy roll-back in case something doesn't work.
## Workflow to deploy new version
This example assumes the DB and job archive versions did not change.
* Stop systemd service: `$ sudo systemctl stop clustercockpit.service`
* Backup the sqlite DB file and Job archive directory tree!
* Copy `cc-backend` binary to `/opt/monitoring/cc-backend/archive` (Tip: Use a
date tag like `YYYYMMDD-cc-backend`)
* Link from cc-backend root to current version
* Start systemd service: `$ sudo systemctl start clustercockpit.service`
* Check if everything is ok: `$ sudo systemctl status clustercockpit.service`
* Check log for issues: `$ sudo journalctl -u clustercockpit.service`
* Check the ClusterCockpit web frontend and your Slurm adapters if anything is broken!
- Stop systemd service: `$ sudo systemctl stop clustercockpit.service`
- Backup the sqlite DB file and Job archive directory tree!
- Copy `cc-backend` binary to `/opt/monitoring/cc-backend/archive` (Tip: Use a
date tag like `YYYYMMDD-cc-backend`)
- Link from cc-backend root to current version
- Start systemd service: `$ sudo systemctl start clustercockpit.service`
- Check if everything is ok: `$ sudo systemctl status clustercockpit.service`
- Check log for issues: `$ sudo journalctl -u clustercockpit.service`
- Check the ClusterCockpit web frontend and your Slurm adapters if anything is broken!

View File

@@ -170,7 +170,6 @@ func setup(t *testing.T) *api.RestAPI {
archiver.Start(repository.GetJobRepository(), context.Background())
t.Setenv("SESSION_KEY", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
if cfg := ccconf.GetPackageConfig("auth"); cfg != nil {
auth.Init(&cfg)
} else {
@@ -356,7 +355,7 @@ func TestRestApi(t *testing.T) {
}
t.Run("CheckArchive", func(t *testing.T) {
data, err := metricdispatch.LoadData(stoppedJob, []string{"load_one"}, []schema.MetricScope{schema.MetricScopeNode}, context.Background(), 60, "")
data, err := metricdispatch.LoadData(stoppedJob, []string{"load_one"}, []schema.MetricScope{schema.MetricScopeNode}, context.Background(), 60)
if err != nil {
t.Fatal(err)
}

View File

@@ -309,7 +309,7 @@ func (api *RestAPI) getCompleteJobByID(rw http.ResponseWriter, r *http.Request)
}
if r.URL.Query().Get("all-metrics") == "true" {
data, err = metricdispatch.LoadData(job, nil, scopes, r.Context(), resolution, "")
data, err = metricdispatch.LoadData(job, nil, scopes, r.Context(), resolution)
if err != nil {
cclog.Warnf("REST: error while loading all-metrics job data for JobID %d on %s", job.JobID, job.Cluster)
return
@@ -405,7 +405,7 @@ func (api *RestAPI) getJobByID(rw http.ResponseWriter, r *http.Request) {
resolution = max(resolution, mc.Timestep)
}
data, err := metricdispatch.LoadData(job, metrics, scopes, r.Context(), resolution, "")
data, err := metricdispatch.LoadData(job, metrics, scopes, r.Context(), resolution)
if err != nil {
cclog.Warnf("REST: error while loading job data for JobID %d on %s", job.JobID, job.Cluster)
return
@@ -1086,7 +1086,7 @@ func (api *RestAPI) getJobMetrics(rw http.ResponseWriter, r *http.Request) {
}
resolver := graph.GetResolverInstance()
data, err := resolver.Query().JobMetrics(r.Context(), id, metrics, scopes, nil, nil)
data, err := resolver.Query().JobMetrics(r.Context(), id, metrics, scopes, nil)
if err != nil {
if err := json.NewEncoder(rw).Encode(Response{
Error: &struct {

View File

@@ -156,7 +156,6 @@ func setupNatsTest(t *testing.T) *NatsAPI {
archiver.Start(repository.GetJobRepository(), context.Background())
t.Setenv("SESSION_KEY", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
if cfg := ccconf.GetPackageConfig("auth"); cfg != nil {
auth.Init(&cfg)
} else {

View File

@@ -59,7 +59,7 @@ func ArchiveJob(job *schema.Job, ctx context.Context) (*schema.Job, error) {
scopes = append(scopes, schema.MetricScopeAccelerator)
}
jobData, err := metricdispatch.LoadData(job, allMetrics, scopes, ctx, 0, "") // 0 Resulotion-Value retrieves highest res (60s)
jobData, err := metricdispatch.LoadData(job, allMetrics, scopes, ctx, 0) // 0 Resulotion-Value retrieves highest res (60s)
if err != nil {
cclog.Error("Error wile loading job data for archiving")
return nil, err

View File

@@ -9,9 +9,8 @@ package auth
import (
"bytes"
"context"
"crypto/rand"
"database/sql"
"encoding/base64"
"encoding/gob"
"encoding/json"
"errors"
"fmt"
@@ -29,7 +28,8 @@ import (
cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
"github.com/ClusterCockpit/cc-lib/v2/schema"
"github.com/ClusterCockpit/cc-lib/v2/util"
"github.com/gorilla/sessions"
"github.com/alexedwards/scs/sqlite3store"
"github.com/alexedwards/scs/v2"
)
// Authenticator is the interface for all authentication methods.
@@ -116,9 +116,21 @@ type AuthConfig struct {
// Keys holds the global authentication configuration
var Keys AuthConfig
// secretFromEnv resolves a secret from the environment or config. The
// environment variable takes precedence when set and non-empty; otherwise the
// value configured in config.json is used. This lets deployments inject secrets
// via the environment (or a secret manager) while keeping config.json
// self-contained for simple setups.
func secretFromEnv(envVar, configValue string) string {
if v := os.Getenv(envVar); v != "" {
return v
}
return configValue
}
// Authentication manages all authentication methods and session handling
type Authentication struct {
sessionStore *sessions.CookieStore
sessionManager *scs.SessionManager
LdapAuth *LdapAuthenticator
JwtAuth *JWTAuthenticator
LocalAuth *LocalAuthenticator
@@ -126,49 +138,80 @@ type Authentication struct {
SessionMaxAge time.Duration
}
// SessionManager exposes the scs session manager so the HTTP router can install
// the session middleware (LoadAndSave on write paths, LoadSession on read paths).
func (auth *Authentication) SessionManager() *scs.SessionManager {
return auth.sessionManager
}
// LoadSession is a non-buffering, read-only session middleware. It loads any
// existing session into the request context so AuthViaSession can read it, but
// (unlike scs.LoadAndSave) it never wraps the ResponseWriter and never commits,
// so large responses (e.g. the GraphQL /query endpoint) stream without being
// buffered in memory. Use it on routes that only read the session to
// authenticate; use scs.LoadAndSave on the login/logout routes that mutate it.
func (auth *Authentication) LoadSession(next http.Handler) http.Handler {
return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
var token string
if c, err := r.Cookie(auth.sessionManager.Cookie.Name); err == nil {
token = c.Value
}
ctx, err := auth.sessionManager.Load(r.Context(), token)
if err != nil {
cclog.Errorf("session load failed: %s", err.Error())
http.Error(rw, "internal server error", http.StatusInternalServerError)
return
}
next.ServeHTTP(rw, r.WithContext(ctx))
})
}
// expireSessionCookie clears a (corrupted) session cookie on the client. Used on
// read paths where there is no commit to drive the deletion.
func expireSessionCookie(rw http.ResponseWriter) {
http.SetCookie(rw, &http.Cookie{
Name: "session",
Path: "/",
MaxAge: -1,
HttpOnly: true,
SameSite: http.SameSiteLaxMode,
})
}
func (auth *Authentication) AuthViaSession(
rw http.ResponseWriter,
r *http.Request,
) (*schema.User, error) {
session, err := auth.sessionStore.Get(r, "session")
if err != nil {
cclog.Error("Error while getting session store")
return nil, err
}
if session.IsNew {
// The session data was loaded into the request context by the LoadSession
// middleware. No active session cookie => not logged in (mirrors session.IsNew).
ctx := r.Context()
if !auth.sessionManager.Exists(ctx, "username") {
return nil, nil
}
// Validate session data with proper type checking
username, ok := session.Values["username"].(string)
if !ok || username == "" {
username := auth.sessionManager.GetString(ctx, "username")
if username == "" {
cclog.Warn("Invalid session: missing or invalid username")
// Invalidate the corrupted session
session.Options.MaxAge = -1
_ = auth.sessionStore.Save(r, rw, session)
expireSessionCookie(rw)
return nil, errors.New("invalid session data")
}
projects, ok := session.Values["projects"].([]string)
projects, ok := auth.sessionManager.Get(ctx, "projects").([]string)
if !ok {
cclog.Warn("Invalid session: projects not found or invalid type, using empty list")
projects = []string{}
}
roles, ok := session.Values["roles"].([]string)
roles, ok := auth.sessionManager.Get(ctx, "roles").([]string)
if !ok || len(roles) == 0 {
cclog.Warn("Invalid session: missing or invalid roles")
// Invalidate the corrupted session
session.Options.MaxAge = -1
_ = auth.sessionStore.Save(r, rw, session)
expireSessionCookie(rw)
return nil, errors.New("invalid session data")
}
authSourceInt, ok := session.Values["authSource"].(int)
if !ok {
authSourceInt = int(schema.AuthViaLocalPassword)
}
// GetInt returns 0 (== schema.AuthViaLocalPassword) when the key is absent.
authSourceInt := auth.sessionManager.GetInt(ctx, "authSource")
return &schema.User{
Username: username,
@@ -186,31 +229,30 @@ func Init(authCfg *json.RawMessage) {
// Start background cleanup of rate limiters
startRateLimiterCleanup()
sessKey := os.Getenv("SESSION_KEY")
if sessKey == "" {
if !config.Keys.DisableAuthentication {
cclog.Fatal("environment variable 'SESSION_KEY' not set: refusing to start with an ephemeral session key. " +
"Set SESSION_KEY in .env (base64-encoded 32 random bytes); a random key would invalidate all sessions on every restart " +
"and prevent sessions from validating across replicas.")
}
// Authentication is disabled: no user sessions are issued, so an
// ephemeral random key is sufficient and SESSION_KEY is not required.
ephemeralKey := make([]byte, 32)
if _, err := rand.Read(ephemeralKey); err != nil {
cclog.Fatalf("Error while initializing authentication -> generating ephemeral session key failed: %v", err)
}
authInstance.sessionStore = sessions.NewCookieStore(ephemeralKey)
} else {
keyBytes, err := base64.StdEncoding.DecodeString(sessKey)
if err != nil {
cclog.Fatal("Error while initializing authentication -> decoding session key failed")
}
authInstance.sessionStore = sessions.NewCookieStore(keyBytes)
}
// Server-side sessions via scs, persisted in the existing SQLite DB so
// sessions survive restarts. Only an opaque random token is stored in the
// cookie, so no secret signing key (the former SESSION_KEY) is required.
gob.Register([]string{}) // user.Projects / user.Roles are stored as []string
sm := scs.New()
sm.Store = sqlite3store.New(repository.GetConnection().DB.DB)
sm.Cookie.Name = "session"
sm.Cookie.Path = "/"
sm.Cookie.HttpOnly = true
sm.Cookie.SameSite = http.SameSiteLaxMode
// scs sets Secure globally (no per-request option). Enable it when this
// process terminates TLS itself. Deployments terminating TLS at a reverse
// proxy can set this via a future config flag if needed.
sm.Cookie.Secure = config.Keys.HTTPSCertFile != ""
if d, err := time.ParseDuration(config.Keys.SessionMaxAge); err == nil {
if d, err := time.ParseDuration(config.Keys.SessionMaxAge); err == nil && d != 0 {
sm.Lifetime = d
authInstance.SessionMaxAge = d
} else {
// SessionMaxAge of 0/empty means "do not expire": approximate with a
// long absolute lifetime (the cookie remains persistent).
sm.Lifetime = 10 * 365 * 24 * time.Hour
}
authInstance.sessionManager = sm
// When authentication is disabled no authenticators are required; the
// session store created above is enough for the server to run with a
@@ -319,36 +361,22 @@ func handleLdapUser(ldapUser *schema.User) {
}
func (auth *Authentication) SaveSession(rw http.ResponseWriter, r *http.Request, user *schema.User) error {
session, err := auth.sessionStore.New(r, "session")
if err != nil {
cclog.Errorf("session creation failed: %s", err.Error())
// The login routes are wrapped by scs.LoadAndSave, which loaded the session
// into the request context and will commit it (persist to the store and write
// the Set-Cookie header) after the handler returns.
ctx := r.Context()
// Generate a new session token to prevent session fixation.
if err := auth.sessionManager.RenewToken(ctx); err != nil {
cclog.Errorf("session renew failed: %s", err.Error())
http.Error(rw, err.Error(), http.StatusInternalServerError)
return err
}
if auth.SessionMaxAge != 0 {
session.Options.MaxAge = int(auth.SessionMaxAge.Seconds())
}
if r.TLS == nil && r.Header.Get("X-Forwarded-Proto") != "https" {
// If neither TLS or an encrypted reverse proxy are used, do not mark cookies as secure.
cclog.Warn("Authenticating with unencrypted request. Session cookies will not have Secure flag set (insecure for production)")
if r.Header.Get("X-Forwarded-Proto") == "" {
// This warning will not be printed if e.g. X-Forwarded-Proto == http
cclog.Warn("If you are using a reverse proxy, make sure X-Forwarded-Proto is set")
}
session.Options.Secure = false
}
session.Options.SameSite = http.SameSiteLaxMode
session.Options.HttpOnly = true
session.Values["username"] = user.Username
session.Values["projects"] = user.Projects
session.Values["roles"] = user.Roles
session.Values["authSource"] = int(user.AuthSource)
if err := auth.sessionStore.Save(r, rw, session); err != nil {
cclog.Warnf("session save failed: %s", err.Error())
http.Error(rw, err.Error(), http.StatusInternalServerError)
return err
}
auth.sessionManager.Put(ctx, "username", user.Username)
auth.sessionManager.Put(ctx, "projects", user.Projects)
auth.sessionManager.Put(ctx, "roles", user.Roles)
auth.sessionManager.Put(ctx, "authSource", int(user.AuthSource))
return nil
}
@@ -609,20 +637,13 @@ func (auth *Authentication) AuthFrontendAPI(
func (auth *Authentication) Logout(onsuccess http.Handler) http.Handler {
return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
session, err := auth.sessionStore.Get(r, "session")
if err != nil {
// The logout route is wrapped by scs.LoadAndSave: Destroy removes the
// session from the store and the middleware clears the cookie on the way out.
if err := auth.sessionManager.Destroy(r.Context()); err != nil {
http.Error(rw, err.Error(), http.StatusInternalServerError)
return
}
if !session.IsNew {
session.Options.MaxAge = -1
if err := auth.sessionStore.Save(r, rw, session); err != nil {
http.Error(rw, err.Error(), http.StatusInternalServerError)
return
}
}
onsuccess.ServeHTTP(rw, r)
})
}

View File

@@ -10,7 +10,6 @@ import (
"encoding/base64"
"errors"
"net/http"
"os"
"strings"
"time"
@@ -39,6 +38,23 @@ type JWTAuthConfig struct {
// Should an existent user be updated in the DB based on the information in the token
UpdateUserOnLogin bool `json:"update-user-on-login"`
// Base64 encoded Ed25519 public key used to validate JWTs.
// Overridden by the JWT_PUBLIC_KEY environment variable when set.
PublicKey string `json:"public-key"`
// Base64 encoded Ed25519 private key used to sign JWTs.
// Overridden by the JWT_PRIVATE_KEY environment variable when set.
PrivateKey string `json:"private-key"`
// Base64 encoded Ed25519 public key for accepting externally generated JWTs.
// Overridden by the CROSS_LOGIN_JWT_PUBLIC_KEY environment variable when set.
CrossLoginPublicKey string `json:"cross-login-public-key"`
// Base64 encoded HMAC (HS256/HS512) key for accepting externally generated
// session login tokens.
// Overridden by the CROSS_LOGIN_JWT_HS512_KEY environment variable when set.
CrossLoginHS512Key string `json:"cross-login-hs512-key"`
}
type JWTAuthenticator struct {
@@ -47,9 +63,10 @@ type JWTAuthenticator struct {
}
func (ja *JWTAuthenticator) Init() error {
pubKey, privKey := os.Getenv("JWT_PUBLIC_KEY"), os.Getenv("JWT_PRIVATE_KEY")
pubKey := secretFromEnv("JWT_PUBLIC_KEY", Keys.JwtConfig.PublicKey)
privKey := secretFromEnv("JWT_PRIVATE_KEY", Keys.JwtConfig.PrivateKey)
if pubKey == "" || privKey == "" {
cclog.Warn("environment variables 'JWT_PUBLIC_KEY' or 'JWT_PRIVATE_KEY' not set (token based authentication will not work)")
cclog.Warn("JWT public/private key not configured ('public-key'/'private-key' in config or 'JWT_PUBLIC_KEY'/'JWT_PRIVATE_KEY' env): token based authentication will not work")
} else {
bytes, err := base64.StdEncoding.DecodeString(pubKey)
if err != nil {
@@ -121,7 +138,7 @@ func (ja *JWTAuthenticator) AuthViaJWT(
// ProvideJWT generates a new JWT that can be used for authentication
func (ja *JWTAuthenticator) ProvideJWT(user *schema.User) (string, error) {
if ja.privateKey == nil {
return "", errors.New("environment variable 'JWT_PRIVATE_KEY' not set")
return "", errors.New("JWT private key not configured ('private-key' in config or 'JWT_PRIVATE_KEY' env)")
}
now := time.Now()

View File

@@ -11,7 +11,6 @@ import (
"errors"
"fmt"
"net/http"
"os"
cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
"github.com/ClusterCockpit/cc-lib/v2/schema"
@@ -27,10 +26,11 @@ type JWTCookieSessionAuthenticator struct {
var _ Authenticator = (*JWTCookieSessionAuthenticator)(nil)
func (ja *JWTCookieSessionAuthenticator) Init() error {
pubKey, privKey := os.Getenv("JWT_PUBLIC_KEY"), os.Getenv("JWT_PRIVATE_KEY")
pubKey := secretFromEnv("JWT_PUBLIC_KEY", Keys.JwtConfig.PublicKey)
privKey := secretFromEnv("JWT_PRIVATE_KEY", Keys.JwtConfig.PrivateKey)
if pubKey == "" || privKey == "" {
cclog.Warn("environment variables 'JWT_PUBLIC_KEY' or 'JWT_PRIVATE_KEY' not set (token based authentication will not work)")
return errors.New("environment variables 'JWT_PUBLIC_KEY' or 'JWT_PRIVATE_KEY' not set (token based authentication will not work)")
cclog.Warn("JWT public/private key not configured ('public-key'/'private-key' in config or 'JWT_PUBLIC_KEY'/'JWT_PRIVATE_KEY' env): token based authentication will not work")
return errors.New("JWT public/private key not configured: token based authentication will not work")
} else {
bytes, err := base64.StdEncoding.DecodeString(pubKey)
if err != nil {
@@ -47,8 +47,8 @@ func (ja *JWTCookieSessionAuthenticator) Init() error {
}
// Look for external public keys
pubKeyCrossLogin, keyFound := os.LookupEnv("CROSS_LOGIN_JWT_PUBLIC_KEY")
if keyFound && pubKeyCrossLogin != "" {
pubKeyCrossLogin := secretFromEnv("CROSS_LOGIN_JWT_PUBLIC_KEY", Keys.JwtConfig.CrossLoginPublicKey)
if pubKeyCrossLogin != "" {
bytes, err := base64.StdEncoding.DecodeString(pubKeyCrossLogin)
if err != nil {
cclog.Warn("Could not decode cross login JWT public key")
@@ -57,8 +57,8 @@ func (ja *JWTCookieSessionAuthenticator) Init() error {
ja.publicKeyCrossLogin = ed25519.PublicKey(bytes)
} else {
ja.publicKeyCrossLogin = nil
cclog.Debug("environment variable 'CROSS_LOGIN_JWT_PUBLIC_KEY' not set (cross login token based authentication will not work)")
return errors.New("environment variable 'CROSS_LOGIN_JWT_PUBLIC_KEY' not set (cross login token based authentication will not work)")
cclog.Debug("cross login JWT public key not configured ('cross-login-public-key' in config or 'CROSS_LOGIN_JWT_PUBLIC_KEY' env): cross login token based authentication will not work")
return errors.New("cross login JWT public key not configured: cross login token based authentication will not work")
}
// Warn if other necessary settings are not configured

View File

@@ -10,7 +10,6 @@ import (
"errors"
"fmt"
"net/http"
"os"
"strings"
cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
@@ -25,12 +24,12 @@ type JWTSessionAuthenticator struct {
var _ Authenticator = (*JWTSessionAuthenticator)(nil)
func (ja *JWTSessionAuthenticator) Init() error {
pubKey := os.Getenv("CROSS_LOGIN_JWT_HS512_KEY")
pubKey := secretFromEnv("CROSS_LOGIN_JWT_HS512_KEY", Keys.JwtConfig.CrossLoginHS512Key)
if pubKey == "" {
// Without a configured key the HMAC verification below would run against
// an empty key, which lets anyone forge a valid token. Refuse to register
// the authenticator in that case so JWT session login is simply disabled.
return errors.New("CROSS_LOGIN_JWT_HS512_KEY not set: JWT session login disabled")
return errors.New("cross login HS512 key not configured ('cross-login-hs512-key' in config or 'CROSS_LOGIN_JWT_HS512_KEY' env): JWT session login disabled")
}
bytes, err := base64.StdEncoding.DecodeString(pubKey)

View File

@@ -9,7 +9,6 @@ import (
"fmt"
"net"
"net/http"
"os"
"strings"
"time"
@@ -33,6 +32,10 @@ type LdapConfig struct {
// Should a non-existent user be added to the DB if user exists in ldap directory
SyncUserOnLogin bool `json:"sync-user-on-login"`
UpdateUserOnLogin bool `json:"update-user-on-login"`
// Password for the LDAP admin account used for syncing (optional).
// Overridden by the LDAP_ADMIN_PASSWORD environment variable when set.
SyncPassword string `json:"sync-password"`
}
type LdapAuthenticator struct {
@@ -44,9 +47,9 @@ type LdapAuthenticator struct {
var _ Authenticator = (*LdapAuthenticator)(nil)
func (la *LdapAuthenticator) Init() error {
la.syncPassword = os.Getenv("LDAP_ADMIN_PASSWORD")
la.syncPassword = secretFromEnv("LDAP_ADMIN_PASSWORD", Keys.LdapConfig.SyncPassword)
if la.syncPassword == "" {
cclog.Warn("environment variable 'LDAP_ADMIN_PASSWORD' not set (ldap sync will not work)")
cclog.Warn("LDAP admin password not configured ('sync-password' in config or 'LDAP_ADMIN_PASSWORD' env): ldap sync will not work")
}
if Keys.LdapConfig.UserAttr != "" {

View File

@@ -12,7 +12,6 @@ import (
"fmt"
"io"
"net/http"
"os"
"time"
"github.com/ClusterCockpit/cc-backend/internal/repository"
@@ -27,6 +26,14 @@ type OpenIDConfig struct {
Provider string `json:"provider"`
SyncUserOnLogin bool `json:"sync-user-on-login"`
UpdateUserOnLogin bool `json:"update-user-on-login"`
// OAuth2 client ID for the OIDC provider.
// Overridden by the OID_CLIENT_ID environment variable when set.
ClientID string `json:"client-id"`
// OAuth2 client secret for the OIDC provider.
// Overridden by the OID_CLIENT_SECRET environment variable when set.
ClientSecret string `json:"client-secret"`
}
type OIDC struct {
@@ -66,13 +73,13 @@ func NewOIDC(a *Authentication) *OIDC {
if err != nil {
cclog.Fatal(err)
}
clientID := os.Getenv("OID_CLIENT_ID")
clientID := secretFromEnv("OID_CLIENT_ID", Keys.OpenIDConfig.ClientID)
if clientID == "" {
cclog.Warn("environment variable 'OID_CLIENT_ID' not set (Open ID connect auth will not work)")
cclog.Warn("OIDC client ID not configured ('client-id' in config or 'OID_CLIENT_ID' env): Open ID connect auth will not work")
}
clientSecret := os.Getenv("OID_CLIENT_SECRET")
clientSecret := secretFromEnv("OID_CLIENT_SECRET", Keys.OpenIDConfig.ClientSecret)
if clientSecret == "" {
cclog.Warn("environment variable 'OID_CLIENT_SECRET' not set (Open ID connect auth will not work)")
cclog.Warn("OIDC client secret not configured ('client-secret' in config or 'OID_CLIENT_SECRET' env): Open ID connect auth will not work")
}
client := &oauth2.Config{

View File

@@ -34,6 +34,22 @@ var configSchema = `
"update-user-on-login": {
"description": "Should an existent user attributes in the DB be updated at login attempt with values provided in JWT.",
"type": "boolean"
},
"public-key": {
"description": "Base64 encoded Ed25519 public key used to validate JWTs. Overridden by the JWT_PUBLIC_KEY environment variable when set.",
"type": "string"
},
"private-key": {
"description": "Base64 encoded Ed25519 private key used to sign JWTs. Overridden by the JWT_PRIVATE_KEY environment variable when set.",
"type": "string"
},
"cross-login-public-key": {
"description": "Base64 encoded Ed25519 public key for accepting externally generated JWTs. Overridden by the CROSS_LOGIN_JWT_PUBLIC_KEY environment variable when set.",
"type": "string"
},
"cross-login-hs512-key": {
"description": "Base64 encoded HMAC (HS256/HS512) key for accepting externally generated session login tokens. Overridden by the CROSS_LOGIN_JWT_HS512_KEY environment variable when set.",
"type": "string"
}
},
"required": ["max-age"]
@@ -52,6 +68,14 @@ var configSchema = `
"update-user-on-login": {
"description": "Should an existent user attributes in the DB be updated at login attempt with values provided.",
"type": "boolean"
},
"client-id": {
"description": "OAuth2 client ID for the OIDC provider. Overridden by the OID_CLIENT_ID environment variable when set.",
"type": "string"
},
"client-secret": {
"description": "OAuth2 client secret for the OIDC provider. Overridden by the OID_CLIENT_SECRET environment variable when set.",
"type": "string"
}
},
"required": ["provider"]
@@ -103,6 +127,10 @@ var configSchema = `
"update-user-on-login": {
"description": "Should an existent user attributes in the DB be updated at login attempt with values from LDAP.",
"type": "boolean"
},
"sync-password": {
"description": "Password for the LDAP admin account used for syncing. Overridden by the LDAP_ADMIN_PASSWORD environment variable when set.",
"type": "string"
}
},
"required": ["url", "user-base", "search-dn", "user-bind", "user-filter"]

View File

@@ -24,7 +24,7 @@ type ProgramConfig struct {
APISubjects *NATSConfig `json:"api-subjects"`
// Drop root permissions once .env was read and the port was taken.
// Drop root permissions once the config was read and the port was taken.
User string `json:"user"`
Group string `json:"group"`
@@ -106,12 +106,12 @@ type NodeStateRetention struct {
}
type ResampleConfig struct {
// Default resample policy when no user preference is set ("low", "medium", "high")
DefaultPolicy string `json:"default-policy"`
// Default resample algorithm when no user preference is set ("lttb", "average", "simple")
DefaultAlgo string `json:"default-algo"`
// Policy-derived target point count (set dynamically from user preference, not from config.json)
TargetPoints int `json:"targetPoints,omitempty"`
// Minimum number of points to trigger resampling of data
MinimumPoints int `json:"minimum-points"`
// Array of resampling target resolutions, in seconds; Example: [600,300,60]
Resolutions []int `json:"resolutions"`
// Trigger next zoom level at less than this many visible datapoints
Trigger int `json:"trigger"`
}
type NATSConfig struct {
@@ -155,24 +155,7 @@ func Init(mainConfig json.RawMessage) {
cclog.Abortf("Config Init: Could not decode config file '%s'.\nError: %s\n", mainConfig, err.Error())
}
if Keys.EnableResampling != nil {
policy := Keys.EnableResampling.DefaultPolicy
if policy == "" {
policy = "medium"
}
resampler.SetMinimumRequiredPoints(targetPointsForPolicy(policy))
}
}
func targetPointsForPolicy(policy string) int {
switch policy {
case "low":
return 200
case "medium":
return 500
case "high":
return 1000
default:
return 500
if Keys.EnableResampling != nil && Keys.EnableResampling.MinimumPoints > 0 {
resampler.SetMinimumRequiredPoints(Keys.EnableResampling.MinimumPoints)
}
}

View File

@@ -21,11 +21,11 @@ var configSchema = `
}
},
"user": {
"description": "Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.",
"description": "Drop root permissions once the config was read and the port was taken. Only applicable if using privileged port.",
"type": "string"
},
"group": {
"description": "Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.",
"description": "Drop root permissions once the config was read and the port was taken. Only applicable if using privileged port.",
"type": "string"
},
"disable-authentication": {
@@ -92,17 +92,23 @@ var configSchema = `
"description": "Enable dynamic zoom in frontend metric plots.",
"type": "object",
"properties": {
"default-policy": {
"description": "Default resample policy when no user preference is set.",
"type": "string",
"enum": ["low", "medium", "high"]
"minimum-points": {
"description": "Minimum points to trigger resampling of time-series data.",
"type": "integer"
},
"default-algo": {
"description": "Default resample algorithm when no user preference is set.",
"type": "string",
"enum": ["lttb", "average", "simple"]
"trigger": {
"description": "Trigger next zoom level at less than this many visible datapoints.",
"type": "integer"
},
"resolutions": {
"description": "Array of resampling target resolutions, in seconds.",
"type": "array",
"items": {
"type": "integer"
}
}
}
},
"required": ["trigger", "resolutions"]
},
"api-subjects": {
"description": "NATS subjects configuration for subscribing to job and node events.",

View File

@@ -327,7 +327,7 @@ type ComplexityRoot struct {
Clusters func(childComplexity int) int
GlobalMetrics func(childComplexity int) int
Job func(childComplexity int, id string) int
JobMetrics func(childComplexity int, id string, metrics []string, scopes []schema.MetricScope, resolution *int, resampleAlgo *model.ResampleAlgo) int
JobMetrics func(childComplexity int, id string, metrics []string, scopes []schema.MetricScope, resolution *int) int
JobStats func(childComplexity int, id string, metrics []string) int
Jobs func(childComplexity int, filter []*model.JobFilter, page *model.PageRequest, order *model.OrderByInput) int
JobsFootprints func(childComplexity int, filter []*model.JobFilter, metrics []string) int
@@ -335,7 +335,7 @@ type ComplexityRoot struct {
JobsStatistics func(childComplexity int, filter []*model.JobFilter, metrics []string, page *model.PageRequest, sortBy *model.SortByAggregate, groupBy *model.Aggregate, numDurationBins *string, numMetricBins *int) int
Node func(childComplexity int, id string) int
NodeMetrics func(childComplexity int, cluster string, nodes []string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time) int
NodeMetricsList func(childComplexity int, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int, resampleAlgo *model.ResampleAlgo) int
NodeMetricsList func(childComplexity int, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int) int
NodeStates func(childComplexity int, filter []*model.NodeFilter) int
NodeStatesTimed func(childComplexity int, filter []*model.NodeFilter, typeArg string) int
Nodes func(childComplexity int, filter []*model.NodeFilter, order *model.OrderByInput) int
@@ -483,7 +483,7 @@ type QueryResolver interface {
NodeStates(ctx context.Context, filter []*model.NodeFilter) ([]*model.NodeStates, error)
NodeStatesTimed(ctx context.Context, filter []*model.NodeFilter, typeArg string) ([]*model.NodeStatesTimed, error)
Job(ctx context.Context, id string) (*schema.Job, error)
JobMetrics(ctx context.Context, id string, metrics []string, scopes []schema.MetricScope, resolution *int, resampleAlgo *model.ResampleAlgo) ([]*model.JobMetricWithName, error)
JobMetrics(ctx context.Context, id string, metrics []string, scopes []schema.MetricScope, resolution *int) ([]*model.JobMetricWithName, error)
JobStats(ctx context.Context, id string, metrics []string) ([]*model.NamedStats, error)
ScopedJobStats(ctx context.Context, id string, metrics []string, scopes []schema.MetricScope) ([]*model.NamedStatsWithScope, error)
Jobs(ctx context.Context, filter []*model.JobFilter, page *model.PageRequest, order *model.OrderByInput) (*model.JobResultList, error)
@@ -492,7 +492,7 @@ type QueryResolver interface {
JobsFootprints(ctx context.Context, filter []*model.JobFilter, metrics []string) (*model.Footprints, error)
RooflineHeatmap(ctx context.Context, filter []*model.JobFilter, rows int, cols int, minX float64, minY float64, maxX float64, maxY float64) ([][]float64, error)
NodeMetrics(ctx context.Context, cluster string, nodes []string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time) ([]*model.NodeMetrics, error)
NodeMetricsList(ctx context.Context, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int, resampleAlgo *model.ResampleAlgo) (*model.NodesResultList, error)
NodeMetricsList(ctx context.Context, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int) (*model.NodesResultList, error)
ClusterMetrics(ctx context.Context, cluster string, metrics []string, from time.Time, to time.Time) (*model.ClusterMetrics, error)
}
type SubClusterResolver interface {
@@ -1666,7 +1666,7 @@ func (e *executableSchema) Complexity(ctx context.Context, typeName, field strin
return 0, false
}
return e.ComplexityRoot.Query.JobMetrics(childComplexity, args["id"].(string), args["metrics"].([]string), args["scopes"].([]schema.MetricScope), args["resolution"].(*int), args["resampleAlgo"].(*model.ResampleAlgo)), true
return e.ComplexityRoot.Query.JobMetrics(childComplexity, args["id"].(string), args["metrics"].([]string), args["scopes"].([]schema.MetricScope), args["resolution"].(*int)), true
case "Query.jobStats":
if e.ComplexityRoot.Query.JobStats == nil {
break
@@ -1754,7 +1754,7 @@ func (e *executableSchema) Complexity(ctx context.Context, typeName, field strin
return 0, false
}
return e.ComplexityRoot.Query.NodeMetricsList(childComplexity, args["cluster"].(string), args["subCluster"].(string), args["stateFilter"].(string), args["nodeFilter"].(string), args["scopes"].([]schema.MetricScope), args["metrics"].([]string), args["from"].(time.Time), args["to"].(time.Time), args["page"].(*model.PageRequest), args["resolution"].(*int), args["resampleAlgo"].(*model.ResampleAlgo)), true
return e.ComplexityRoot.Query.NodeMetricsList(childComplexity, args["cluster"].(string), args["subCluster"].(string), args["stateFilter"].(string), args["nodeFilter"].(string), args["scopes"].([]schema.MetricScope), args["metrics"].([]string), args["from"].(time.Time), args["to"].(time.Time), args["page"].(*model.PageRequest), args["resolution"].(*int)), true
case "Query.nodeStates":
if e.ComplexityRoot.Query.NodeStates == nil {
break
@@ -2525,12 +2525,6 @@ type TimeWeights {
coreHours: [NullableFloat!]!
}
enum ResampleAlgo {
LTTB
AVERAGE
SIMPLE
}
enum Aggregate {
USER
PROJECT
@@ -2621,7 +2615,6 @@ type Query {
metrics: [String!]
scopes: [MetricScope!]
resolution: Int
resampleAlgo: ResampleAlgo
): [JobMetricWithName!]!
jobStats(id: ID!, metrics: [String!]): [NamedStats!]!
@@ -2681,7 +2674,6 @@ type Query {
to: Time!
page: PageRequest
resolution: Int
resampleAlgo: ResampleAlgo
): NodesResultList!
clusterMetrics(
@@ -3890,11 +3882,6 @@ func (ec *executionContext) field_Query_jobMetrics_args(ctx context.Context, raw
return nil, err
}
args["resolution"] = arg3
arg4, err := graphql.ProcessArgField(ctx, rawArgs, "resampleAlgo", ec.unmarshalOResampleAlgo2ᚖgithubᚗcomᚋClusterCockpitᚋccᚑbackendᚋinternalᚋgraphᚋmodelᚐResampleAlgo)
if err != nil {
return nil, err
}
args["resampleAlgo"] = arg4
return args, nil
}
@@ -4153,11 +4140,6 @@ func (ec *executionContext) field_Query_nodeMetricsList_args(ctx context.Context
return nil, err
}
args["resolution"] = arg9
arg10, err := graphql.ProcessArgField(ctx, rawArgs, "resampleAlgo", ec.unmarshalOResampleAlgo2ᚖgithubᚗcomᚋClusterCockpitᚋccᚑbackendᚋinternalᚋgraphᚋmodelᚐResampleAlgo)
if err != nil {
return nil, err
}
args["resampleAlgo"] = arg10
return args, nil
}
@@ -9325,7 +9307,7 @@ func (ec *executionContext) _Query_jobMetrics(ctx context.Context, field graphql
},
func(ctx context.Context) (any, error) {
fc := graphql.GetFieldContext(ctx)
return ec.Resolvers.Query().JobMetrics(ctx, fc.Args["id"].(string), fc.Args["metrics"].([]string), fc.Args["scopes"].([]schema.MetricScope), fc.Args["resolution"].(*int), fc.Args["resampleAlgo"].(*model.ResampleAlgo))
return ec.Resolvers.Query().JobMetrics(ctx, fc.Args["id"].(string), fc.Args["metrics"].([]string), fc.Args["scopes"].([]schema.MetricScope), fc.Args["resolution"].(*int))
},
nil,
func(ctx context.Context, selections ast.SelectionSet, v []*model.JobMetricWithName) graphql.Marshaler {
@@ -9721,7 +9703,7 @@ func (ec *executionContext) _Query_nodeMetricsList(ctx context.Context, field gr
},
func(ctx context.Context) (any, error) {
fc := graphql.GetFieldContext(ctx)
return ec.Resolvers.Query().NodeMetricsList(ctx, fc.Args["cluster"].(string), fc.Args["subCluster"].(string), fc.Args["stateFilter"].(string), fc.Args["nodeFilter"].(string), fc.Args["scopes"].([]schema.MetricScope), fc.Args["metrics"].([]string), fc.Args["from"].(time.Time), fc.Args["to"].(time.Time), fc.Args["page"].(*model.PageRequest), fc.Args["resolution"].(*int), fc.Args["resampleAlgo"].(*model.ResampleAlgo))
return ec.Resolvers.Query().NodeMetricsList(ctx, fc.Args["cluster"].(string), fc.Args["subCluster"].(string), fc.Args["stateFilter"].(string), fc.Args["nodeFilter"].(string), fc.Args["scopes"].([]schema.MetricScope), fc.Args["metrics"].([]string), fc.Args["from"].(time.Time), fc.Args["to"].(time.Time), fc.Args["page"].(*model.PageRequest), fc.Args["resolution"].(*int))
},
nil,
func(ctx context.Context, selections ast.SelectionSet, v *model.NodesResultList) graphql.Marshaler {
@@ -18697,22 +18679,6 @@ func (ec *executionContext) unmarshalOPageRequest2ᚖgithubᚗcomᚋClusterCockp
return &res, graphql.ErrorOnPath(ctx, err)
}
func (ec *executionContext) unmarshalOResampleAlgo2ᚖgithubᚗcomᚋClusterCockpitᚋccᚑbackendᚋinternalᚋgraphᚋmodelᚐResampleAlgo(ctx context.Context, v any) (*model.ResampleAlgo, error) {
if v == nil {
return nil, nil
}
var res = new(model.ResampleAlgo)
err := res.UnmarshalGQL(v)
return res, graphql.ErrorOnPath(ctx, err)
}
func (ec *executionContext) marshalOResampleAlgo2ᚖgithubᚗcomᚋClusterCockpitᚋccᚑbackendᚋinternalᚋgraphᚋmodelᚐResampleAlgo(ctx context.Context, sel ast.SelectionSet, v *model.ResampleAlgo) graphql.Marshaler {
if v == nil {
return graphql.Null
}
return v
}
func (ec *executionContext) unmarshalOSchedulerState2ᚖgithubᚗcomᚋClusterCockpitᚋccᚑlibᚋv2ᚋschemaᚐSchedulerState(ctx context.Context, v any) (*schema.SchedulerState, error) {
if v == nil {
return nil, nil

View File

@@ -328,63 +328,6 @@ func (e Aggregate) MarshalJSON() ([]byte, error) {
return buf.Bytes(), nil
}
type ResampleAlgo string
const (
ResampleAlgoLttb ResampleAlgo = "LTTB"
ResampleAlgoAverage ResampleAlgo = "AVERAGE"
ResampleAlgoSimple ResampleAlgo = "SIMPLE"
)
var AllResampleAlgo = []ResampleAlgo{
ResampleAlgoLttb,
ResampleAlgoAverage,
ResampleAlgoSimple,
}
func (e ResampleAlgo) IsValid() bool {
switch e {
case ResampleAlgoLttb, ResampleAlgoAverage, ResampleAlgoSimple:
return true
}
return false
}
func (e ResampleAlgo) String() string {
return string(e)
}
func (e *ResampleAlgo) UnmarshalGQL(v any) error {
str, ok := v.(string)
if !ok {
return fmt.Errorf("enums must be strings")
}
*e = ResampleAlgo(str)
if !e.IsValid() {
return fmt.Errorf("%s is not a valid ResampleAlgo", str)
}
return nil
}
func (e ResampleAlgo) MarshalGQL(w io.Writer) {
fmt.Fprint(w, strconv.Quote(e.String()))
}
func (e *ResampleAlgo) UnmarshalJSON(b []byte) error {
s, err := strconv.Unquote(string(b))
if err != nil {
return err
}
return e.UnmarshalGQL(s)
}
func (e ResampleAlgo) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
e.MarshalGQL(&buf)
return buf.Bytes(), nil
}
type SortByAggregate string
const (

View File

@@ -1,145 +0,0 @@
// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package graph
import (
"context"
"strings"
"github.com/ClusterCockpit/cc-backend/internal/config"
"github.com/ClusterCockpit/cc-backend/internal/graph/model"
"github.com/ClusterCockpit/cc-backend/internal/metricdispatch"
"github.com/ClusterCockpit/cc-backend/internal/repository"
"github.com/ClusterCockpit/cc-backend/pkg/archive"
)
// resolveResolutionFromPolicy reads the user's resample policy preference and
// computes a resolution based on job duration and metric frequency. Returns nil
// if the user has no policy set.
func resolveResolutionFromPolicy(ctx context.Context, duration int64, cluster string, metrics []string) *int {
user := repository.GetUserFromContext(ctx)
if user == nil {
return nil
}
conf, err := repository.GetUserCfgRepo().GetUIConfig(user)
if err != nil {
return nil
}
policyVal, ok := conf["plotConfiguration_resamplePolicy"]
if !ok {
return nil
}
policyStr, ok := policyVal.(string)
if !ok || policyStr == "" {
return nil
}
policy := metricdispatch.ResamplePolicy(policyStr)
targetPoints := metricdispatch.TargetPointsForPolicy(policy)
if targetPoints == 0 {
return nil
}
// Find the smallest metric frequency across the requested metrics
frequency := smallestFrequency(cluster, metrics)
if frequency <= 0 {
return nil
}
res := metricdispatch.ComputeResolution(duration, int64(frequency), targetPoints)
return &res
}
// resolveResampleAlgo returns the resampling algorithm name to use, checking
// the explicit GraphQL parameter first, then the user's preference.
func resolveResampleAlgo(ctx context.Context, resampleAlgo *model.ResampleAlgo) string {
if resampleAlgo != nil {
return strings.ToLower(resampleAlgo.String())
}
user := repository.GetUserFromContext(ctx)
if user == nil {
return ""
}
conf, err := repository.GetUserCfgRepo().GetUIConfig(user)
if err != nil {
return ""
}
algoVal, ok := conf["plotConfiguration_resampleAlgo"]
if ok {
if algoStr, ok := algoVal.(string); ok && algoStr != "" {
return algoStr
}
}
// Fall back to global default algo
if config.Keys.EnableResampling != nil && config.Keys.EnableResampling.DefaultAlgo != "" {
return config.Keys.EnableResampling.DefaultAlgo
}
return ""
}
// resolveResolutionFromDefaultPolicy computes a resolution using the global
// default policy from config. Returns nil if no policy is configured.
func resolveResolutionFromDefaultPolicy(duration int64, cluster string, metrics []string) *int {
cfg := config.Keys.EnableResampling
if cfg == nil {
return nil
}
policyStr := cfg.DefaultPolicy
if policyStr == "" {
policyStr = "medium"
}
policy := metricdispatch.ResamplePolicy(policyStr)
targetPoints := metricdispatch.TargetPointsForPolicy(policy)
if targetPoints == 0 {
return nil
}
frequency := smallestFrequency(cluster, metrics)
if frequency <= 0 {
return nil
}
res := metricdispatch.ComputeResolution(duration, int64(frequency), targetPoints)
return &res
}
// smallestFrequency returns the smallest metric timestep (in seconds) among the
// requested metrics for the given cluster. Falls back to 0 if nothing is found.
func smallestFrequency(cluster string, metrics []string) int {
cl := archive.GetCluster(cluster)
if cl == nil {
return 0
}
minFreq := 0
for _, mc := range cl.MetricConfig {
if len(metrics) > 0 {
found := false
for _, m := range metrics {
if mc.Name == m {
found = true
break
}
}
if !found {
continue
}
}
if minFreq == 0 || mc.Timestep < minFreq {
minFreq = mc.Timestep
}
}
return minFreq
}

View File

@@ -498,30 +498,24 @@ func (r *queryResolver) Job(ctx context.Context, id string) (*schema.Job, error)
}
// JobMetrics is the resolver for the jobMetrics field.
func (r *queryResolver) JobMetrics(ctx context.Context, id string, metrics []string, scopes []schema.MetricScope, resolution *int, resampleAlgo *model.ResampleAlgo) ([]*model.JobMetricWithName, error) {
func (r *queryResolver) JobMetrics(ctx context.Context, id string, metrics []string, scopes []schema.MetricScope, resolution *int) ([]*model.JobMetricWithName, error) {
if resolution == nil { // Load from Config
if config.Keys.EnableResampling != nil {
defaultRes := slices.Max(config.Keys.EnableResampling.Resolutions)
resolution = &defaultRes
} else { // Set 0 (Loads configured metric timestep)
defaultRes := 0
resolution = &defaultRes
}
}
job, err := r.Query().Job(ctx, id)
if err != nil {
cclog.Warn("Error while querying job for metrics")
return nil, err
}
// Resolve resolution: explicit param > user policy > global config > 0
if resolution == nil {
resolution = resolveResolutionFromPolicy(ctx, int64(job.Duration), job.Cluster, metrics)
}
if resolution == nil {
if config.Keys.EnableResampling != nil {
resolution = resolveResolutionFromDefaultPolicy(int64(job.Duration), job.Cluster, metrics)
}
if resolution == nil {
defaultRes := 0
resolution = &defaultRes
}
}
algoName := resolveResampleAlgo(ctx, resampleAlgo)
data, err := metricdispatch.LoadData(job, metrics, scopes, ctx, *resolution, algoName)
data, err := metricdispatch.LoadData(job, metrics, scopes, ctx, *resolution)
if err != nil {
cclog.Warn("Error while loading job data")
return nil, err
@@ -883,17 +877,12 @@ func (r *queryResolver) NodeMetrics(ctx context.Context, cluster string, nodes [
}
// NodeMetricsList is the resolver for the nodeMetricsList field.
func (r *queryResolver) NodeMetricsList(ctx context.Context, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int, resampleAlgo *model.ResampleAlgo) (*model.NodesResultList, error) {
// Resolve resolution: explicit param > user policy > global config > 0
duration := int64(to.Sub(from).Seconds())
if resolution == nil {
resolution = resolveResolutionFromPolicy(ctx, duration, cluster, metrics)
}
if resolution == nil {
func (r *queryResolver) NodeMetricsList(ctx context.Context, cluster string, subCluster string, stateFilter string, nodeFilter string, scopes []schema.MetricScope, metrics []string, from time.Time, to time.Time, page *model.PageRequest, resolution *int) (*model.NodesResultList, error) {
if resolution == nil { // Load from Config
if config.Keys.EnableResampling != nil {
resolution = resolveResolutionFromDefaultPolicy(duration, cluster, metrics)
}
if resolution == nil {
defaultRes := slices.Max(config.Keys.EnableResampling.Resolutions)
resolution = &defaultRes
} else { // Set 0 (Loads configured metric timestep)
defaultRes := 0
resolution = &defaultRes
}
@@ -917,10 +906,8 @@ func (r *queryResolver) NodeMetricsList(ctx context.Context, cluster string, sub
}
}
algoName := resolveResampleAlgo(ctx, resampleAlgo)
// data -> map hostname:jobdata
data, err := metricdispatch.LoadNodeListData(cluster, subCluster, nodes, metrics, scopes, *resolution, from, to, ctx, algoName)
data, err := metricdispatch.LoadNodeListData(cluster, subCluster, nodes, metrics, scopes, *resolution, from, to, ctx)
if err != nil {
cclog.Warn("error while loading node data (Resolver.NodeMetricsList")
return nil, err

View File

@@ -55,7 +55,7 @@ func (r *queryResolver) rooflineHeatmap(
// resolution = max(resolution, mc.Timestep)
// }
jobdata, err := metricdispatch.LoadData(job, []string{"flops_any", "mem_bw"}, []schema.MetricScope{schema.MetricScopeNode}, ctx, 0, "")
jobdata, err := metricdispatch.LoadData(job, []string{"flops_any", "mem_bw"}, []schema.MetricScope{schema.MetricScopeNode}, ctx, 0)
if err != nil {
cclog.Warnf("Error while loading roofline metrics for job %d", *job.ID)
return nil, err

View File

@@ -62,10 +62,9 @@ func cacheKey(
metrics []string,
scopes []schema.MetricScope,
resolution int,
resampleAlgo string,
) string {
return fmt.Sprintf("%d(%s):[%v],[%v]-%d-%s",
*job.ID, job.State, metrics, scopes, resolution, resampleAlgo)
return fmt.Sprintf("%d(%s):[%v],[%v]-%d",
*job.ID, job.State, metrics, scopes, resolution)
}
// LoadData retrieves metric data for a job from the appropriate backend (memory store for running jobs,
@@ -88,9 +87,8 @@ func LoadData(job *schema.Job,
scopes []schema.MetricScope,
ctx context.Context,
resolution int,
resampleAlgo string,
) (schema.JobData, error) {
data := cache.Get(cacheKey(job, metrics, scopes, resolution, resampleAlgo), func() (_ any, ttl time.Duration, size int) {
data := cache.Get(cacheKey(job, metrics, scopes, resolution), func() (_ any, ttl time.Duration, size int) {
var jd schema.JobData
var err error
@@ -138,17 +136,13 @@ func LoadData(job *schema.Job,
jd = deepCopy(jdTemp)
// Resample archived data to reduce data points to the requested resolution,
// improving transfer performance and client-side rendering.
resampleFn, rfErr := resampler.GetResampler(resampleAlgo)
if rfErr != nil {
return rfErr, 0, 0
}
// Resample archived data using Largest Triangle Three Bucket algorithm to reduce data points
// to the requested resolution, improving transfer performance and client-side rendering.
for _, v := range jd {
for _, v_ := range v {
timestep := int64(0)
for i := 0; i < len(v_.Series); i += 1 {
v_.Series[i].Data, timestep, err = resampleFn(v_.Series[i].Data, int64(v_.Timestep), int64(resolution))
v_.Series[i].Data, timestep, err = resampler.LargestTriangleThreeBucket(v_.Series[i].Data, int64(v_.Timestep), int64(resolution))
if err != nil {
return err, 0, 0
}
@@ -420,7 +414,6 @@ func LoadNodeListData(
resolution int,
from, to time.Time,
ctx context.Context,
resampleAlgo string,
) (map[string]schema.JobData, error) {
if metrics == nil {
for _, m := range archive.GetCluster(cluster).MetricConfig {
@@ -435,7 +428,7 @@ func LoadNodeListData(
return nil, err
}
data, err := ms.LoadNodeListData(cluster, subCluster, nodes, metrics, scopes, resolution, from, to, ctx, resampleAlgo)
data, err := ms.LoadNodeListData(cluster, subCluster, nodes, metrics, scopes, resolution, from, to, ctx)
if err != nil {
if len(data) != 0 {
cclog.Warnf("partial error loading node list data from metric store for cluster %s, subcluster %s: %s",

View File

@@ -51,8 +51,7 @@ type MetricDataRepository interface {
scopes []schema.MetricScope,
resolution int,
from, to time.Time,
ctx context.Context,
resampleAlgo string) (map[string]schema.JobData, error)
ctx context.Context) (map[string]schema.JobData, error)
// HealthCheck evaluates the monitoring state for a set of nodes against expected metrics.
HealthCheck(cluster string,

View File

@@ -1,49 +0,0 @@
// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package metricdispatch
import "math"
type ResamplePolicy string
const (
ResamplePolicyLow ResamplePolicy = "low"
ResamplePolicyMedium ResamplePolicy = "medium"
ResamplePolicyHigh ResamplePolicy = "high"
)
// TargetPointsForPolicy returns the target number of data points for a given policy.
func TargetPointsForPolicy(policy ResamplePolicy) int {
switch policy {
case ResamplePolicyLow:
return 200
case ResamplePolicyMedium:
return 500
case ResamplePolicyHigh:
return 1000
default:
return 0
}
}
// ComputeResolution computes the resampling resolution in seconds for a given
// job duration, metric frequency, and target point count. Returns 0 if the
// total number of data points is already at or below targetPoints (no resampling needed).
func ComputeResolution(duration int64, frequency int64, targetPoints int) int {
if frequency <= 0 || targetPoints <= 0 || duration <= 0 {
return 0
}
totalPoints := duration / frequency
if totalPoints <= int64(targetPoints) {
return 0
}
targetRes := math.Ceil(float64(duration) / float64(targetPoints))
// Round up to nearest multiple of frequency
resolution := int(math.Ceil(targetRes/float64(frequency))) * int(frequency)
return resolution
}

View File

@@ -1,68 +0,0 @@
// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package metricdispatch
import "testing"
func TestTargetPointsForPolicy(t *testing.T) {
tests := []struct {
policy ResamplePolicy
want int
}{
{ResamplePolicyLow, 200},
{ResamplePolicyMedium, 500},
{ResamplePolicyHigh, 1000},
{ResamplePolicy("unknown"), 0},
{ResamplePolicy(""), 0},
}
for _, tt := range tests {
if got := TargetPointsForPolicy(tt.policy); got != tt.want {
t.Errorf("TargetPointsForPolicy(%q) = %d, want %d", tt.policy, got, tt.want)
}
}
}
func TestComputeResolution(t *testing.T) {
tests := []struct {
name string
duration int64
frequency int64
targetPoints int
want int
}{
// 24h job, 60s frequency, 1440 total points
{"low_24h_60s", 86400, 60, 200, 480},
{"medium_24h_60s", 86400, 60, 500, 180},
{"high_24h_60s", 86400, 60, 1000, 120},
// 2h job, 60s frequency, 120 total points — no resampling needed
{"low_2h_60s", 7200, 60, 200, 0},
{"medium_2h_60s", 7200, 60, 500, 0},
{"high_2h_60s", 7200, 60, 1000, 0},
// Edge: zero/negative inputs
{"zero_duration", 0, 60, 200, 0},
{"zero_frequency", 86400, 0, 200, 0},
{"zero_target", 86400, 60, 0, 0},
{"negative_duration", -100, 60, 200, 0},
// 12h job, 30s frequency, 1440 total points
{"medium_12h_30s", 43200, 30, 500, 90},
// Exact fit: total points == target points
{"exact_fit", 12000, 60, 200, 0},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := ComputeResolution(tt.duration, tt.frequency, tt.targetPoints)
if got != tt.want {
t.Errorf("ComputeResolution(%d, %d, %d) = %d, want %d",
tt.duration, tt.frequency, tt.targetPoints, got, tt.want)
}
})
}
}

View File

@@ -617,7 +617,6 @@ func (ccms *CCMetricStore) LoadNodeListData(
resolution int,
from, to time.Time,
ctx context.Context,
resampleAlgo string,
) (map[string]schema.JobData, error) {
queries, assignedScope, err := ccms.buildNodeQueries(cluster, subCluster, nodes, metrics, scopes, resolution)
if err != nil {

View File

@@ -21,11 +21,12 @@ import (
// is added to internal/repository/migrations/sqlite3/.
//
// Version history:
// - Version 12: Sessions table (server-side sessions via alexedwards/scs)
// - Version 11: Optimize job table indexes (reduce from ~78 to 48, add covering/partial indexes)
// - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 11
const Version uint = 12
//go:embed migrations/*
var migrationFiles embed.FS

View File

@@ -0,0 +1 @@
DROP TABLE IF EXISTS sessions;

View File

@@ -0,0 +1,7 @@
CREATE TABLE sessions (
token TEXT PRIMARY KEY,
data BLOB NOT NULL,
expiry REAL NOT NULL
);
CREATE INDEX sessions_expiry_idx ON sessions(expiry);

Binary file not shown.

View File

@@ -15,7 +15,6 @@ import (
"github.com/ClusterCockpit/cc-backend/internal/config"
"github.com/ClusterCockpit/cc-backend/internal/graph/model"
"github.com/ClusterCockpit/cc-backend/internal/metricdispatch"
"github.com/ClusterCockpit/cc-backend/internal/repository"
"github.com/ClusterCockpit/cc-backend/web"
cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
@@ -497,15 +496,13 @@ func SetupRoutes(router chi.Router, buildInfo web.Build) {
// Get Roles
availableRoles, _ := schema.GetValidRolesMap(user)
resampling := resamplingForUser(conf)
page := web.Page{
Title: title,
User: *user,
Roles: availableRoles,
Build: buildInfo,
Config: conf,
Resampling: resampling,
Resampling: config.Keys.EnableResampling,
Infos: infos,
}
@@ -592,36 +589,3 @@ func HandleSearchBar(rw http.ResponseWriter, r *http.Request, buildInfo web.Buil
web.RenderTemplate(rw, "message.tmpl", &web.Page{Title: "Warning", MsgType: "alert-warning", Message: "Empty search", User: *user, Roles: availableRoles, Build: buildInfo})
}
}
// resamplingForUser returns a ResampleConfig that incorporates the user's
// resample policy preference. If the user has a policy set, it creates a
// policy-derived config with targetPoints and trigger. Otherwise falls back
// to the global config.
func resamplingForUser(conf map[string]any) *config.ResampleConfig {
globalCfg := config.Keys.EnableResampling
policyStr := ""
if policyVal, ok := conf["plotConfiguration_resamplePolicy"]; ok {
if s, ok := policyVal.(string); ok {
policyStr = s
}
}
// Fall back to global default policy, then to "medium"
if policyStr == "" && globalCfg != nil {
policyStr = globalCfg.DefaultPolicy
}
if policyStr == "" {
policyStr = "medium"
}
policy := metricdispatch.ResamplePolicy(policyStr)
targetPoints := metricdispatch.TargetPointsForPolicy(policy)
if targetPoints == 0 {
return globalCfg
}
return &config.ResampleConfig{
TargetPoints: targetPoints,
}
}

View File

@@ -51,15 +51,14 @@ type APIMetricData struct {
//
// The request can be customized with flags to include/exclude statistics, raw data, and padding.
type APIQueryRequest struct {
Cluster string `json:"cluster"`
Queries []APIQuery `json:"queries"`
ForAllNodes []string `json:"for-all-nodes"`
From int64 `json:"from"`
To int64 `json:"to"`
WithStats bool `json:"with-stats"`
WithData bool `json:"with-data"`
WithPadding bool `json:"with-padding"`
ResampleAlgo string `json:"resample-algo,omitempty"`
Cluster string `json:"cluster"`
Queries []APIQuery `json:"queries"`
ForAllNodes []string `json:"for-all-nodes"`
From int64 `json:"from"`
To int64 `json:"to"`
WithStats bool `json:"with-stats"`
WithData bool `json:"with-data"`
WithPadding bool `json:"with-padding"`
}
// APIQueryResponse represents the response to an APIQueryRequest.
@@ -280,7 +279,7 @@ func FetchData(req APIQueryRequest) (*APIQueryResponse, error) {
for _, sel := range sels {
data := APIMetricData{}
data.Data, data.From, data.To, data.Resolution, err = ms.Read(sel, query.Metric, req.From, req.To, query.Resolution, req.ResampleAlgo)
data.Data, data.From, data.To, data.Resolution, err = ms.Read(sel, query.Metric, req.From, req.To, query.Resolution)
if err != nil {
// Skip Error If Just Missing Host or Metric, Continue
// Empty Return For Metric Handled Gracefully By Frontend

View File

@@ -701,7 +701,7 @@ func (m *MemoryStore) WriteToLevel(l *Level, selector []string, ts int64, metric
// If the level does not hold the metric itself, the data will be aggregated recursively from the children.
// The second and third return value are the actual from/to for the data. Those can be different from
// the range asked for if no data was available.
func (m *MemoryStore) Read(selector util.Selector, metric string, from, to, resolution int64, resampleAlgo string) ([]schema.Float, int64, int64, int64, error) {
func (m *MemoryStore) Read(selector util.Selector, metric string, from, to, resolution int64) ([]schema.Float, int64, int64, int64, error) {
if from > to {
return nil, 0, 0, 0, errors.New("[METRICSTORE]> invalid time range")
}
@@ -759,11 +759,7 @@ func (m *MemoryStore) Read(selector util.Selector, metric string, from, to, reso
}
}
resampleFn, rfErr := resampler.GetResampler(resampleAlgo)
if rfErr != nil {
return nil, 0, 0, 0, rfErr
}
data, resolution, err = resampleFn(data, minfo.Frequency, resolution)
data, resolution, err = resampler.LargestTriangleThreeBucket(data, minfo.Frequency, resolution)
if err != nil {
return nil, 0, 0, 0, err
}

View File

@@ -621,7 +621,6 @@ func (ccms *InternalMetricStore) LoadNodeListData(
resolution int,
from, to time.Time,
ctx context.Context,
resampleAlgo string,
) (map[string]schema.JobData, error) {
// Note: Order of node data is not guaranteed after this point
queries, assignedScope, err := buildNodeQueries(cluster, subCluster, nodes, metrics, scopes, int64(resolution))
@@ -637,13 +636,12 @@ func (ccms *InternalMetricStore) LoadNodeListData(
}
req := APIQueryRequest{
Cluster: cluster,
Queries: queries,
From: from.Unix(),
To: to.Unix(),
WithStats: true,
WithData: true,
ResampleAlgo: resampleAlgo,
Cluster: cluster,
Queries: queries,
From: from.Unix(),
To: to.Unix(),
WithStats: true,
WithData: true,
}
resBody, err := FetchData(req)

View File

@@ -11,7 +11,7 @@ MCowBQYDK2VwAyEA+51iXX8BdLFocrppRxIw52xCOf8xFSH/eNilN5IHVGc=
Unfortunately, ClusterCockpit does not handle this format (yet). You can use this tool to convert the public PEM key into a representation for CC:
```
CROSS_LOGIN_JWT_PUBLIC_KEY="+51iXX8BdLFocrppRxIw52xCOf8xFSH/eNilN5IHVGc="
cross-login-public-key: "+51iXX8BdLFocrppRxIw52xCOf8xFSH/eNilN5IHVGc="
```
Instructions
@@ -19,7 +19,9 @@ Instructions
- `cd tools/convert-pem-pubkey/`
- Insert your public ed25519 PEM key into `dummy.pub`
- `go run . dummy.pub`
- Copy the result into ClusterCockpit's `.env`
- Set the result as `cross-login-public-key` under `auth.jwts` in ClusterCockpit's
`config.json` (or supply it via the `CROSS_LOGIN_JWT_PUBLIC_KEY` environment
variable, which takes precedence)
- (Re)start ClusterCockpit backend
Now CC can validate generated JWTs from the external provider.

View File

@@ -44,8 +44,11 @@ func main() {
os.Exit(1)
}
// Print the value for use as auth.jwts.cross-login-public-key in config.json.
// It may alternatively be supplied via the CROSS_LOGIN_JWT_PUBLIC_KEY
// environment variable, which takes precedence.
fmt.Fprintf(os.Stdout,
"CROSS_LOGIN_JWT_PUBLIC_KEY=%#v\n",
"cross-login-public-key: %#v\n",
base64.StdEncoding.EncodeToString(pubkey))
}

View File

@@ -16,7 +16,6 @@
import Options from "./admin/Options.svelte";
import NoticeEdit from "./admin/NoticeEdit.svelte";
import RunTaggers from "./admin/RunTaggers.svelte";
import PlotRenderOptions from "./user/PlotRenderOptions.svelte";
/* Svelte 5 Props */
let {
@@ -30,8 +29,6 @@
/* State Init */
let users = $state([]);
let roles = $state([]);
let message = $state({ msg: "", target: "", color: "#d63384" });
let displayMessage = $state(false);
/* Functions */
function getUserList() {
@@ -55,37 +52,6 @@
getValidRoles();
}
async function handleSettingSubmit(event, setting) {
event.preventDefault();
const selector = setting.selector
const target = setting.target
let form = document.querySelector(selector);
let formData = new FormData(form);
try {
const res = await fetch(form.action, { method: "POST", body: formData });
if (res.ok) {
let text = await res.text();
popMessage(text, target, "#048109");
} else {
let text = await res.text();
throw new Error("Response Code " + res.status + "-> " + text);
}
} catch (err) {
popMessage(err, target, "#d63384");
}
return false;
}
function popMessage(response, restarget, rescolor) {
message = { msg: response, target: restarget, color: rescolor };
displayMessage = true;
setTimeout(function () {
displayMessage = false;
}, 3500);
}
/* on Mount */
onMount(() => initAdmin());
</script>
@@ -107,4 +73,3 @@
<NoticeEdit {ncontent}/>
<RunTaggers />
</Row>
<PlotRenderOptions config={ccconfig} bind:message bind:displayMessage updateSetting={(e, newSetting) => handleSettingSubmit(e, newSetting)}/>

View File

@@ -16,7 +16,6 @@
Card,
CardTitle,
} from "@sveltestrap/sveltestrap";
import { getContext } from "svelte";
import { fade } from "svelte/transition";
/* Svelte 5 Props */
@@ -26,8 +25,6 @@
displayMessage = $bindable(),
updateSetting
} = $props();
const resampleConfig = getContext("resampling");
</script>
<Row cols={3} class="p-2 g-2">
@@ -67,7 +64,7 @@
id="lwvalue"
name="value"
aria-describedby="lineWidthHelp"
value={config?.plotConfiguration_lineWidth}
value={config.plotConfiguration_lineWidth}
min="1"
/>
<div id="lineWidthHelp" class="form-text">
@@ -114,7 +111,7 @@
id="pprvalue"
name="value"
aria-describedby="plotsperrowHelp"
value={config?.plotConfiguration_plotsPerRow}
value={config.plotConfiguration_plotsPerRow}
min="1"
/>
<div id="plotsperrowHelp" class="form-text">
@@ -156,7 +153,7 @@
<input type="hidden" name="key" value="plotConfiguration_colorBackground" />
<div class="mb-3">
<div>
{#if config?.plotConfiguration_colorBackground}
{#if config.plotConfiguration_colorBackground}
<input type="radio" id="colb-true-checked" name="value" value="true" checked />
{:else}
<input type="radio" id="colb-true" name="value" value="true" />
@@ -164,7 +161,7 @@
<label for="true">Yes</label>
</div>
<div>
{#if config?.plotConfiguration_colorBackground}
{#if config.plotConfiguration_colorBackground}
<input type="radio" id="colb-false" name="value" value="false" />
{:else}
<input type="radio" id="colb-false-checked" name="value" value="false" checked />
@@ -222,90 +219,4 @@
</form>
</Card>
</Col>
<!-- RESAMPLE POLICY -->
<Col>
<Card class="h-100">
<form
id="resample-policy-form"
method="post"
action="/frontend/configuration/"
class="card-body"
onsubmit={(e) => updateSetting(e, {
selector: "#resample-policy-form",
target: "rsp",
})}
>
<CardTitle
style="margin-bottom: 1em; display: flex; align-items: center;"
>
<div>Resample Policy</div>
{#if displayMessage && message.target == "rsp"}
<div style="margin-left: auto; font-size: 0.9em;">
<code style="color: {message.color};" out:fade>
Update: {message.msg}
</code>
</div>
{/if}
</CardTitle>
<input type="hidden" name="key" value="plotConfiguration_resamplePolicy" />
<div class="mb-3">
{#each [["", "Default"], ["low", "Low"], ["medium", "Medium"], ["high", "High"]] as [val, label]}
<div>
<input type="radio" id="rsp-{val || 'default'}" name="value" value={JSON.stringify(val)}
checked={(!config?.plotConfiguration_resamplePolicy && val === "") || config?.plotConfiguration_resamplePolicy === val} />
<label for="rsp-{val || 'default'}">{label}</label>
</div>
{/each}
<div id="resamplePolicyHelp" class="form-text">
Controls how many data points are shown in metric plots. Low = fast overview (~200 points), Medium = balanced (~500), High = maximum detail (~1000).
</div>
</div>
<Button color="primary" type="submit">Submit</Button>
</form>
</Card>
</Col>
<!-- RESAMPLE ALGORITHM -->
<Col>
<Card class="h-100">
<form
id="resample-algo-form"
method="post"
action="/frontend/configuration/"
class="card-body"
onsubmit={(e) => updateSetting(e, {
selector: "#resample-algo-form",
target: "rsa",
})}
>
<CardTitle
style="margin-bottom: 1em; display: flex; align-items: center;"
>
<div>Resample Algorithm</div>
{#if displayMessage && message.target == "rsa"}
<div style="margin-left: auto; font-size: 0.9em;">
<code style="color: {message.color};" out:fade>
Update: {message.msg}
</code>
</div>
{/if}
</CardTitle>
<input type="hidden" name="key" value="plotConfiguration_resampleAlgo" />
<div class="mb-3">
{#each [["", "Default"], ["lttb", "LTTB"], ["average", "Average"], ["simple", "Simple"]] as [val, label]}
<div>
<input type="radio" id="rsa-{val || 'default'}" name="value" value={JSON.stringify(val)}
checked={(!config?.plotConfiguration_resampleAlgo && val === "") || config?.plotConfiguration_resampleAlgo === val} />
<label for="rsa-{val || 'default'}">{label}</label>
</div>
{/each}
<div id="resampleAlgoHelp" class="form-text">
Algorithm used when downsampling time-series data. LTTB preserves visual shape, Average smooths data, Simple picks every Nth point.
</div>
</div>
<Button color="primary" type="submit">Submit</Button>
</form>
</Card>
</Col>
</Row>

View File

@@ -73,10 +73,9 @@
const subClusterTopology = $derived(getContext("getHardwareTopology")(cluster, subCluster));
const metricConfig = $derived(getContext("getMetricConfig")(cluster, subCluster, metric));
const usesMeanStatsSeries = $derived((statisticsSeries?.mean && statisticsSeries.mean.length != 0));
const resampleTrigger = $derived(resampleConfig?.trigger ? Number(resampleConfig.trigger) : (resampleConfig?.targetPoints ? Math.floor(resampleConfig.targetPoints / 4) : null));
const resampleTrigger = $derived(resampleConfig?.trigger ? Number(resampleConfig.trigger) : null);
const resampleResolutions = $derived(resampleConfig?.resolutions ? [...resampleConfig.resolutions] : null);
const resampleMinimum = $derived(resampleConfig?.resolutions ? Math.min(...resampleConfig.resolutions) : null);
const resampleTargetPoints = $derived(resampleConfig?.targetPoints ? Number(resampleConfig.targetPoints) : null);
const useStatsSeries = $derived(!!statisticsSeries); // Display Stats Series By Default if Exists
const thresholds = $derived(findJobAggregationThresholds(
subClusterTopology,
@@ -516,29 +515,24 @@
if (resampleConfig && !forNode && key === 'x') {
const numX = (u.series[0].idxs[1] - u.series[0].idxs[0])
if (numX <= resampleTrigger && timestep !== resampleMinimum) {
let newRes;
if (resampleTargetPoints && !resampleResolutions) {
// Policy-based: compute resolution dynamically from visible window
const visibleDuration = (u.scales.x.max - u.scales.x.min);
const nativeTimestep = metricConfig?.timestep || timestep;
newRes = Math.ceil(visibleDuration / resampleTargetPoints / nativeTimestep) * nativeTimestep;
if (newRes < nativeTimestep) newRes = nativeTimestep;
} else if (resampleResolutions) {
// Array-based: find closest configured resolution
const target = (numX * timestep) / resampleTrigger;
newRes = resampleResolutions.reduce(function(prev, curr) {
return (Math.abs(curr - target) < Math.abs(prev - target) ? curr : prev);
});
}
/* Get closest zoom level; prevents multiple iterative zoom requests for big zoom-steps (e.g. 600 -> 300 -> 120 -> 60) */
// Which resolution to theoretically request to achieve 30 or more visible data points:
const target = (numX * timestep) / resampleTrigger
// Which configured resolution actually matches the closest to theoretical target:
const closest = resampleResolutions.reduce(function(prev, curr) {
return (Math.abs(curr - target) < Math.abs(prev - target) ? curr : prev);
});
// Prevents non-required dispatches
if (newRes && timestep !== newRes) {
if (timestep !== closest) {
// console.log('Dispatch: Zoom with Res from / to', timestep, closest)
onZoom({
newRes: newRes,
newRes: closest,
lastZoomState: u?.scales,
lastThreshold: thresholds?.normal
});
}
} else {
// console.log('Dispatch: Zoom Update States')
onZoom({
lastZoomState: u?.scales,
lastThreshold: thresholds?.normal

View File

@@ -4,7 +4,7 @@ import {
setContextClient,
fetchExchange,
} from "@urql/svelte";
import { setContext, getContext, hasContext, onDestroy, tick } from "svelte";
import { setContext, getContext, onDestroy, tick } from "svelte";
import { readable } from "svelte/store";
import { round } from "mathjs";
@@ -21,14 +21,11 @@ import { round } from "mathjs";
* - Adds 'getHardwareTopology' to the context, a function that takes a cluster nad subCluster and returns the subCluster topology (or undefined)
*/
export function init(extraInitQuery = "") {
const jwt = hasContext("jwt")
? getContext("jwt")
: getContext("cc-config")["jwt"];
// The web UI authenticates GraphQL requests via the session cookie
// (same-origin), so no Authorization header is attached here. External
// clients use a JWT against /query directly.
const client = new Client({
url: `${window.location.origin}/query`,
fetchOptions:
jwt != null ? { headers: { Authorization: `Bearer ${jwt}` } } : {},
exchanges: [
expiringCacheExchange({
ttl: 5 * 60 * 1000,

View File

@@ -72,8 +72,6 @@ type PlotConfiguration struct {
PlotsPerRow int `json:"plots-per-row"`
LineWidth int `json:"line-width"`
ColorScheme []string `json:"color-scheme"`
ResampleAlgo string `json:"resample-algo"`
ResamplePolicy string `json:"resample-policy"`
}
var UIDefaults = WebConfig{
@@ -146,8 +144,6 @@ func Init(rawConfig json.RawMessage) error {
UIDefaultsMap["plotConfiguration_plotsPerRow"] = UIDefaults.PlotConfiguration.PlotsPerRow
UIDefaultsMap["plotConfiguration_lineWidth"] = UIDefaults.PlotConfiguration.LineWidth
UIDefaultsMap["plotConfiguration_colorScheme"] = UIDefaults.PlotConfiguration.ColorScheme
UIDefaultsMap["plotConfiguration_resampleAlgo"] = UIDefaults.PlotConfiguration.ResampleAlgo
UIDefaultsMap["plotConfiguration_resamplePolicy"] = UIDefaults.PlotConfiguration.ResamplePolicy
for _, c := range UIDefaults.MetricConfig.Clusters {
if c.JobListMetrics != nil {