The schema string was the inside of a "properties" block: jwts, oidc, ldap and
required sat at the root with no enclosing "type"/"properties". jsonschema
compiled it as a document with only unknown keywords, so every auth section
validated and no option was ever checked.
Wrapped it correctly and dropped the root-level required: ["jwts"]. Init logs
and continues when the jwts section is absent ("Missing JWT configuration: No
JWT token support!"), so enforcing it would abort OIDC- or LDAP-only
deployments that work today. The per-subsection required lists are unchanged.
Validation errors are now printed with err.Error() instead of %#v, which
rendered a jsonschema.ValidationError unreadably.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
plot-configuration.resample-policy and resample-algo are read into
PlotConfiguration and exposed through UIDefaultsMap as the site default for new
users, but neither appeared in the schema, so they looked unsupported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shipped examples had drifted from the code that parses them:
- configs/cluster.json was still in the pre-v3 format: no top-level "name",
"numberOfNodes" instead of the required "nodes" node-list expression, plain
numbers instead of MetricValue objects for flopRateScalar/flopRateSimd/
memoryBandwidth, unit strings instead of unit objects, "aggregation": null,
and hwthread 72 missing from the core topology. It neither validated against
cluster.schema.json nor decoded into schema.Cluster. Rewritten in the current
format and extended with the accelerator metrics the GPU subclusters need.
- archive.retention.location is not a field of taskmanager.Retention. That
decode is not strict, so the key was silently dropped and a "move" policy ran
with an empty target path. Replaced with target-kind/target-path.
- "ui-file" pointed at ui-config.json while the shipped file is uiConfig.json,
so ccConfig logged a load error and the UI fell back to defaults.
- Added the options introduced since the examples were written: checkpoint
file-format/interval/max-wal-size, api-subjects concurrency, db-config,
nodestate-retention, archive retention/compression, smoothing-window and the
resample policy/algo defaults.
The config written by -init had no metric-store section, so a server started
right after -init aborted with "missing metricstore configuration".
internal/configexample guards all of this: it resolves "-file" references the
way ccConfig does, strict-decodes every section into the struct that actually
parses it, and validates the cluster configs against the cc-lib schema.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cc-lib's resamplers cannot smooth: validateFrequency bails out when the new
frequency is not coarser than the old one, and every algorithm produces an
output strictly shorter than its input. They consolidate for transport,
which is a separate job from filtering for display, and they only act at all
once a series exceeds the target point count - shorter jobs arrive raw.
Add a centered, NaN-aware moving average in the plot layer, applied after
whatever downsampling the backend performed. It preserves length and index
alignment, so the X array, the visible point count and the zoom-resampling
hook are unaffected. NaN and null samples are skipped, an all-NaN window
stays NaN so gaps keep rendering, and the window shrinks at the series edges
rather than introducing new gaps there.
The window is a user setting in data points, defaulting to 3, and is
orthogonal to the resample algorithm - it stacks on top of average
consolidation rather than replacing it. All series of a stats plot are
smoothed together, since smoothing only some would break the
min <= mid <= max invariant the plot bands rely on. Smoothing is display
only: reported statistics and job footprints come from JobMetric.Statistics
and are untouched. The uPlot cursor readout does show the smoothed value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Average consolidation makes every plotted point the true arithmetic mean of
its interval, which is what the Ganglia-style plot appearance depends on.
LTTB deliberately keeps extremes instead, so a plot downsampled with it
cannot be read as interval means.
resolveResampleAlgo used to return an empty string when neither the user nor
the config had chosen an algorithm, and cc-lib's GetResampler maps that to
LTTB. Route every fallback through config.ResampleAlgo() and ship "average"
as the generated and example default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
config.ResampleConfig stopped emitting the trigger and resolutions fields,
but MetricPlot still read them. resampleResolutions and resampleMinimum
were therefore always null, which made the array-based resolution branch in
the setScale hook unreachable and the timestep !== resampleMinimum guard
always true. Keep only the policy-based branch and derive the zoom trigger
straight from targetPoints.
configs/config-large.json still carried the removed minimum-points, trigger
and resolutions keys, which DisallowUnknownFields now rejects, so starting
against it aborted. Replace them with the current keys and add a test that
loads the file, so the example configs cannot drift out of the schema
unnoticed again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only archived job data and internal-store node-list data honoured the
selected resample algorithm. For running jobs the algorithm was dropped:
MetricDataRepository.LoadData had no such parameter, so the memory store
fell back to an empty string, which cc-lib's GetResampler maps to LTTB.
The external store client was worse - its APIQueryRequest had no
ResampleAlgo field at all, and LoadNodeListData accepted the parameter
without using it.
Add resampleAlgo to the LoadData interface (mirroring LoadNodeListData),
forward it from metricdispatch, and set it on both stores' requests. The
field is tagged omitempty, so the wire format is unchanged when empty -
verify the deployed cc-metric-store accepts it before relying on it there.
The REST job endpoints pass a non-zero resolution and therefore do
resample, so they now request the configured default instead of an empty
string. Add config.ResampleAlgo() for that, since "" is not a neutral
value at this layer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The policy -> target-points mapping existed twice with different values:
internal/config (300/600/1000) fed the resampler's MinimumRequiredPoints
threshold, while internal/metricdispatch (200/500/1000) fed the requested
resolution and the frontend target point count.
Because the threshold was larger than the target, the resampler refused to
downsample series whose length fell between the two numbers, silently
dropping the resolution the backend had asked for. With the medium policy
that covered every series between 500 and 600 points.
Move the table into internal/config as the single source of truth (import
direction rules out the reverse, since metricdispatch already imports
config) and keep the 200/500/1000 values, which already drove the requested
resolution. metricdispatch.TargetPointsForPolicy now delegates to it, so
MinimumRequiredPoints equals the target and resampling happens exactly when
a series exceeds it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pre-existing formatting drift picked up by `make fmt`. Whitespace and
comment layout only, no semantic change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve conflicts in the generated GraphQL code by regenerating it against
the merged schema. cc-lib v2.13.0 adds Tooltip to schema.MetricConfig and
schema.GlobalMetricListItem, so gqlgen now binds the tooltip field directly
and the hand-written globalMetricListItem/metricConfig resolvers introduced
on the tooltip branch are no longer needed.
Also migrate to the cc-lib v2.13.0 metric container types, which changed
from bare maps to structs carrying array-valued metric groups:
schema.JobData map -> {Metrics, Groups}
schema.ScopedJobStats map -> {Metrics, Groups}
job.Statistics map -> schema.JobStatisticsSet{Metrics, Groups}
Callers index .Metrics, return the zero struct instead of nil, and
deepCopy/DecodeJobStats now also carry the Groups payload through.
archive.GetStatistics returns the full JobStatisticsSet so group
statistics survive the round trip.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PlotRenderOptions was rendered both in AdminSettings.svelte and in
UserSettings.svelte. Since Config.root.svelte always renders UserSettings,
including for admins, admins saw the identical plot-render cards twice on
/config.
The admin copy was also misleading: every form in the component posts to
/frontend/configuration/, whose handler (RestAPI.updateConfiguration) always
writes per-user config via UserCfgRepo.UpdateConfig — never a global default.
On top of that, the admin-side handleSettingSubmit lacked the colorblind-mode
special case that UserSettings uses to keep PlotColorScheme in sync, so
toggling the option there did not refresh the palette preview.
Removes the component from AdminSettings.svelte along with the message,
displayMessage, handleSettingSubmit and popMessage members that existed only
to feed it. The remaining admin children manage their own messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverts 22c442db "Enable entire integration" and drops the accompanying
.gitignore entry:
- .claude/settings.json: removes all seven "entire hooks claude-code ..."
command hooks (PostToolUse on Task and TodoWrite, PreToolUse on Task,
SessionEnd, SessionStart, Stop, UserPromptSubmit) and the
Read(./.entire/metadata/**) deny rule. The file held nothing else, so it
is deleted rather than left as an empty object.
- .entire/settings.json, .entire/.gitignore: untracked and removed.
- .gitignore: drops the now-pointless .entire entry.
.claude/settings.local.json is untouched; it carries unrelated local
permission settings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OIDC roles were translated by a hardcoded switch that only recognised the
literal names user/admin/manager/support (dropping api) and required the
IdP to emit exactly those names.
Add an optional auth.oidc.role-mapping (OIDC role/group claim value -> CC
role) so operators can map their own realm/client role names, including
api. The mapping is the sole source of roles: only mapped names are
honored, unmapped token roles are ignored, and users without any mapped
role receive the base "user" role. Mapping targets are validated once at
startup. Role assignment stays authoritative on every login as before.
Note: deployments relying on the IdP emitting literal CC role names must
now map them explicitly (e.g. "admin": "admin").
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 66451678484b
The LDAP sync only ever granted the base "user" role. Add an optional
auth.ldap.role-filters map (role -> LDAP filter) so accounts matching a
filter are granted that elevated role (admin/support/api/manager).
LDAP is authoritative for the configured roles: sync both adds and removes
them to match group membership, while roles not listed (e.g. a manually
granted manager) are preserved. A managed manager that still has assigned
projects is never stripped. With no role-filters configured behaviour is
identical to before.
Roles are reconciled during periodic sync and at login. Sync evaluates
each filter once over the whole base (one search per role, not per user)
and reconciles existing users via a single ListUsers lookup plus the new
UserRepository.UpdateRoles helper.
Closes#74
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: e38526c3259d
Bump gqlgen, aws-sdk-go-v2, go-oidc, go-sqlite3, gqlparser, go-openapi,
golang.org/x/* and other transitive dependencies to their latest patch
and minor releases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 964ca6774882
The backend resample config changed from {trigger, resolutions[]} to a
policy/targetPoints model, but several components still spread
resampleConfig.resolutions, throwing "resolutions is not iterable" and
breaking the job list render. Default the initial resolution to null (the
backend resolves it from the configured policy; zoom overrides it) and
drop the now-obsolete resolution selector and admin display fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 27516ce259fd
A setting nested at the wrong level (e.g. "resampling" placed next to
"main" instead of inside it) was silently ignored. Emit a startup warning
for any top-level config section not consumed by the backend so such
misconfigurations surface instead of failing silently.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 69d1b85c8db3
Add a "main.footer-links" config option so the footer Imprint and
Privacy Policy links can point at internal pages (default) or external
URLs. External http(s) targets open in a new tab; empty/unset values
fall back to the built-in /imprint and /privacy routes, keeping the
existing ./var/*.tmpl override mechanism intact.
Closes#517
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: c5dca72c848f
Secrets (JWT keys, LDAP sync password, OIDC client id/secret, cross-login
keys) are now configured directly in config.json under the auth section
where they are used. Each secret can still be supplied via its existing
environment variable, which takes precedence over the config value.
The godotenv dependency, the .env file, configs/env-template.txt and the
loadEnvironment() bootstrap step are removed. -init now writes the demo
JWT keys into config.json instead of a .env file.
Closes#283
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3a7cb814c53f
Browser sessions are now server-side, stored in the SQLite database via
scs/sqlite3store (new `sessions` table, DB migration to version 12) instead
of gorilla/sessions client-side cookie storage. Only an opaque random token
is kept in the cookie; session data lives server-side and survives restarts.
Session middleware is wired as a hybrid to avoid buffering large responses:
scs.LoadAndSave on the login/logout write paths, and a non-buffering
read-only LoadSession middleware on the secured/config/frontend read paths
so the large GraphQL /query responses stream unbuffered. JWT-only APIs
(/api, /userapi, /api/metricstore) and static files are left unwrapped.
The session cookie Secure flag is now derived from the server config (set
when cc-backend terminates TLS itself); previously it was effectively never
set. The SESSION_KEY env var is removed as server-side tokens need no
signing secret. The dormant Bearer-JWT branch in the frontend urql client
is removed; the web UI authenticates GraphQL via the session cookie.
Closes#558
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b51075f43cc7
- auth: do not abort the server when authentication is disabled. auth.Init
is now always called; with disable-authentication it sets up an ephemeral
session store (SESSION_KEY not required) and registers no authenticators,
so the unconditional auth.GetAuthInstance() callers (server init,
api.New()) always get a valid instance.
- main: run the graceful-shutdown sequence on the startup-error path. runServer
derives a cancelable context and, on a server-start failure, cancels it and
waits so the metricstore final checkpoint / WAL rotation, archiver flush and
taskmanager shutdown actually run before exit.
- server: log the :80 HTTP->HTTPS redirect listener error instead of dropping it.
- archiver: guard Shutdown against being called when Start never ran
(avoids close(nil) panic / blocking on a nil workerDone).
- nats API: stop worker goroutines on shutdown via a stop channel + idempotent
Shutdown(); workers and subscription callbacks select on stop and the
channels are never closed, so no send-on-closed-channel can occur. Wired
into Server.Shutdown after the NATS client is closed.
- metricstore: make Shutdown idempotent (nil shutdownFunc, early return) and
release shutdownFuncMu before the checkpoint write.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 3c179f9caa8f
Addresses the remaining medium findings from the second-pass audit:
- DoS hardening: bound GraphQL query cost with FixedComplexityLimit, and
reject non-positive items-per-page / page values so uint64 conversion
cannot underflow into an unbounded LIMIT/OFFSET. The -1 "load all"
sentinel stays valid for dashboards; REST now returns 400 for bad input.
- Security headers: add X-Content-Type-Options, X-Frame-Options,
Referrer-Policy and a conservative CSP (frame-ancestors/object-src/
base-uri) that hardens against clickjacking and base-tag injection
without restricting the self-hosted SPA's inline scripts.
- Stored XSS: render job.metaData.message as escaped text instead of
{@html ...} in Job.root and JobFootprint, preserving line breaks via
white-space: pre-wrap.
- SQL injection hardening: parameterize the tag-scope IN list and the
manager project subquery in CountTags instead of interpolating
user.Username / user.Projects (externally sourced via OIDC/LDAP).
- CSRF defense-in-depth: reject cross-site state-changing requests via
Sec-Fetch-Site, failing open for non-browser clients, on top of the
existing SameSite=Lax session cookie.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: de7d47a85c7c
A second-pass audit surfaced three severe issues missed by the previous
review, each a sibling code path of a bug class that was only partially
fixed before:
- auth: JWT session login (jwtSession.go) registered its authenticator
even when CROSS_LOGIN_JWT_HS512_KEY was unset, leaving an empty HMAC
key. golang-jwt verifies any HS256/HS512 signature against an empty
key, allowing unauthenticated admin token forgery. Init() now refuses
to register without a key, with a defense-in-depth empty-key guard in
the keyfunc.
- repository: metric names from GraphQL ([String!]) were interpolated
raw into json_extract(footprint, "$.<name>") SQL. SQLite parses
double-quoted strings as literals, enabling SQL injection by any
authenticated user. Validate metric names against ^[a-zA-Z0-9_]+$ in
jobsMetricStatisticsHistogram and buildFloatJSONCondition.
- metricstore: cluster/host line-protocol tags flowed unvalidated into
path.Join(RootDir, cluster, host) for checkpoint/WAL files, allowing
arbitrary file write outside the checkpoint root via NATS
(unauthenticated) or POST /api/write. Reject path-traversal sequences
in DecodeLine before the tags become path components.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: b57246993ec1
WAL writes during checkpoint are redundant since the binary snapshot
captures all in-memory data. Pausing eliminates channel saturation
(1.4M+ dropped messages) caused by disk I/O contention between
checkpoint writes and WAL staging. Also removes direct WAL file
deletion in checkpoint workers that raced with the staging goroutine.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 34d698f40bac
RotateWALFiles used a non-blocking send (select/default) on rotation
channels buffered at 64. With thousands of nodes and few shards, the
channel fills instantly and nearly all hosts are skipped, leaving WAL
files unrotated indefinitely.
Replace with a blocking send using a shared 2-minute deadline so the
checkpoint goroutine waits for the staging goroutine to drain the
channel instead of immediately giving up.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: a1ec897216fa
Stream CheckpointFile trees directly to parquet rows instead of
materializing all rows in a giant intermediate slice. This eliminates
~1.9GB per host of redundant allocations (repeated string headers)
and removes the expensive sort on millions of 104-byte structs.
Key changes:
- Replace flattenCheckpointFile + sortParquetRows + WriteHostRows with
streaming WriteCheckpointFile that walks the tree with sorted keys
- Reduce results channel buffer from len(hostEntries) to 2 for
back-pressure (at most NumWorkers+2 results in flight)
- Workers send CheckpointFile trees instead of []ParquetMetricRow
- Write rows in small 1024-element batches via reusable buffer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: f31dc1847539
These conversation transcript files were committed before the gitignore
rule existed. They are now properly ignored via .entire/.gitignore.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
moebiusbandandClaude Opus 5