Commit Graph
216 Commits
Author SHA1 Message Date
moebiusbandandClaude Opus 4.8 0724b2dc60 Adopt cc-lib hierarchical JobData/Statistics structs
cc-lib changed JobData, ScopedJobStats and Job.Statistics from flat maps
to structs (a .Metrics map plus array-valued Groups) to represent
filesystem (and future interconnect) metric groups. Migrate all
construction, indexing and iteration to the new API across the archive
backends, metricstore query path, metric dispatcher, archiver, importer,
tagger, taskmanager, repository and API layers.

Semantics: DecodeJobStats now projects JobData.Groups into
ScopedJobStats.Groups, and the archiver derives per-filesystem node-scope
statistics into Job.Statistics.Groups. deepCopy, resampling and the
metric/scope filter are group-aware. The metricstore internal storage
(buffers, selector tree, checkpoint/parquet) is unchanged; all conversion
stays at the LoadData/archive-codec seam.

Note: requires the corresponding cc-lib release; bump the cc-lib
dependency version once tagged (a local go.mod replace was used during
development and is intentionally not committed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 19:14:12 +02:00
moebiusband 3bef199cbe Regenerate GraphQL 2026-06-04 20:12:27 +02:00
moebiusband 641dc0e3b8 Run gofumpt 2026-03-30 16:49:27 +02:00
moebiusband bf48389aeb Optimize sortby in stats queries
Entire-Checkpoint: 9b5b833472e1
2026-03-20 05:39:22 +01:00
Christoph Kluge 10b4fa5a06 change: remove heuristic metricHealth, replace with DB metricHealth
- add metricHealth to single Node view
2026-03-19 15:55:58 +01:00
Christoph Kluge 886791cf8a remove deprecated minRunningFor filter remnants 2026-03-19 14:09:10 +01:00
moebiusband 22057ff281 Pass reqKey as CacheKey
Entire-Checkpoint: b95ef43221bb
2026-03-19 11:04:32 +01:00
moebiusband e38396a081 Upgrade dependencies. Rebuild GraphQL.
Entire-Checkpoint: f770853c9fa0
2026-03-13 17:22:34 +01:00
moebiusbandandClaude Opus 4.6 d586fe4b43 Optimize usage dashboard: partial indexes, request cache, parallel histograms
- Add migration 14: partial covering indexes WHERE job_state='running'
  for user/project/subcluster groupings (tiny B-tree vs full table)
- Inline literal state value in BuildWhereClause so SQLite matches
  partial indexes instead of parameterized placeholders
- Add per-request statsGroupCache (sync.Once per filter+groupBy key)
  so identical grouped stats queries execute only once per GQL operation
- Parallelize 4 histogram queries in AddHistograms using errgroup
- Consolidate frontend from 6 GQL aliases to 2, sort+slice top-10
  client-side via $derived

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 5b26a6e5ff10
2026-03-13 14:31:37 +01:00
moebiusband 3aacc669b6 Remove debug timer 2026-03-13 06:18:21 +01:00
moebiusband 8e86e8720d Make stats query selective. Add stats index. Add paging to user list.
Entire-Checkpoint: d42431eee30d
2026-03-12 20:16:55 +01:00
moebiusband af78f06ced fix: Reduce complexity for groupBy stats queries
Entire-Checkpoint: fc899a70a751
2026-03-11 15:14:59 +01:00
Christoph Kluge f11ff3302d revert urql/svelte bump, bump svelte patch version, regenerate backend 2026-03-03 17:47:17 +01:00
Christoph Kluge fe3ebe0abc add healthStatus tab to status details, add graphql endpoints and handlers 2026-02-17 14:38:06 +01:00
Christoph Kluge 12e9f6700e fix nodeList resolver data handling, increase nodestate filter cutoff
- add comment on cutoff
2026-02-11 16:16:09 +01:00
moebiusband bca7dd743b Merge branch 'dev' of github.com:ClusterCockpit/cc-backend into dev 2026-02-11 07:06:33 +01:00
moebiusband 8d6c6b819b Update and port to cc-lib 2026-02-11 07:06:06 +01:00
Christoph Kluge 5e4994a64c revert external config supply for nodeList component 2026-02-10 16:46:18 +01:00
moebiusband db8772dc0b Merge branch 'dev' of github.com:ClusterCockpit/cc-backend into dev 2026-02-06 14:06:59 +01:00
moebiusband fa7727c6ca Print job db id instead of its address 2026-02-06 14:06:56 +01:00
Christoph Kluge 5655639320 add subCluster jobFilter for statusDetail queries 2026-02-06 11:10:08 +01:00
Christoph Kluge 13cca1ee62 change log msg on clusterMetrics 2026-02-02 14:45:19 +01:00
Christoph Kluge 7b4e2fcf59 add isNan to clusterMetric aggregation 2026-02-02 14:34:49 +01:00
Christoph Kluge 1ffcc5e241 apply ccUnit for clusterMetrics normalitazion 2026-01-30 15:52:59 +01:00
Christoph Kluge f26cabbdf1 Streamline missing data warnings, review logging 2026-01-29 15:17:33 +01:00
Christoph Kluge 9d9babe94d review clusterMetrics aggregation handling, fixes index error 2026-01-27 19:04:29 +01:00
Christoph Kluge cd3d133f0d Merge branch 'dev' of https://github.com/ClusterCockpit/cc-backend into dev 2026-01-15 10:12:02 +01:00
moebiusband e1efc68476 Update dependencies. Rebuild graphql and swagger 2026-01-15 08:32:06 +01:00
Christoph Kluge e5c620ca20 filter metrics for NodeMetrics query 2026-01-14 19:22:31 +01:00
Christoph Kluge 9deee54e41 make nextPage query conditional if no return limit is requested 2026-01-14 17:18:21 +01:00
Christoph Kluge 170a9ace8a Merge branch 'dev' of https://github.com/ClusterCockpit/cc-backend into dev 2026-01-13 16:59:57 +01:00
Christoph Kluge 518e9950ea add job exclusivity filter, review db indices 2026-01-13 16:59:52 +01:00
moebiusband 11ec2267da Major refactor of metric data handling
- make the  internal memory store required and default
- Rename memorystore to metricstore
- Rename metricDataDispatcher to metricdispatch
- Remove metricdata package
- Introduce metricsync package for upstream metric data pull
2025-12-25 08:42:54 +01:00
moebiusband 8576ae458d Switch to cc-lib v2 2025-12-24 09:24:18 +01:00
moebiusband 6e74fa294a Add role-based visibility for metrics
Fixes #387
2025-12-18 15:47:30 +01:00
Christoph Kluge 19c8e9beb1 move extensive NodeMetricsList handling to node repo func 2025-12-18 10:44:58 +01:00
Jan EitzingerandGitHub 60a69aa0a2 Merge pull request #453 from ClusterCockpit/status_dashboard
Status dashboard
2025-12-16 10:04:49 +01:00
moebiusband e6286768a7 Refactor variabel naming and update doc comments 2025-12-16 08:56:48 +01:00
Christoph Kluge 290a71bd48 Merge branch 'dev' into status_dashboard 2025-12-11 18:56:09 +01:00
Christoph Kluge 4083de2a51 Add public dashboard and route, add DoubleMetricPlot and GQL queries
- add roofline legend display switch
- small fixes
2025-12-09 10:26:55 +01:00
moebiusband 2b64b31393 Merge branch 'ai-review' into dev 2025-12-03 15:01:18 +01:00
Christoph Kluge 399af8592c switch nodeList logic to SQLite as source of truth, fix nodeList continuous scroll
- keep notindb logic for now
2025-11-20 12:18:13 +01:00
moebiusband 6239e7f19b Merge branch 'dev' into ai-review 2025-11-20 08:59:52 +01:00
Christoph Kluge 90c3381954 add nodeState info display and filtering to systems views 2025-11-18 15:56:55 +01:00
Christoph Kluge fb278182d3 add schedulerState resolver 2025-11-12 13:50:09 +01:00
Christoph Kluge e49e5a0474 finalize timed node state backend code, concat functions 2025-11-05 18:17:29 +01:00
moebiusband 8555a88202 Upgrade dependencies. Regenerate GraphQL API. 2025-10-29 08:55:06 +01:00
Christoph Kluge 714d6af7cd initial branch commit, improve countstate backend logic
- stacked component rough sketch
- gql data request pipeline layed out
2025-10-17 18:24:05 +02:00
moebiusband 440cd59e50 Revert hpc_cluster to cluster. Refactor. 2025-10-16 14:32:06 +02:00
moebiusband f5e1226837 Merge branch 'dev' of github.com:ClusterCockpit/cc-backend into dev 2025-10-16 13:01:35 +02:00