Implement the following plan:

# Step 5: Add covering indexes for status queries

## Problem
Migration 12 indexes: `(cluster, hpc_user, start_time, ..., job_state, ...)` — `job_state` at pos 5, can't seek.
3-col indexes `(cluster, job_state, hpc_user)` — correct order but not covering.

## Changes
- **New `internal/repository/migrations/sqlite3/13_status-covering-indexes.up.sql`**:
  - `jobs_cluster_jobstate_user_stats ON job (cluster, job_state, hpc_user, duration, start_time, num_nodes, num_hwthreads, num_acc)`
  - `jobs_cluster_jobstate_project_stats ON job (cluster, job_state, project, duration, start_time, num_nodes, num_hwthreads, num_acc)`
  - `jobs_cluster_jobstate_subcluster_stats ON job (cluster, job_state, subcluster, duration, start_time, num_nodes, num_hwthreads, num_acc)`
  - `DROP INDEX IF EXISTS jobs_cluster_jobstate_user` (subsumed)
  - `DROP INDEX IF EXISTS jobs_cluster_jobstate_project` (subsumed)
- **New `internal/repository/migrations/sqlite3/13_status-covering-indexes.down.sql`**: reverse
- **`internal/repository/migration.go:29`**: `Version` 12 → 13

## Verify
```bash
go build ./cmd/cc-backend
```

If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /Users/jan/.claude/projects/-Users-jan-prg-ClusterCockpit-cc-backend/8522172b-31dc-406b-a928-9f9a227eea85.jsonl


Let me check the current migration version and existing indexes first.


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package repository
     7→
     8→import (
     9→	"database/sql"
    10→	"embed"
    11→	"fmt"
    12→
    13→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    14→	"github.com/golang-migrate/migrate/v4"
    15→	"github.com/golang-migrate/migrate/v4/database/sqlite3"
    16→	"github.com/golang-migrate/migrate/v4/source/iofs"
    17→)
    18→
    19→// Version is the current database schema version required by this version of cc-backend.
    20→// When the database schema changes, this version is incremented and a new migration file
    21→// is added to internal/repository/migrations/sqlite3/.
    22→//
    23→// Version history:
    24→//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
    25→//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
    26→//   - Version 10: Node table
    27→//
    28→// Migration files are embedded at build time from the migrations directory.
    29→const Version uint = 12
    30→
    31→//go:embed migrations/*
    32→var migrationFiles embed.FS
    33→
    34→// checkDBVersion verifies that the database schema version matches the expected version.
    35→// This is called automatically during Connect() to ensure schema compatibility.

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package repository

import (
	"database/sql"
	"embed"
	"fmt"

	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/golang-migrate/migrate/v4"
	"github.com/golang-migrate/migrate/v4/database/sqlite3"
	"github.com/golang-migrate/migrate/v4/source/iofs"
)

// Version is the current database schema version required by this version of cc-backend.
// When the database schema changes, this version is incremented and a new migration file
// is added to internal/repository/migrations/sqlite3/.
//
// Version history:
//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
//   - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 12

//go:embed migrations/*
var migrationFiles embed.FS

// checkDBVersion verifies that the database schema version matches the expected version.
// This is called automatically during Connect() to ensure schema compatibility.


Bash

List existing migrations


01_init-schema.down.sql
01_init-schema.up.sql
02_add-index.down.sql
02_add-index.up.sql
03_add-userprojects.down.sql
03_add-userprojects.up.sql
04_add-constraints.down.sql
04_add-constraints.up.sql
05_extend-tags.down.sql
05_extend-tags.up.sql
06_change-config.down.sql
06_change-config.up.sql
07_fix-tag-id.down.sql
07_fix-tag-id.up.sql
08_add-footprint.down.sql
08_add-footprint.up.sql
09_add-job-cache.down.sql
09_add-job-cache.up.sql
10_node-table.down.sql
10_node-table.up.sql
11_optimize-indexes.down.sql
11_optimize-indexes.up.sql
12_stats-covering-index.down.sql
12_stats-covering-index.up.sql


Let me check the existing migration 12 to understand what's already there.


     1→-- Migration 12: Add covering index for grouped stats queries
     2→-- Column order: cluster (equality), hpc_user (GROUP BY), start_time (range scan)
     3→-- Includes aggregated columns to avoid main table lookups entirely.
     4→
     5→CREATE INDEX IF NOT EXISTS jobs_cluster_user_starttime_stats
     6→  ON job (cluster, hpc_user, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc);
     7→
     8→CREATE INDEX IF NOT EXISTS jobs_cluster_project_starttime_stats
     9→  ON job (cluster, project, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc);
    10→
    11→PRAGMA optimize;
    12→

-- Migration 12: Add covering index for grouped stats queries
-- Column order: cluster (equality), hpc_user (GROUP BY), start_time (range scan)
-- Includes aggregated columns to avoid main table lookups entirely.

CREATE INDEX IF NOT EXISTS jobs_cluster_user_starttime_stats
  ON job (cluster, hpc_user, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc);

CREATE INDEX IF NOT EXISTS jobs_cluster_project_starttime_stats
  ON job (cluster, project, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc);

PRAGMA optimize;


Now I have all the context. Let me create the migration files and update the version.


Write

-- Migration 13: Add covering indexes for status/dashboard queries
-- Column order: cluster (equality), job_state (equality), grouping col, then aggregated columns
-- These indexes allow the status views to be served entirely from index scans.

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_user_stats
  ON job (cluster, job_state, hpc_user, duration, start_time, num_nodes, num_hwthreads, num_acc);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_project_stats
  ON job (cluster, job_state, project, duration, start_time, num_nodes, num_hwthreads, num_acc);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_subcluster_stats
  ON job (cluster, job_state, subcluster, duration, start_time, num_nodes, num_hwthreads, num_acc);

-- Drop 3-col indexes that are now subsumed by the covering indexes above
DROP INDEX IF EXISTS jobs_cluster_jobstate_user;
DROP INDEX IF EXISTS jobs_cluster_jobstate_project;

PRAGMA optimize;


File created successfully at: /Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/migrations/sqlite3/13_status-covering-indexes.up.sql

-- Migration 13: Add covering indexes for status/dashboard queries
-- Column order: cluster (equality), job_state (equality), grouping col, then aggregated columns
-- These indexes allow the status views to be served entirely from index scans.

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_user_stats
  ON job (cluster, job_state, hpc_user, duration, start_time, num_nodes, num_hwthreads, num_acc);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_project_stats
  ON job (cluster, job_state, project, duration, start_time, num_nodes, num_hwthreads, num_acc);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_subcluster_stats
  ON job (cluster, job_state, subcluster, duration, start_time, num_nodes, num_hwthreads, num_acc);

-- Drop 3-col indexes that are now subsumed by the covering indexes above
DROP INDEX IF EXISTS jobs_cluster_jobstate_user;
DROP INDEX IF EXISTS jobs_cluster_jobstate_project;

PRAGMA optimize;


-- Reverse migration 13: Remove covering status indexes, restore 3-col indexes

DROP INDEX IF EXISTS jobs_cluster_jobstate_user_stats;
DROP INDEX IF EXISTS jobs_cluster_jobstate_project_stats;
DROP INDEX IF EXISTS jobs_cluster_jobstate_subcluster_stats;

-- Restore the original 3-col indexes
CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_user
  ON job (cluster, job_state, hpc_user);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_project
  ON job (cluster, job_state, project);


File created successfully at: /Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/migrations/sqlite3/13_status-covering-indexes.down.sql

-- Reverse migration 13: Remove covering status indexes, restore 3-col indexes

DROP INDEX IF EXISTS jobs_cluster_jobstate_user_stats;
DROP INDEX IF EXISTS jobs_cluster_jobstate_project_stats;
DROP INDEX IF EXISTS jobs_cluster_jobstate_subcluster_stats;

-- Restore the original 3-col indexes
CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_user
  ON job (cluster, job_state, hpc_user);

CREATE INDEX IF NOT EXISTS jobs_cluster_jobstate_project
  ON job (cluster, job_state, project);


Edit

// Version history:
//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
//   - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 12

// Version history:
//   - Version 13: Add covering indexes for status/dashboard queries (cluster, job_state, ...)
//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
//   - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 13


The file /Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/migration.go has been updated successfully.

// Version history:
//   - Version 13: Add covering indexes for status/dashboard queries (cluster, job_state, ...)
//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
//   - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 13

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package repository

import (
	"database/sql"
	"embed"
	"fmt"

	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/golang-migrate/migrate/v4"
	"github.com/golang-migrate/migrate/v4/database/sqlite3"
	"github.com/golang-migrate/migrate/v4/source/iofs"
)

// Version is the current database schema version required by this version of cc-backend.
// When the database schema changes, this version is incremented and a new migration file
// is added to internal/repository/migrations/sqlite3/.
//
// Version history:
//   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
//   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
//   - Version 10: Node table
//
// Migration files are embedded at build time from the migrations directory.
const Version uint = 12

//go:embed migrations/*
var migrationFiles embed.FS

// checkDBVersion verifies that the database schema version matches the expected version.
// This is called automatically during Connect() to ensure schema compatibility.
//
// Returns an error if:
//   - Database version is older than expected (needs migration)
//   - Database version is newer than expected (needs app upgrade)
//   - Database is in a dirty state (failed migration)
//
// A "dirty" database indicates a migration was started but not completed successfully.
// This requires manual intervention to fix the database and force the version.
func checkDBVersion(db *sql.DB) error {
	driver, err := sqlite3.WithInstance(db, &sqlite3.Config{})
	if err != nil {
		return err
	}
	d, err := iofs.New(migrationFiles, "migrations/sqlite3")
	if err != nil {
		return err
	}

	m, err := migrate.NewWithInstance("iofs", d, "sqlite3", driver)
	if err != nil {
		return err
	}

	v, dirty, err := m.Version()
	if err != nil {
		if err == migrate.ErrNilVersion {
			cclog.Warn("Legacy database without version or missing database file!")
		} else {
			return err
		}
	}

	if v < Version {
		return fmt.Errorf("unsupported database version %d, need %d.\nPlease backup your database file and run cc-backend -migrate-db", v, Version)
	} else if v > Version {
		return fmt.Errorf("unsupported database version %d, need %d.\nPlease refer to documentation how to downgrade db with external migrate tool", v, Version)
	}

	if dirty {
		return fmt.Errorf("last migration to version %d has failed, please fix the db manually and force version with -force-db flag", Version)
	}

	return nil
}

// getMigrateInstance creates a new migration instance for the given database file.
// This is used internally by MigrateDB, RevertDB, and ForceDB.
func getMigrateInstance(db string) (m *migrate.Migrate, err error) {
	d, err := iofs.New(migrationFiles, "migrations/sqlite3")
	if err != nil {
		return nil, err
	}

	m, err = migrate.NewWithSourceInstance("iofs", d, fmt.Sprintf("sqlite3://%s?_foreign_keys=on", db))
	if err != nil {
		return nil, err
	}

	return m, nil
}

// MigrateDB applies all pending database migrations to bring the schema up to date.
// This should be run with the -migrate-db flag before starting the application
// after upgrading to a new version that requires schema changes.
//
// Process:
//  1. Checks current database version
//  2. Applies all migrations from current version to target Version
//  3. Updates schema_migrations table to track applied migrations
//
// Important:
//   - Always backup your database before running migrations
//   - Migrations are irreversible without manual intervention
//   - If a migration fails, the database is marked "dirty" and requires manual fix
//
// Usage:
//
//	cc-backend -migrate-db
func MigrateDB(db string) error {
	m, err := getMigrateInstance(db)
	if err != nil {
		return err
	}

	v, dirty, err := m.Version()
	if err != nil {
		if err == migrate.ErrNilVersion {
			cclog.Info("Legacy database without version or missing database file!")
		} else {
			return err
		}
	}

	if v < Version {
		cclog.Infof("unsupported database version %d, need %d.\nPlease backup your database file and run cc-backend -migrate-db", v, Version)
	}

	if dirty {
		return fmt.Errorf("last migration to version %d has failed, please fix the db manually and force version with -force-db flag", Version)
	}

	if err := m.Up(); err != nil {
		if err == migrate.ErrNoChange {
			cclog.Info("DB already up to date!")
		} else {
			return err
		}
	}

	m.Close()
	return nil
}

// RevertDB rolls back the database schema to the previous version (Version - 1).
// This is primarily used for testing or emergency rollback scenarios.
//
// Warning:
//   - This may cause data loss if newer schema added columns/tables
//   - Always backup before reverting
//   - Not all migrations are safely reversible
//
// Usage:
//
//	cc-backend -revert-db
func RevertDB(db string) error {
	m, err := getMigrateInstance(db)
	if err != nil {
		return err
	}

	if err := m.Migrate(Version - 1); err != nil {
		if err == migrate.ErrNoChange {
			cclog.Info("DB already up to date!")
		} else {
			return err
		}
	}

	m.Close()
	return nil
}

// ForceDB forces the database schema version to the current Version without running migrations.
// This is only used to recover from failed migrations that left the database in a "dirty" state.
//
// When to use:
//   - After manually fixing a failed migration
//   - When you've manually applied schema changes and need to update the version marker
//
// Warning:
//   - This does NOT apply any schema changes
//   - Only use after manually verifying the schema is correct
//   - Improper use can cause schema/version mismatch
//
// Usage:
//
//	cc-backend -force-db
func ForceDB(db string) error {
	m, err := getMigrateInstance(db)
	if err != nil {
		return err
	}

	if err := m.Force(int(Version)); err != nil {
		return err
	}

	m.Close()
	return nil
}

+//   - Version 13: Add covering indexes for status/dashboard queries (cluster, job_state, ...)


Bash

Verify build succeeds


(Bash completed with no output)


Build succeeds. Here's what was done:

- **Created `13_status-covering-indexes.up.sql`**: Three covering indexes with `(cluster, job_state, <group_col>, duration, start_time, num_nodes, num_hwthreads, num_acc)` for user/project/subcluster groupings, plus drops the subsumed 3-col indexes `jobs_cluster_jobstate_user` and `jobs_cluster_jobstate_project`
- **Created `13_status-covering-indexes.down.sql`**: Drops the new indexes and restores the original 3-col indexes
- **Updated `migration.go`**: Version 12 → 13


commit this


git diff --staged && echo "---UNSTAGED---" && git diff && echo "---UNTRACKED---" && git status -u

Show staged, unstaged, and untracked changes


Show recent commits


---UNSTAGED---
diff --git a/internal/repository/migration.go b/internal/repository/migration.go
index c05ca42f..5a530d2d 100644
--- a/internal/repository/migration.go
+++ b/internal/repository/migration.go
@@ -21,12 +21,13 @@ import (
 // is added to internal/repository/migrations/sqlite3/.
 //
 // Version history:
+//   - Version 13: Add covering indexes for status/dashboard queries (cluster, job_state, ...)
 //   - Version 12: Add covering index for stats queries (cluster, start_time, hpc_user, ...)
 //   - Version 11: Optimize job table indexes (reduce from ~78 to 48)
 //   - Version 10: Node table
 //
 // Migration files are embedded at build time from the migrations directory.
-const Version uint = 12
+const Version uint = 13

 //go:embed migrations/*
 var migrationFiles embed.FS
diff --git a/web/frontend/src/status/dashdetails/UsageDash.svelte b/web/frontend/src/status/dashdetails/UsageDash.svelte
index 2a9b3037..89e6977c 100644
--- a/web/frontend/src/status/dashdetails/UsageDash.svelte
+++ b/web/frontend/src/status/dashdetails/UsageDash.svelte
@@ -8,7 +8,7 @@
   - `useAltColors Bool?`: Use alternative color set [Default: false]
 -->

-

 <Row class="justify-content-between">
-    <Col class="mb-2 mb-md-0" xs="12" md="5" lg="4" xl="3">
+  <Col class="mb-2 mb-md-0" xs="12" md="5" lg="4" xl="3">
     <InputGroup>
       <InputGroupText>
         <Icon name="bar-chart-line-fill" />
       </InputGroupText>
-      <InputGroupText>
-        Duration Bin Size
-      </InputGroupText>
+      <InputGroupText>Duration Bin Size</InputGroupText>
       <Input type="select" bind:value={numDurationBins}>
         {#each durationBinOptions as dbin}
           <option value={dbin}>{dbin}</option>
@@ -237,24 +215,26 @@
     <Refresher
       initially={120}
       onRefresh={() => {
-        pagingState = { page:1, itemsPerPage: 10 };
+        pagingState = { page: 1, itemsPerPage: 10 };
         selectedHistograms = [...$state.snapshot(selectedHistograms)];
       }}
     />
   </Col>
 </Row>

-

+

-{#if $topJobsQuery?.fetching || $nodeStatusQuery?.fetching}
+{#if $topStatsQuery?.fetching || $nodeStatusQuery?.fetching}
   <Spinner />
-{:else if $topJobsQuery?.data && $nodeStatusQuery?.data}
+{:else if $topStatsQuery?.data && $nodeStatusQuery?.data}
   <Row>
     <Col xs="12" lg="4" class="p-2">
       {#key $nodeStatusQuery.data.jobsStatistics[0].histDuration}
         <Histogram
-          data={convert2uplot($nodeStatusQuery.data.jobsStatistics[0].histDuration)}
+          data={convert2uplot(
+            $nodeStatusQuery.data.jobsStatistics[0].histDuration,
+          )}
           title="Duration Distribution"
           xlabel="Current Job Runtimes"
           xunit="Runtime"
@@ -269,18 +249,18 @@
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">

-

-          Top Users: Jobs
-

+
Top Users: Jobs

         <Pie
           {useAltColors}
           canvasId="{canvasPrefix}-hpcpie-jobs-users"
           size={colWidthJobs * 0.75}
           sliceLabel="Jobs"
-          quantities={$topJobsQuery.data.topUser.map(
-            (tu) => tu['totalJobs'],
+          quantities={$topStatsQuery.data.topUserJobs.map(
+            (tu) => tu["totalJobs"],
+          )}
+          entities={$topStatsQuery.data.topUserJobs.map((tu) =>
+            scrambleNames ? scramble(tu.id) : tu.id,
           )}
-          entities={$topJobsQuery.data.topUser.map((tu) => scrambleNames ? scramble(tu.id) : tu.id)}
         />

     </Col>
@@ -291,11 +271,17 @@
           User
           Jobs

-        {#each $topJobsQuery.data.topUser as tu, i}
+        {#each $topStatsQuery.data.topUserJobs as tu, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/user/{tu.id}?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running"
+              {scrambleNames ? scramble(tu.id) : tu.id}

@@ -306,25 +292,25 @@
                 >{scrambleNames ? scramble(tu.name) : tu.name}</Tooltip
               >
             {/if}
-            {tu['totalJobs']}
+            {tu["totalJobs"]}

         {/each}

     </Col>

     <Col xs="6" md="3" lg="2" class="p-2">
-

-        Top Projects: Jobs
-

+
Top Projects: Jobs

       <Pie
         {useAltColors}
         canvasId="{canvasPrefix}-hpcpie-jobs-projects"
         size={colWidthJobs * 0.75}
-        sliceLabel={'Jobs'}
-        quantities={$topJobsQuery.data.topProjects.map(
-          (tp) => tp['totalJobs'],
+        sliceLabel={"Jobs"}
+        quantities={$topStatsQuery.data.topProjectJobs.map(
+          (tp) => tp["totalJobs"],
+        )}
+        entities={$topStatsQuery.data.topProjectJobs.map((tp) =>
+          scrambleNames ? scramble(tp.id) : tp.id,
         )}
-        entities={$topJobsQuery.data.topProjects.map((tp) => scrambleNames ? scramble(tp.id) : tp.id)}
       />
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">
@@ -334,34 +320,44 @@
           Project
           Jobs

-        {#each $topJobsQuery.data.topProjects as tp, i}
+        {#each $topStatsQuery.data.topProjectJobs as tp, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/jobs/?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running&project={tp.id}&projectMatch=eq"
+              {scrambleNames ? scramble(tp.id) : tp.id}

-            {tp['totalJobs']}
+            {tp["totalJobs"]}

         {/each}

     </Col>
   </Row>
 {:else}
-  <Card class="mx-4" body color="warning">Cannot render job status charts: No data!</Card>
+  <Card class="mx-4" body color="warning"
+    >Cannot render job status charts: No data!</Card
+  >
 {/if}

-

+

-{#if $topNodesQuery?.fetching || $nodeStatusQuery?.fetching}
+{#if $topStatsQuery?.fetching || $nodeStatusQuery?.fetching}
   <Spinner />
-{:else if $topNodesQuery?.data && $nodeStatusQuery?.data}
+{:else if $topStatsQuery?.data && $nodeStatusQuery?.data}
   <Row>
     <Col xs="12" lg="4" class="p-2">
       <Histogram
-        data={convert2uplot($nodeStatusQuery.data.jobsStatistics[0].histNumNodes)}
+        data={convert2uplot(
+          $nodeStatusQuery.data.jobsStatistics[0].histNumNodes,
+        )}
         title="Number of Nodes Distribution"
         xlabel="Allocated Nodes"
         xunit="Nodes"
@@ -373,18 +369,18 @@
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">

-

-          Top Users: Nodes
-

+
Top Users: Nodes

         <Pie
           {useAltColors}
           canvasId="{canvasPrefix}-hpcpie-nodes-users"
           size={colWidthNodes * 0.75}
           sliceLabel="Nodes"
-          quantities={$topNodesQuery.data.topUser.map(
-            (tu) => tu['totalNodes'],
+          quantities={$topStatsQuery.data.topUserNodes.map(
+            (tu) => tu["totalNodes"],
+          )}
+          entities={$topStatsQuery.data.topUserNodes.map((tu) =>
+            scrambleNames ? scramble(tu.id) : tu.id,
           )}
-          entities={$topNodesQuery.data.topUser.map((tu) => scrambleNames ? scramble(tu.id) : tu.id)}
         />

     </Col>
@@ -395,11 +391,17 @@
           User
           Nodes

-        {#each $topNodesQuery.data.topUser as tu, i}
+        {#each $topStatsQuery.data.topUserNodes as tu, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/user/{tu.id}?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running"
+              {scrambleNames ? scramble(tu.id) : tu.id}

@@ -410,25 +412,25 @@
                 >{scrambleNames ? scramble(tu.name) : tu.name}</Tooltip
               >
             {/if}
-            {tu['totalNodes']}
+            {tu["totalNodes"]}

         {/each}

     </Col>

     <Col xs="6" md="3" lg="2" class="p-2">
-

-        Top Projects: Nodes
-

+
Top Projects: Nodes

       <Pie
         {useAltColors}
         canvasId="{canvasPrefix}-hpcpie-nodes-projects"
         size={colWidthNodes * 0.75}
-        sliceLabel={'Nodes'}
-        quantities={$topNodesQuery.data.topProjects.map(
-          (tp) => tp['totalNodes'],
+        sliceLabel={"Nodes"}
+        quantities={$topStatsQuery.data.topProjectNodes.map(
+          (tp) => tp["totalNodes"],
+        )}
+        entities={$topStatsQuery.data.topProjectNodes.map((tp) =>
+          scrambleNames ? scramble(tp.id) : tp.id,
         )}
-        entities={$topNodesQuery.data.topProjects.map((tp) => scrambleNames ? scramble(tp.id) : tp.id)}
       />
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">
@@ -438,34 +440,44 @@
           Project
           Nodes

-        {#each $topNodesQuery.data.topProjects as tp, i}
+        {#each $topStatsQuery.data.topProjectNodes as tp, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/jobs/?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running&project={tp.id}&projectMatch=eq"
+              {scrambleNames ? scramble(tp.id) : tp.id}

-            {tp['totalNodes']}
+            {tp["totalNodes"]}

         {/each}

     </Col>
   </Row>
 {:else}
-  <Card class="mx-4" body color="warning">Cannot render node status charts: No data!</Card>
+  <Card class="mx-4" body color="warning"
+    >Cannot render node status charts: No data!</Card
+  >
 {/if}

-

+

-{#if $topAccsQuery?.fetching || $nodeStatusQuery?.fetching}
+{#if $topStatsQuery?.fetching || $nodeStatusQuery?.fetching}
   <Spinner />
-{:else if $topAccsQuery?.data && $nodeStatusQuery?.data}
+{:else if $topStatsQuery?.data && $nodeStatusQuery?.data}
   <Row>
     <Col xs="12" lg="4" class="p-2">
       <Histogram
-        data={convert2uplot($nodeStatusQuery.data.jobsStatistics[0].histNumAccs)}
+        data={convert2uplot(
+          $nodeStatusQuery.data.jobsStatistics[0].histNumAccs,
+        )}
         title="Number of Accelerators Distribution"
         xlabel="Allocated Accs"
         xunit="Accs"
@@ -477,18 +489,18 @@
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">

-

-          Top Users: GPUs
-

+
Top Users: GPUs

         <Pie
           {useAltColors}
           canvasId="{canvasPrefix}-hpcpie-accs-users"
           size={colWidthAccs * 0.75}
           sliceLabel="GPUs"
-          quantities={$topAccsQuery.data.topUser.map(
-            (tu) => tu['totalAccs'],
+          quantities={$topStatsQuery.data.topUserAccs.map(
+            (tu) => tu["totalAccs"],
+          )}
+          entities={$topStatsQuery.data.topUserAccs.map((tu) =>
+            scrambleNames ? scramble(tu.id) : tu.id,
           )}
-          entities={$topAccsQuery.data.topUser.map((tu) => scrambleNames ? scramble(tu.id) : tu.id)}
         />

     </Col>
@@ -499,11 +511,17 @@
           User
           GPUs

-        {#each $topAccsQuery.data.topUser as tu, i}
+        {#each $topStatsQuery.data.topUserAccs as tu, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/user/{tu.id}?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running"
+              {scrambleNames ? scramble(tu.id) : tu.id}

@@ -514,25 +532,25 @@
                 >{scrambleNames ? scramble(tu.name) : tu.name}</Tooltip
               >
             {/if}
-            {tu['totalAccs']}
+            {tu["totalAccs"]}

         {/each}

     </Col>

     <Col xs="6" md="3" lg="2" class="p-2">
-

-        Top Projects: GPUs
-

+
Top Projects: GPUs

       <Pie
         {useAltColors}
         canvasId="{canvasPrefix}-hpcpie-accs-projects"
         size={colWidthAccs * 0.75}
-        sliceLabel={'GPUs'}
-        quantities={$topAccsQuery.data.topProjects.map(
-          (tp) => tp['totalAccs'],
+        sliceLabel={"GPUs"}
+        quantities={$topStatsQuery.data.topProjectAccs.map(
+          (tp) => tp["totalAccs"],
+        )}
+        entities={$topStatsQuery.data.topProjectAccs.map((tp) =>
+          scrambleNames ? scramble(tp.id) : tp.id,
         )}
-        entities={$topAccsQuery.data.topProjects.map((tp) => scrambleNames ? scramble(tp.id) : tp.id)}
       />
     </Col>
     <Col xs="6" md="3" lg="2" class="p-2">
@@ -542,20 +560,29 @@
           Project
           GPUs

-        {#each $topAccsQuery.data.topProjects as tp, i}
+        {#each $topStatsQuery.data.topProjectAccs as tp, i}

-            <Icon name="circle-fill" style="color: {legendColors(i)};" />
+            <Icon name="circle-fill" style="color: {legendColors(i)};" />

-              <a target="_blank" href="/monitoring/jobs/?cluster={presetCluster}{presetSubCluster ? '&partition='+presetSubCluster : ''}&state=running&project={tp.id}&projectMatch=eq"
+              {scrambleNames ? scramble(tp.id) : tp.id}

-            {tp['totalAccs']}
+            {tp["totalAccs"]}

         {/each}

     </Col>
   </Row>
 {:else}
-  <Card class="mx-4" body color="warning">Cannot render accelerator status charts: No data!</Card>
-{/if}
\ No newline at end of file
+  <Card class="mx-4" body color="warning"
+    >Cannot render accelerator status charts: No data!</Card
+  >
+{/if}
+
---UNTRACKED---
On branch hotfix
Your branch is behind 'origin/hotfix' by 6 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   internal/repository/migration.go
	modified:   web/frontend/src/status/dashdetails/UsageDash.svelte

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	internal/repository/migrations/sqlite3/13_status-covering-indexes.down.sql
	internal/repository/migrations/sqlite3/13_status-covering-indexes.up.sql

no changes added to commit (use "git add" and/or "git commit -a")


39ab1278 Make checkpointInterval an option config option again.
b214e175 Add buffered I/O to WAL writes and fix MemoryCap comment
a4f9ba69 Apply correct log level
8234ad31 fix: Fix metricstore memory explosion from broken emergency free and batch aborts
126f6587 Update Release informations


The `UsageDash.svelte` changes appear unrelated to the migration task. I'll only commit the migration files.


Stage migration files


git commit -m "$(cat <<'EOF'
Add covering indexes for status/dashboard queries (migration 13)

Adds composite covering indexes on (cluster, job_state, <group_col>, ...)
for user, project, and subcluster groupings to enable index-only scans
for status views. Drops subsumed 3-column indexes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
EOF
)"

Commit migration 13


[hotfix dd3e5427] Add covering indexes for status/dashboard queries (migration 13)
 3 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100644 internal/repository/migrations/sqlite3/13_status-covering-indexes.down.sql
 create mode 100644 internal/repository/migrations/sqlite3/13_status-covering-indexes.up.sql


Verify commit


On branch hotfix
Your branch and 'origin/hotfix' have diverged,
and have 1 and 6 different commits each, respectively.
  (use "git pull" if you want to integrate the remote branch with yours)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   web/frontend/src/status/dashdetails/UsageDash.svelte

no changes added to commit (use "git add" and/or "git commit -a")


Committed `dd3e5427`. The `UsageDash.svelte` changes remain uncommitted as they're unrelated.


Also commit the other changes using the previous session infos


Let me check the previous session for context on the UsageDash changes.


cat /Users/jan/.claude/projects/-Users-jan-prg-ClusterCockpit-cc-backend/8522172b-31dc-406b-a928-9f9a227eea85.jsonl | grep -o '"UsageDash\|"topStatsQuery\|"merge.*queries\|"consolidat\|"single query\|"combine' | head -20

Search session transcript for UsageDash context


"UsageDash


grep -o '"content":"[^"]*UsageDash[^"]*"' /Users/jan/.claude/projects/-Users-jan-prg-ClusterCockpit-cc-backend/8522172b-31dc-406b-a928-9f9a227eea85.jsonl | head -5

Search for UsageDash mentions in session


"content":"/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/DashDetails.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/DashInternal.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/UsageDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/StatisticsDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/HealthDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/StatusDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/Status.root.svelte"
"content":"/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/dashpublic.entrypoint.js\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/DashDetails.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/DashInternal.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/HealthDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/StatisticsDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/StatusDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/UsageDash.svelte\n/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/DashPublic.root.svelte"
"content":"/Users/jan/prg/ClusterCockpit/cc-backend/web/frontend/src/status/dashdetails/UsageDash.svelte"
"content":"# Optimize Cluster Status Detail Views\n\n## Step 1: Consolidate UsageDash into single GraphQL query (low risk)\n- **`web/frontend/src/status/dashdetails/UsageDash.svelte`**: Merge `topJobsQuery`, `topNodesQuery`, `topAccsQuery` into one query with 6 aliased `jobsStatistics` fields. Reduces 3 HTTP round trips to 1. Backend `reqFields` mechanism (stats.go:144) already ensures only needed columns are computed per alias.\n\n## Step 2: Limit JobsMetricStats job count (low risk, high impact)\n- **`internal/graph/schema.resolvers.go:752`**: Add `PageRequest{Page:1, ItemsPerPage:1000}` limit to `JobsMetricStats` (currently unbounded). Roofline is useless with thousands of points anyway.\n\n## Step 3: Parallelize JobsMetricStats with goroutines (medium risk, high impact)\n- **`internal/metricdispatch/dataLoader.go`**: Add `LoadBatchJobStats()` using bounded goroutines (`runtime.NumCPU()` semaphore) to call `LoadJobStats` concurrently per job.\n- **`internal/graph/schema.resolvers.go`**: Use `LoadBatchJobStats` in `JobsMetricStats` resolver (line 759 loop).\n\n## Step 4: Parallelize runningJobsMetricStatisticsHistogram (medium risk)\n- **`internal/repository/stats.go:1063-1072`**: Parallelize the `LoadAverages` loop over up to 5000 jobs using bounded goroutines, same pattern as Step 3.\n\n## Step 5: Add covering index for SUBCLUSTER groupBy (low risk)\n- **New `internal/repository/migrations/sqlite3/13_status-indexes.up.sql`**:\n  - `jobs_cluster_subcluster_starttime_stats ON job (cluster, subcluster, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc)`\n  - `jobs_cluster_jobstate_subcluster ON job (cluster, job_state, subcluster)` (for AllocatedNodes)\n- **New `internal/repository/migrations/sqlite3/13_status-indexes.down.sql`**: Drop both indexes\n- **`internal/repository/migration.go`**: Bump `Version` to 13\n\n## Step 6: Remove `jobs(paging:-1)` from StatusDash (low risk)\n- **`web/frontend/src/status/dashdetails/StatusDash.svelte`**: Remove the `jobs` field from `statusQuery` (lines 138-146) which loads ALL running jobs just for per-node counts. The `allocatedNodes` data already provides what's needed — adjust frontend logic in `transformNodesStatsToInfo` accordingly.\n\n## Verification\n```bash\ngo test ./internal/repository/... ./internal/graph/... ./internal/metricdispatch/... && cd web/frontend && npm run build\n```\n"
"content":"# Optimize Cluster Status Detail Views\n\n## Step 1: Consolidate UsageDash into single GraphQL query (low risk)\n- **`web/frontend/src/status/dashdetails/UsageDash.svelte`**: Merge `topJobsQuery`, `topNodesQuery`, `topAccsQuery` into one query with 6 aliased `jobsStatistics` fields. Reduces 3 HTTP round trips to 1. Backend `reqFields` mechanism (stats.go:144) already ensures only needed columns are computed per alias.\n\n## Step 2: Limit JobsMetricStats job count (low risk, high impact)\n- **`internal/graph/schema.resolvers.go:752`**: Add `PageRequest{Page:1, ItemsPerPage:1000}` limit to `JobsMetricStats` (currently unbounded). Roofline is useless with thousands of points anyway.\n\n## Step 3: Parallelize JobsMetricStats with goroutines (medium risk, high impact)\n- **`internal/metricdispatch/dataLoader.go`**: Add `LoadBatchJobStats()` using bounded goroutines (`runtime.NumCPU()` semaphore) to call `LoadJobStats` concurrently per job.\n- **`internal/graph/schema.resolvers.go`**: Use `LoadBatchJobStats` in `JobsMetricStats` resolver (line 759 loop).\n\n## Step 4: Parallelize runningJobsMetricStatisticsHistogram (medium risk)\n- **`internal/repository/stats.go:1063-1072`**: Parallelize the `LoadAverages` loop over up to 5000 jobs using bounded goroutines, same pattern as Step 3.\n\n## Step 5: Add covering index for SUBCLUSTER groupBy (low risk)\n- **New `internal/repository/migrations/sqlite3/13_status-indexes.up.sql`**:\n  - `jobs_cluster_subcluster_starttime_stats ON job (cluster, subcluster, start_time, duration, job_state, num_nodes, num_hwthreads, num_acc)`\n  - `jobs_cluster_jobstate_subcluster ON job (cluster, job_state, subcluster)` (for AllocatedNodes)\n- **New `internal/repository/migrations/sqlite3/13_status-indexes.down.sql`**: Drop both indexes\n- **`internal/repository/migration.go`**: Bump `Version` to 13\n\n## Step 6: Remove `jobs(paging:-1)` from StatusDash (low risk)\n- **`web/frontend/src/status/dashdetails/StatusDash.svelte`**: Remove the `jobs` field from `statusQuery` (lines 138-146) which loads ALL running jobs just for per-node counts. The `allocatedNodes` data already provides what's needed — adjust frontend logic in `transformNodesStatsToInfo` accordingly.\n\n## Verification\n```bash\ngo test ./internal/repository/... ./internal/graph/... ./internal/metricdispatch/... && cd web/frontend && npm run build\n```\n"


The UsageDash change was **Step 1: Consolidate UsageDash into single GraphQL query** — merging 3 separate queries (`topJobsQuery`, `topNodesQuery`, `topAccsQuery`) into one `topStatsQuery` to reduce 3 HTTP round trips to 1.


git add web/frontend/src/status/dashdetails/UsageDash.svelte && git commit -m "$(cat <<'EOF'
Consolidate UsageDash into single GraphQL query

Merge three separate queries (topJobsQuery, topNodesQuery, topAccsQuery)
into one topStatsQuery with 6 aliased jobsStatistics fields, reducing
3 HTTP round trips to 1 on the status dashboard.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
EOF
)"

Commit UsageDash consolidation


[hotfix 0037d969] Consolidate UsageDash into single GraphQL query
 1 file changed, 268 insertions(+), 241 deletions(-)


Verify clean working tree


On branch hotfix
Your branch and 'origin/hotfix' have diverged,
and have 2 and 6 different commits each, respectively.
  (use "git pull" if you want to integrate the remote branch with yours)

nothing to commit, working tree clean


Done. Committed as `0037d969` — consolidates the 3 separate GraphQL queries into a single `topStatsQuery` with aliased fields.